Pangenome (hosting your own graph)

JBrowse v5 beta

The tutorials target the JBrowse v5 beta, and the v4.3.0 release on the download page lacks some of what they show. To install the beta of JBrowse Web, run npm install -g @jbrowse/cli@next, then jbrowse create jbrowse2 --branch v5.0.0-beta.21. Desktop beta builds for Linux and macOS are on the releases page; the Windows build waits on code signing.

A pangenome graph of many genomes is too large to load whole, so JBrowse browses one from small indexed files, a window at a time, from a whole chromosome down to single nodes. One command cuts a graph into those files and writes the config that puts them on tracks. We run it on HPRC release 2's human graph, with E. coli and Arabidopsis graphs for the steps whose input the human graph lacks. We:

install the GraphGenomeView plugin and index a graph with gfa-to-tabix build
load the graph as a graph track, and as a row of segments with Display types
add the tracks that record which haplotypes pass through each segment
index the haplotypes' walks and load them as a graph track with one lane per haplotype
convert the walks to PAF for a linear synteny view

Experimental

The graph view is a beta plugin. We welcome your feedback.

Prerequisites

the GraphGenomeView plugin
htslib (bgzip, tabix) and bcftools
gfa-to-tabix 4.0.0 or later, for the graph's files in format 2, which the configs below read, and the walk files (walk format 7, the only one the plugin reads)
gfatools, for an rGFA's bubbles
minigraph, for each assembly's path through the graph
vg, for a plain GFA's bubbles
for the walks as PAF: python3 and the JBrowse CLI for make-pif

# Debian/Ubuntu
sudo apt install tabix bcftools
# or conda, which has every tool above
conda install -c bioconda -c conda-forge htslib bcftools gfatools minigraph \
  vg
# any platform
cargo install gfa-to-tabix
npm install -g @jbrowse/cli@next

Download the data

The input is HPRC release 2's Minigraph-Cactus graphs. gfa-to-tabix build reads the rGFA, and the walk files come from the base-level GFAs, all as local files. Fetch the rGFA first and the base-level GFAs when you reach the walks:

B=https://s3-us-west-2.amazonaws.com/human-pangenomics/pangenomes/freeze/release2/minigraph-cactus/v2.1/hprc-v2.1-mc-grch38/hprc-v2.1-mc-grch38
# SV-resolution rGFA, 0.8 GB
wget -c $B.sv.gfa.gz
# base-level GFA with every haplotype's walk, 64 GB
wget -c $B.gfa.gz
C=https://s3-us-west-2.amazonaws.com/human-pangenomics/pangenomes/freeze/release2/minigraph-cactus/v2.1/hprc-v2.1-mc-chm13/hprc-v2.1-mc-chm13
# the same graph built on T2T-CHM13, for walks on hs1, 66 GB
wget -c $C.gfa.gz

Other files (no download needed)

What it is	File
The finished files, for comparison	README.txt

The GraphGenomeView plugin

Install GraphGenomeView, which needs JBrowse 5, from the plugin store, at Tools → Plugin store in JBrowse Web or Global plugins... on the JBrowse Desktop start screen (Installing plugins). The config the command writes names it with the plugins entry below, in which storePlugin picks the build for the JBrowse reading the config, and esmUrl is the fallback when the store cannot be read (configuring plugins).

{
  "plugins": [
    {
      "name": "GraphGenomeView",
      "storePlugin": "GraphGenomeView",
      "esmUrl": "https://jbrowse.org/plugins/jbrowse-plugin-graphgenomeviewer/latest/dist/jbrowse-plugin-graphgenomeviewer.esm.js"
    }
  ]
}

Indexing a graph with gfa-to-tabix build

The gfa-to-tabix build command writes every file below. Without a Rust toolchain, take a Linux or macOS binary from its releases.

The sections that apply depend on the file you have:

An rGFA gives the graph track, the bubbles and the coarse tier here, and with the assemblies, each haplotype's allele at every bubble.
A plain GFA gives the same tracks from a snarl VCF, and the haplotypes on each segment from its paths.
A base-level GFA with W lines also gives haplotype lanes and whole-genome synteny.
A GBZ converts to that GFA with vg convert -f.

An rGFA, from minigraph or the minigraph stage of Minigraph-Cactus, needs an output prefix and the assembly name your config uses. Its bubbles come from gfatools bubble, so put gfatools on PATH:

gfa-to-tabix build graph.rgfa.gz -o hprc --assembly hg38

A plain GFA, from pggb, odgi, vg or base-level Minigraph-Cactus, also needs the backbone sample and its bubbles. The bubbles come from a snarl VCF (vg's word for a bubble is snarl):

vg deconstruct -a writes one record per snarl against the reference path, tagged LV for its level in the snarl tree, and pggb -V writes the same file. gfa-to-tabix bubbles keeps the LV=0 records, so the tag has to be there.
The VCF's CHROM must be the assembly's refName, so rename the PanSN path (sample#haplotype#contig) as the pggb tutorial does:

# -p: the reference path to decompose against
vg deconstruct -a -p K12#1#chr graph.gfa > graph.snarls.vcf
printf 'K12#1#chr\tchr\n' > rename_chrs.tsv
bcftools annotate --rename-chrs rename_chrs.tsv graph.snarls.vcf \
  | bcftools sort -Oz -o graph.snarls.vcf.gz

gfa-to-tabix build graph.gfa -o ecoli --reference K12 --assembly K12 --snarls graph.snarls.vcf.gz

At human-chromosome scale, index the SV-resolution rGFA; a pggb graph's index does not finish there.

The command writes these files beside the prefix:

file	what it holds
`.graph.bed.gz`	one row per segment and one per link, filed under reference coordinates for an rGFA and under the segment's coordinate on its source contig for a plain GFA; a link row states both ends in full
`.bubbles.bed.gz`	where haplotypes diverge and rejoin, with each bubble's shortest and longest allele
`.fold10000.graph.bed.gz`	the graph with variants under 10 kb folded into the reference, so a whole chromosome draws; a plain GFA folds under 50 bp, as `.fold50.graph.bed.gz`
`.contig.graph.bed.gz`	an rGFA's segment and link rows under the segment's coordinate on its source contig
`.graph.json`	a manifest naming the files above and the tier's fold size
`.config.json`	the tracks below, with the plugin entry

Each .bed.gz gets a tabix index beside it. --format 1 writes the index, the tier and the contig layout as .segs.bed.gz and .links.bed.gz pairs, and a track reading such a pair leaves out "format": 2.

The coarse tier for whole-chromosome views

The coarse tier is a copy of the graph with every variant under a threshold folded into the reference. It keeps each allele whose length, or the reference it replaces, reaches the threshold, and the way back to the reference from both of its ends. The graph track folds what it draws the same way at ten of the linear view's pixels, so zooming out onto the tier keeps every loop the fine cut drew. The threshold is in the file name, 10,000 bp by default for an rGFA; --tier sets it, and a plain GFA defaults to 50, since most of its variants are single bases.

Configuring the graph track

The config's first track is the graph. Its adapter holds:

uri, the prefix, and "format": 2, which reads <prefix>.graph.bed.gz
coarse, which names the tier and its threshold, foldBelowBp; the track draws the tier past a tenth of that in bp per pixel
assemblyNameToPanSN, which maps your assembly name to the graph's PanSN sample; the command writes it only when the two differ

Config file

Goes in the tracks array of config.json. See Tracks.

{
  "type": "GraphTrack",
  "trackId": "hprc_graph",
  "name": "hprc graph",
  "assemblyNames": ["hg38"],
  "adapter": {
    "type": "RgfaTabixAdapter",
    "uri": "hprc",
    "format": 2,
    "assemblyNameToPanSN": { "hg38": "GRCh38" },
    "coarse": { "uri": "hprc.fold10000", "format": 2, "foldBelowBp": 10000 }
  },
  "displayDefaults": {
    "labels": { "name": { "show": false }, "description": { "show": false } }
  },
  "displays": [
    { "type": "LinearGraphDisplay" },
    { "type": "LinearBasicDisplay" }
  ]
}

CLI

Edits config.json in the current folder. See JBrowse CLI with Desktop.

jbrowse add-track-json '{
  "type": "GraphTrack",
  "trackId": "hprc_graph",
  "name": "hprc graph",
  "assemblyNames": ["hg38"],
  "adapter": {
    "type": "RgfaTabixAdapter",
    "uri": "hprc",
    "format": 2,
    "assemblyNameToPanSN": { "hg38": "GRCh38" },
    "coarse": { "uri": "hprc.fold10000", "format": 2, "foldBelowBp": 10000 }
  },
  "displayDefaults": {
    "labels": { "name": { "show": false }, "description": { "show": false } }
  },
  "displays": [
    { "type": "LinearGraphDisplay" },
    { "type": "LinearBasicDisplay" }
  ]
}'

In the app

In JBrowse Desktop, or in any running JBrowse Web session, open a view on this track’s assembly, then File → Open track..., choose Add track from pasted JSON, and paste:

{
  "type": "GraphTrack",
  "trackId": "hprc_graph",
  "name": "hprc graph",
  "assemblyNames": ["hg38"],
  "adapter": {
    "type": "RgfaTabixAdapter",
    "uri": "hprc",
    "format": 2,
    "assemblyNameToPanSN": { "hg38": "GRCh38" },
    "coarse": { "uri": "hprc.fold10000", "format": 2, "foldBelowBp": 10000 }
  },
  "displayDefaults": {
    "labels": { "name": { "show": false }, "description": { "show": false } }
  },
  "displays": [
    { "type": "LinearGraphDisplay" },
    { "type": "LinearBasicDisplay" }
  ]
}

hprc, hprc.fold10000 are relative to a config.json. Replace each with its URL or its path on this computer.

build writes the config as <prefix>.config.json, here hprc.config.json. Merge its plugins and tracks entries into your own config, then run jbrowse validate config.json, which reports a misspelled or undeclared slot JBrowse otherwise ignores. Without a config, Open track... → Add pangenome graph track reads hprc.graph.json beside hprc.graph.bed.gz, so a track opened from that url gets the coarse tier too.

A plain GFA's index and tier are in the contig layout, each segment under its coordinate on its source contig. The anchored layout an rGFA gets files a segment under every reference position its branch spans, which on the E. coli pggb and Minigraph-Cactus graphs makes the track draw multi-megabase rows across every window.

Turned on, the track draws as a graph. Display types → Feature display in its track menu draws the same segments as a row.

The HPRC graph's segment index over C4 on hg38.

The other two tracks in the config draw the bubbles as a row and as a curve. Turn them on and switch the graph back with Display types → Graph.

The three tracks the command writes, over the C4 region on hg38: the bubbles as a row and as a curve, and the graph track at the bottom.

A node's right-click menu offers Open in the haplotype its rGFA id names (NA20809#2#CM094351.1 opens NA20809#2), when the session holds an assembly named or aliased sample#haplotype. The HPRC tutorial takes that route.

Checking the index against the graph

Query a locus out of the index, by the graph's PanSN name for the reference contig:

tabix hprc.graph.bed.gz 'GRCh38#0#chr1:103,690,000-103,700,000' \
  | awk '$4 == "N"' | head -3

GRCh38#0#chr1	103668404	103713477	N	s653929	95	HG00323#2#CM087328.1	104135607	104135610
GRCh38#0#chr1	103668404	103713477	N	s653930	95	HG00323#2#CM087328.1	104135610	104144685
GRCh38#0#chr1	103668404	103713477	N	s663586	205	HG02723#2#JAHEOT020000031.1	103971922	104013350

The awk keeps the segment rows, which have N in column four where a link row has L. In a segment row, columns one to three give the reference interval the command filed it under, five the segment id, six its rank (build order), and seven to nine the segment's contig and span.

An empty result over a tiled reference means the query used the wrong name: chr1 finds nothing where GRCh38#0#chr1 finds every segment.
Backbone rows with no alleles mark a place where the graph collapsed, which minigraph does to near-identical segmental duplications.

Recording which haplotypes pass through each segment

The rank in an rGFA is build order, so it names the first assembly a segment came from. The full list of haplotypes whose paths pass through each segment comes from one of two sources.

With the assemblies, map each one back through the graph, reference first, and read the path it takes:

# --call: the path this sample takes through every bubble, one line per
#   `gfatools bubble` line, in the same order for every sample
# -xasm: the assembly-to-graph preset
# -c: base-level alignment, which --call reads
minigraph -cxasm --call -t 8 graph.rgfa.gz sample.fa > sample.call.bed

gfa-to-tabix paths joins the call files, the reference's first, into one tabix-indexed row per bubble per sample, drawn as one row per haplotype:

gfa-to-tabix paths -o graph.paths ref.call.bed sample.call.bed

The file loads as a feature track with one row per haplotype, partitioned on strain. The itemRgb column sets each block's color by class, so the track needs no color config:

Config file

Goes in the tracks array of config.json. See Tracks.

{
  "type": "FeatureTrack",
  "trackId": "graph_paths",
  "name": "graph: each haplotype's allele at every bubble",
  "assemblyNames": ["TAIR10"],
  "adapter": {
    "type": "BedTabixAdapter",
    "uri": "graph.paths.bed.gz",
    "disableGeneHeuristic": true
  },
  "displays": [
    {
      "type": "LinearMultiRowFeatureDisplay",
      "rows": { "field": "strain" }
    }
  ]
}

CLI

Edits config.json in the current folder. See JBrowse CLI with Desktop.

jbrowse add-track-json '{
  "type": "FeatureTrack",
  "trackId": "graph_paths",
  "name": "graph: each haplotype'\''s allele at every bubble",
  "assemblyNames": ["TAIR10"],
  "adapter": {
    "type": "BedTabixAdapter",
    "uri": "graph.paths.bed.gz",
    "disableGeneHeuristic": true
  },
  "displays": [
    {
      "type": "LinearMultiRowFeatureDisplay",
      "rows": { "field": "strain" }
    }
  ]
}'

In the app

In JBrowse Desktop, or in any running JBrowse Web session, open a view on this track’s assembly, then File → Open track..., choose Add track from pasted JSON, and paste:

{
  "type": "FeatureTrack",
  "trackId": "graph_paths",
  "name": "graph: each haplotype's allele at every bubble",
  "assemblyNames": ["TAIR10"],
  "adapter": {
    "type": "BedTabixAdapter",
    "uri": "graph.paths.bed.gz",
    "disableGeneHeuristic": true
  },
  "displays": [
    {
      "type": "LinearMultiRowFeatureDisplay",
      "rows": { "field": "strain" }
    }
  ]
}

graph.paths.bed.gz is relative to a config.json. Replace it with its URL or its path on this computer.

The arabidopsis pangenome config hosts one such file as arabidopsis_minigraph_paths. Open it over FLC at Chr5:3,170,000-3,186,000, under TAIR10_genes:

The 1001 Genomes+ minigraph graph's paths file over FLC on TAIR10: one row per accession, each block its allele at a bubble.

With a plain GFA, the command records the haplotypes whose paths pass through each segment while it reads the paths, as an SB:H: bitmask over the haplotypes the file's #samples header lists (SM:Z: names in format 1). The track below is written for a build like the pggb graph of five E. coli strains, indexed against K12.

The node panel lists them as samples; a track reads them as feature.samples and their count as feature.sampleCount.
Color by → Attribute... with sampleCount gives each count a separate color.
For more than a handful of haplotypes, Edit plot... in the same dialog takes a ramp, here red for a segment on one haplotype to grey for a segment on most:

Config file

Goes in the tracks array of config.json. See Tracks.

{
  "type": "FeatureTrack",
  "trackId": "graph_haplotypes_per_segment",
  "name": "graph: haplotypes per segment",
  "assemblyNames": ["K12"],
  "adapter": {
    "type": "RgfaTabixAdapter",
    "uri": "graph",
    "format": 2
  },
  "displayDefaults": {
    "encoding": {
      "color": {
        "field": "sampleCount",
        "scale": {
          "type": "linear",
          "domainMin": 1,
          "range": ["#e31a1c", "#bdbdbd"]
        },
        "legend": { "title": "Haplotypes" }
      }
    }
  }
}

CLI

Edits config.json in the current folder. See JBrowse CLI with Desktop.

jbrowse add-track-json '{
  "type": "FeatureTrack",
  "trackId": "graph_haplotypes_per_segment",
  "name": "graph: haplotypes per segment",
  "assemblyNames": ["K12"],
  "adapter": {
    "type": "RgfaTabixAdapter",
    "uri": "graph",
    "format": 2
  },
  "displayDefaults": {
    "encoding": {
      "color": {
        "field": "sampleCount",
        "scale": {
          "type": "linear",
          "domainMin": 1,
          "range": ["#e31a1c", "#bdbdbd"]
        },
        "legend": { "title": "Haplotypes" }
      }
    }
  }
}'

In the app

In JBrowse Desktop, or in any running JBrowse Web session, open a view on this track’s assembly, then File → Open track..., choose Add pangenome graph track, and fill in:

File type: Indexed GFA (gfa-to-tabix .segs.bed.gz or .graph.bed.gz)
Path: graph.graph.bed.gz
Track name: graph: haplotypes per segment
Assembly: K12

graph is relative to a config.json. Replace it with its URL or its path on this computer.

The count is per haplotype (HG002.1), so a diploid sample's two copies count separately.

The haplotypes-per-segment track over the pggb E. coli graph near insH21, red for a segment on one strain and grey for a segment on all five.

Indexing haplotype walks with gfa-to-tabix

A walk is one haplotype's route through the graph. gfa-to-tabix --walks writes every walk to one walk file per reference, which the graph track and the haplotype lanes read one range at a time:

each walk is cut into pieces, one for each stretch it spends in a 64 kb tile of the reference, and each piece is filed under its tile
a tile holds the reference's path through it once, each distinct route a haplotype takes off that path once, and every other walk's piece as the list of routes it takes
a departure record gives each place a walk leaves the reference and rejoins it in another piece, and is filed over the whole stretch between, so a window inside a deletion still reads the haplotype that skips it

The base-level GFA holds each walk as a W line. Index the GRCh38 graph against GRCh38:¹

# --refs: the reference sample whose coordinates the walk file is in
gfa-to-tabix hprc-v2.1-mc-grch38.gfa.gz --walks --refs GRCh38 \
  -o hprc-v2.1-mc-grch38

A Minigraph-Cactus graph's own reference passes no node twice, so its walk file places every step exactly. On another reference, a step on a node that reference passes more than once takes the pass the tool picks, so a track on T2T-CHM13 reads the walks of the graph built on CHM13, indexed the same way:

gfa-to-tabix hprc-v2.1-mc-chm13.gfa.gz --walks --refs CHM13 -o hprc-v2.1-mc-chm13

vg convert -f graph.gbz writes the same GFA for a graph published only as a GBZ.

The tool writes a walk file and its directory for each sample in --refs, named <prefix>.<sample>.walks.gz and <prefix>.<sample>.walks.gz.idx, so a track on GRCh38 reads files that cover GRCh38 alone:

file	what it holds
`hprc-v2.1-mc-grch38.GRCh38.walks.gz`	a header, then for each GRCh38 sequence its tiles, its departure records and a table of their byte ranges
`hprc-v2.1-mc-grch38.GRCh38.walks.gz.idx`	the byte range of the header and of each sequence's table
`hprc-v2.1-mc-chm13.CHM13.walks.gz`	the same for T2T-CHM13, with its own `.idx`

A browser reads the directory, then fetches the tiles and departure records over a window by range request. gzip -dc prints the whole file as text, header first:

gzip -dc hprc-v2.1-mc-grch38.GRCh38.walks.gz | head -4

#walks	format:i:7	tile:i:65536	revisited:i:0	stay:i:311	abridge:i:1000000	abridgebytes:i:3000000
#reference	GRCh38
#haplotype	CHM13#0
#haplotype	HG00097#1

#walks gives the file's format, the tile size, revisited, the number of nodes the reference passes more than once (0 when every step is placed exactly), and the --stay the tool ran with
abridge and abridgebytes are the --abridge-steps and --abridge-bytes budgets. Where a tile's walks cost more, usually in a satellite array, gfa-to-tabix moves the costliest walks to detail records that a window read skips
#reference and #haplotype list the reference sample, then every other haplotype with rows in the file, the other reference included

A track's walksUri names one reference's walk file by its prefix, <prefix>.<sample>, and reads <walksUri>.walks.gz with the .idx beside it. The track below draws the graph and, as a second display, one lane per haplotype. Each haplotype in assemblyNames is an assembly whose aliases include its sample#haplotype name, as the HPRC tutorial declares one; assemblyNameToPanSN covers the reference, which has none:

Config file

Goes in the tracks array of config.json. See Tracks.

{
  "type": "GraphTrack",
  "trackId": "my_graph_lanes",
  "name": "My graph: haplotypes vs the reference",
  "assemblyNames": ["hg38", "HG00097.1", "HG00099.1"],
  "adapter": {
    "type": "WalkTabixSyntenyAdapter",
    "walksUri": "https://example.com/graphs/my_graph.GRCh38",
    "assemblyNames": ["hg38"],
    "assemblyNameToPanSN": { "hg38": "GRCh38#0" }
  },
  "displays": [
    { "type": "MultiWaySyntenyDisplay", "height": 600 },
    { "type": "LinearGraphDisplay" }
  ]
}

CLI

Edits config.json in the current folder. See JBrowse CLI with Desktop.

jbrowse add-track-json '{
  "type": "GraphTrack",
  "trackId": "my_graph_lanes",
  "name": "My graph: haplotypes vs the reference",
  "assemblyNames": ["hg38", "HG00097.1", "HG00099.1"],
  "adapter": {
    "type": "WalkTabixSyntenyAdapter",
    "walksUri": "https://example.com/graphs/my_graph.GRCh38",
    "assemblyNames": ["hg38"],
    "assemblyNameToPanSN": { "hg38": "GRCh38#0" }
  },
  "displays": [
    { "type": "MultiWaySyntenyDisplay", "height": 600 },
    { "type": "LinearGraphDisplay" }
  ]
}'

In the app

In JBrowse Desktop, or in any running JBrowse Web session, open a view on this track’s assembly, then File → Open track..., choose Add track from pasted JSON, and paste:

{
  "type": "GraphTrack",
  "trackId": "my_graph_lanes",
  "name": "My graph: haplotypes vs the reference",
  "assemblyNames": ["hg38", "HG00097.1", "HG00099.1"],
  "adapter": {
    "type": "WalkTabixSyntenyAdapter",
    "walksUri": "https://example.com/graphs/my_graph.GRCh38",
    "assemblyNames": ["hg38"],
    "assemblyNameToPanSN": { "hg38": "GRCh38#0" }
  },
  "displays": [
    { "type": "MultiWaySyntenyDisplay", "height": 600 },
    { "type": "LinearGraphDisplay" }
  ]
}

The track aligns each walk to the reference walk at the nodes both pass through, so it needs no sequence beyond the walk file.

A graph track with no lanes names a walk file through RgfaTabixAdapter, here the CHM13 graph's:

Config file

Goes in the tracks array of config.json. See Tracks.

{
  "type": "GraphTrack",
  "trackId": "hprc_walks_chm13",
  "name": "HPRC v2.1 haplotype walks",
  "assemblyNames": ["hs1"],
  "adapter": {
    "type": "RgfaTabixAdapter",
    "walksUri": "hprc-v2.1-mc-chm13.CHM13",
    "assemblyNameToPanSN": { "hs1": "CHM13" },
    "defaultHaplotypes": [
      "HG00097#1",
      "HG00099#1",
      "HG00128#1",
      "HG00133#1",
      "HG01109#1",
      "HG01123#1",
      "HG01960#1",
      "HG02055#1"
    ]
  }
}

CLI

Edits config.json in the current folder. See JBrowse CLI with Desktop.

jbrowse add-track-json '{
  "type": "GraphTrack",
  "trackId": "hprc_walks_chm13",
  "name": "HPRC v2.1 haplotype walks",
  "assemblyNames": ["hs1"],
  "adapter": {
    "type": "RgfaTabixAdapter",
    "walksUri": "hprc-v2.1-mc-chm13.CHM13",
    "assemblyNameToPanSN": { "hs1": "CHM13" },
    "defaultHaplotypes": [
      "HG00097#1",
      "HG00099#1",
      "HG00128#1",
      "HG00133#1",
      "HG01109#1",
      "HG01123#1",
      "HG01960#1",
      "HG02055#1"
    ]
  }
}'

In the app

In JBrowse Desktop, or in any running JBrowse Web session, open a view on this track’s assembly, then File → Open track..., choose Add track from pasted JSON, and paste:

{
  "type": "GraphTrack",
  "trackId": "hprc_walks_chm13",
  "name": "HPRC v2.1 haplotype walks",
  "assemblyNames": ["hs1"],
  "adapter": {
    "type": "RgfaTabixAdapter",
    "walksUri": "hprc-v2.1-mc-chm13.CHM13",
    "assemblyNameToPanSN": { "hs1": "CHM13" },
    "defaultHaplotypes": [
      "HG00097#1",
      "HG00099#1",
      "HG00128#1",
      "HG00133#1",
      "HG01109#1",
      "HG01123#1",
      "HG01960#1",
      "HG02055#1"
    ]
  }
}

assemblyNameToPanSN maps the assembly name to the walk file's reference sample, here hs1 to CHM13
defaultHaplotypes lists the haplotypes the track draws until the reader picks others, as sample#haplotype, or a bare sample for both of its haplotypes

The HPRC tracks built this way open at chr22:20,000,000-20,100,000 on hs1, from the CHM13 graph's walk file and hg38, from the GRCh38 graph's walk file.

The track menu's Haplotypes submenu switches the track between The track's default haplotypes, Every haplotype in the graph and Chosen in Graph settings..., whose dialog offers the names from the walk file's header. A graph view opened from the track draws the same set. Where the graph collapses the copies of a repeat onto one set of nodes, the track draws a walk through them where the reference places those nodes.

Two budgets in the adapter's config set when a window is refused with a notice to zoom in:

walkByteBudget caps the compressed bytes a window would fetch from the walk file, which its directory gives before any row is read. A tile holds every haplotype's rows, so the bytes are the same for any set of haplotypes.
walkStepBudget caps the walk steps a window keeps, counting only the haplotypes the track draws, so with fewer haplotypes the track can draw a wider window.

Whole-genome synteny from the walks, as PAF

The track reads the walk files one window at a time. For a whole-genome synteny view, a GFA's walks also convert to PAF against the reference walk, one record per stretch of nodes the two walks share. Convert the walks you want:

curl -fO https://raw.githubusercontent.com/cmdcolin/gfa-to-pairwise-paf/v1.0.0/gfa_to_pairwise_paf.py
# --contig-lengths: each assembly's .fai, since walks omit contig lengths
gzip -dc graph.gfa.gz | python3 gfa_to_pairwise_paf.py --reference GRCh38#0 \
  --queries HG01109#1,HG00099#1 --contig-lengths contigs.fai > graph.paf

Index the PAF for JBrowse:

jbrowse make-pif graph.paf --csi --out graph.pif.gz

build_hprc_multiway_synteny.sh runs both on the release graph for eight haplotypes, with each haplotype's contig lengths and CAT genes, and its output is hosted, one lane per haplotype under GRCh38.

Reproduce it end to end

The one command builds the graph track, the bubbles and the tier, with the tools under Prerequisites. It:

places every segment on a genome: an rGFA holds each position in tags on the segment, and for a plain GFA the command follows the backbone's paths first, then places each remaining segment on the first other haplotype whose path passes through it
finds the bubbles, with gfatools bubble on an rGFA or from the snarl VCF on a plain GFA, and builds the tier from them
writes the manifest and the config that name those files

gfa-to-tabix build hprc-v2.1-mc-grch38.sv.gfa.gz -o hprc --assembly hg38

To rebuild the coarse tier alone, at another threshold, run gfa-to-tabix fold:

gfa-to-tabix fold hprc-v2.1-mc-grch38.sv.gfa.gz --below 10000 -o hprc.fold10000

The walk files come from the commands under Indexing haplotype walks.

External links

Li H. The rGFA format and gfatools: the SN/SO/SR tags and the bubble calls.
gfa-to-tabix: --walks, and the rows it writes.
gfa-to-tabix and gbz-base compared: what each holds, sizes and requests on HPRC v2.1.
ggbandage and ggtubemap: R packages that draw a region of a walk file as ggplot2 layers.

Citations

HPRC release 2, the worked example here.

Notes

On a shared 24-core machine gfa-to-tabix took 90 minutes and 8 GB of memory for GRCh38's 1.6 GB walk file, and 50 minutes and 9 GB for CHM13's 1.8 GB. ↩

Feedback on this tutorial is welcome: contact us.

Pangenome (hosting your own graph)

Prerequisites#

Download the data#

The GraphGenomeView plugin#

Indexing a graph with gfa-to-tabix build#

The coarse tier for whole-chromosome views#

Configuring the graph track#

Checking the index against the graph#

Recording which haplotypes pass through each segment#

Indexing haplotype walks with gfa-to-tabix#

Whole-genome synteny from the walks, as PAF#

Reproduce it end to end#

See also#

External links#

Citations#

Notes

Prerequisites

Download the data

The GraphGenomeView plugin

Indexing a graph with gfa-to-tabix build

The coarse tier for whole-chromosome views

Configuring the graph track

Checking the index against the graph

Recording which haplotypes pass through each segment

Indexing haplotype walks with gfa-to-tabix

Whole-genome synteny from the walks, as PAF

Reproduce it end to end

See also

External links

Citations