Synteny visualization (all-vs-all minimap2)
The tutorials target the JBrowse v5 beta, and the v4.3.0 release
on the download page lacks some of what they show. To install the
beta of JBrowse Web, run npm install -g @jbrowse/cli@next, then
jbrowse create jbrowse2 --branch v5.0.0-beta.8. Desktop beta builds are coming soon.
A synteny track shows which stretches of DNA correspond between genomes. We stack five E. coli strains in one linear view, built from a single all-vs-all PAF, the file minimap2 writes when every genome is aligned against every other.
Prerequisites
- a JBrowse to open them in: Desktop takes a local file by path, Web through Add track
- the NCBI
datasetsCLI minimap2samtools- htslib (
bgzip,tabix) unzipnode, for the JBrowse CLI
On Debian/Ubuntu, apt install minimap2 samtools tabix unzip covers most of
these. The NCBI datasets CLI is a single-binary download, and node comes
from nodejs.org.
Where the data comes from
Five E. coli RefSeq assemblies, each fetched by accession with the datasets
CLI.
-
K12: https://ftp.ncbi.nlm.nih.gov/genomes/all/GCF/000/005/845/GCF_000005845.2_ASM584v2/
-
Sakai: https://ftp.ncbi.nlm.nih.gov/genomes/all/GCF/000/008/865/GCF_000008865.2_ASM886v2/
-
CFT073: https://ftp.ncbi.nlm.nih.gov/genomes/all/GCF/000/007/445/GCF_000007445.1_ASM744v1/
-
NCTC86: https://ftp.ncbi.nlm.nih.gov/genomes/all/GCF/002/007/705/GCF_002007705.1_ASM200770v1/
-
IAI39: https://ftp.ncbi.nlm.nih.gov/genomes/all/GCF/000/026/345/GCF_000026345.1_ASM2634v1/
-
the all-vs-all PAF, per-strain gene tracks and config, rehosted so the stacked view loads without rerunning the pipeline: https://jbrowse.org/demos/ecoli_pangenome/
Many genome rows from one PAF file
A linear synteny view stacks N genome rows with a ribbon band between each adjacent pair. For closely related genomes, the most convenient source is a single all-vs-all PAF, every genome aligned to every other. This page builds a five-strain E. coli view from one such file.
Synteny from ortholog tables stacks cross-species comparisons from gene-level ortholog tables, and the gene-symbol join scales this page's five strains (same accessions, K12 there is MG1655) to forty-four lanes by skipping the alignment altogether.
Producing an all-vs-all PAF
The PGGB mapping step produces one, or
concatenate PanSN-named genomes and
self-align them with minimap2. PanSN names
every sequence sample#haplotype#contig, e.g. K12#1#chr, and the adapter uses
the sample prefix to classify each PAF record.
The script downloads five RefSeq assemblies with the
NCBI datasets CLI, annotation included, and reduces each to one chr record
by dropping the plasmids and renaming the chromosome:
| Strain | RefSeq accession |
|---|---|
| K12 | GCF_000005845.2 |
| Sakai | GCF_000008865.2 |
| CFT073 | GCF_000007445.1 |
| NCTC86 | GCF_002007705.1 |
| IAI39 | GCF_000026345.1 |
Those five FASTAs become the JBrowse assemblies as-is. The PanSN names exist
only inside the PAF, so the concatenated copy for minimap2 is its own file. The
haplotype is 1 throughout, since these are haploid:
for strain in K12 Sakai CFT073 NCTC86 IAI39; do
# '>chr' -> '>K12#1#chr'
awk -v s="$strain" '/^>/{print ">" s "#1#chr"; next} {print}' "$strain.fa"
done > all.fa
minimap2 -c -x asm20 -X all.fa all.fa > all_vs_all.paf-c emits the base-level CIGAR the linear synteny view needs. -X skips each
sequence's own diagonal and the reciprocal copy of each pair, leaving every
cross-strain pair once. Paralogy is untouched, which
one-vs-all mode reads as a strain's own
repeats.
Setting up the five assemblies
Each strain FASTA becomes an assembly whose name matches an entry in the track's
assemblyNames:
for strain in K12 Sakai CFT073 NCTC86 IAI39; do
bgzip -f "$strain.fa"
samtools faidx "$strain.fa.gz" # writes the .fai and .gzi JBrowse needs
jbrowse add-assembly "$strain.fa.gz" --name "$strain" --load copy
doneThe adapter strips the PanSN prefix before matching, so K12#1#chr in the PAF
resolves to chr in the K12 assembly. An assembly whose refNames still carry
the prefix draws empty. The
assemblies configuration guide has the
equivalent JSON.
Loading the PAF with MultiGenomePAFAdapter
One track backs every band of the stacked view. List every assembly the file
covers in assemblyNames; the adapter keeps only the records whose PanSN
prefixes match the pair each band draws:
{
"type": "SyntenyTrack",
"trackId": "ecoli_ava",
"name": "E. coli pangenome (all-vs-all PAF)",
"assemblyNames": ["K12", "Sakai", "CFT073", "NCTC86", "IAI39"],
"adapter": {
"type": "MultiGenomePAFAdapter",
"uri": "all_vs_all.paf",
"assemblyNames": ["K12", "Sakai", "CFT073", "NCTC86", "IAI39"]
}
}jbrowse add-track all_vs_all.paf \
--trackId ecoli_ava \
--name "E. coli pangenome (all-vs-all PAF)" \
--assemblyNames K12,Sakai,CFT073,NCTC86,IAI39 \
--adapterType MultiGenomePAFAdapter \
--load copyIn JBrowse Desktop, or in any running JBrowse Web session, open a view on this track’s assembly, then File → Open track..., choose Add track from pasted JSON, and paste:
{
"type": "SyntenyTrack",
"trackId": "ecoli_ava",
"name": "E. coli pangenome (all-vs-all PAF)",
"assemblyNames": ["K12", "Sakai", "CFT073", "NCTC86", "IAI39"],
"adapter": {
"type": "MultiGenomePAFAdapter",
"uri": "all_vs_all.paf",
"assemblyNames": ["K12", "Sakai", "CFT073", "NCTC86", "IAI39"]
}
}all_vs_all.paf is relative to a config.json. Replace it with its URL or its path on this computer.
MultiGenomePAFAdapter has to be named; a .paf guessed from its extension is
the pairwise PAFAdapter, which reads only the first two assembly names.
If an assembly name differs from its PanSN sample prefix, map it with
assemblyNameToPanSN, e.g. { "Ecoli_K12": "K12" }. A name matching no sample
raises an error listing the samples the file holds.
Haplotype-resolved genomes
Here K12 covers everything named K12#1#.... A haplotype-resolved pangenome
can load each haplotype as its own assembly mapped to a sample#haplotype
prefix, making hap1 against hap2 a band in its own right. See
PanSN depth.
Large files: index with make-pif
MultiGenomePAFAdapter reads the whole PAF into memory. For a whole-genome
pangenome of many samples, index it with jbrowse make-pif and switch to
MultiGenomeIndexedPAFAdapter, which fetches only the records overlapping the
region in view:
# produces all_vs_all.pif.gz and all_vs_all.pif.gz.tbi
jbrowse make-pif all_vs_all.pafmake-pif finishes by printing the add-track command for the samples it
found:
jbrowse add-track all_vs_all.pif.gz --adapterType MultiGenomeIndexedPAFAdapter \
-a CFT073,IAI39,K12,NCTC86,Sakai --load copyOnly the adapter block differs from the un-indexed version:
{
"type": "SyntenyTrack",
"trackId": "ecoli_ava_indexed",
"name": "E. coli all-vs-all (indexed)",
"assemblyNames": ["K12", "Sakai", "CFT073", "NCTC86", "IAI39"],
"adapter": {
"type": "MultiGenomeIndexedPAFAdapter",
"uri": "all_vs_all.pif.gz",
"assemblyNames": ["K12", "Sakai", "CFT073", "NCTC86", "IAI39"]
}
}jbrowse add-track all_vs_all.pif.gz \
--trackId ecoli_ava_indexed \
--name "E. coli all-vs-all (indexed)" \
--assemblyNames K12,Sakai,CFT073,NCTC86,IAI39 \
--adapterType MultiGenomeIndexedPAFAdapter \
--load copyIn JBrowse Desktop, or in any running JBrowse Web session, open a view on this track’s assembly, then File → Open track..., choose Add track from pasted JSON, and paste:
{
"type": "SyntenyTrack",
"trackId": "ecoli_ava_indexed",
"name": "E. coli all-vs-all (indexed)",
"assemblyNames": ["K12", "Sakai", "CFT073", "NCTC86", "IAI39"],
"adapter": {
"type": "MultiGenomeIndexedPAFAdapter",
"uri": "all_vs_all.pif.gz",
"assemblyNames": ["K12", "Sakai", "CFT073", "NCTC86", "IAI39"]
}
}all_vs_all.pif.gz is relative to a config.json. Replace it with its URL or its path on this computer.
make-pif emits a coarse zoomed-out tier by default so whole-genome views stay
responsive. --coarse tunes that tier and --csi swaps the TBI index for
sequences longer than ~512 Mb. Raising --coarse means raising the adapter's
coarseBpPerPxThreshold
with it, so coarse ribbons stay at zooms where their flattened indels are too
small to see.
Stacking the genomes
From the UI
- Add → Linear synteny view opens the form in Quick start.
- Choose
ecoli_ava. Its five assemblies each become a row. - Click Launch.
Manual mode builds the stack by hand: Add row per strain, and the connector button between each pair to pick its track.
Declaratively with defaultSession
A defaultSession holding a LinearSyntenyView opens the stack on load. Five
rows means four bands, so tracks has four entries:
{
"defaultSession": {
"name": "E. coli 5-strain pangenome",
"views": [
{
"type": "LinearSyntenyView",
"views": [
{ "assembly": "K12" },
{ "assembly": "Sakai" },
{ "assembly": "CFT073" },
{ "assembly": "NCTC86" },
{ "assembly": "IAI39" }
],
"tracks": [["ecoli_ava"], ["ecoli_ava"], ["ecoli_ava"], ["ecoli_ava"]],
"minAlignmentLength": 10000,
"collapseEmptyRows": true
}
]
}
}jbrowse set-default-session --session - << 'EOF'
{
"name": "E. coli 5-strain pangenome",
"views": [
{
"type": "LinearSyntenyView",
"views": [
{ "assembly": "K12" },
{ "assembly": "Sakai" },
{ "assembly": "CFT073" },
{ "assembly": "NCTC86" },
{ "assembly": "IAI39" }
],
"tracks": [["ecoli_ava"], ["ecoli_ava"], ["ecoli_ava"], ["ecoli_ava"]],
"minAlignmentLength": 10000,
"collapseEmptyRows": true
}
]
}
EOFtracksis one entry per band:tracks[0]connects rows 0-1,tracks[1]rows 1-2, and so onminAlignmentLengthhides minimap2's many short alignments, leaving the shared backbonecollapseEmptyRowsgives a ribbon-only row a bare scalebar
The
ortholog-tables tutorial
walks through the defaultSession structure. Row order is a free choice here,
since an all-vs-all file has a direct alignment for every pair.
The gaps are where the strains differ: Sakai's largest carry its prophage Shiga-toxin genes, and CFT073's are its pathogenicity islands.
Adding gene tracks
The annotations downloaded alongside each genome show what a gap holds. Each GFF
gets the same two adjustments as the FASTA, in the
script: seqid renamed to chr, plasmid features
dropped.
for strain in K12 Sakai CFT073 NCTC86 IAI39; do
jbrowse sort-gff "$strain.gff" | bgzip > "$strain.gff.gz"
tabix "$strain.gff.gz"
jbrowse add-track "$strain.gff.gz" -a "$strain" --name "$strain genes" --load copy
done-a "$strain" adds each track to one strain's assembly, so the track stays with
that row. Navigate Sakai's row to chr:1,267,000-1,268,400 and the gap holds
stx2A and stx2B, the Shiga-toxin subunits, with no alignment to K-12.
One strain against all the others
In a plain linear genome view, with no target assembly, the same track draws the strain you are looking at against every other sample in the file. A mate the track does not list still draws, labelled by its PanSN prefix, as does a strain's own paralogy. Clicking a feature offers to launch a synteny view against its mate, for the mates the track lists.
Every alignment lands in one pileup, so three track-menu items name which strain a block came from:
- Group by... → Mate assembly gives one labelled lane per sample, shading darker where several alignments cover the same base. Untick Show... → Collapse groups to one row to stack every lane, or expand one from its label. Sections in the track menu then lists the lanes drawn, each with Move up, Move down and Hide section, and Reset section order returns them to the sorted order.
- Group by... → Hide self-alignment lane drops the lane for the strain you
are viewing.
minimap2 -Xskipped each genome's own diagonal, so that lane holds K-12's own internal repeats rather than its diagonal. The figures below have it ticked. - Show... → Show coverage adds a histogram of how many other strains cover each base. The rest of that menu is the one from alignments tracks.
The figure below adds a second pane, the same window in the pangenome graph, which the next section picks up. The shaded band is K-12's phenylacetate (paa) operon: Sakai, CFT073 and IAI39 stop at its left edge where NCTC86 runs through.
Zoomed out to the whole chromosome, the lanes give a per-strain overview, and they can sit on the K-12 row of the stack above since a synteny view's rows are ordinary linear genome views. For a real pangenome, index first with make-pif:
Each strain's lane in its own coordinates
On K-12's axis a stretch of a strain with no alignment to the backbone is a white gap, and the lane cannot show what that strain carries instead. Display types → Multi-way synteny display redraws the track with each strain's lane in its own coordinates, the reading the ortholog-table tutorial walks through on gene names. With no gene names:
- each PAF record is its own ribbon, keyed by the adapter's
syntenyId - the gutters carry each adjacent pair's direct alignments from the same file
Above the lanes sits the pggb graph-depth wiggle the E. coli pangenome tutorial builds from these same strains.
{
"defaultSession": {
"name": "E. coli all-vs-all multi-way track",
"views": [
{
"type": "LinearGenomeView",
"assembly": "K12",
"loc": "chr:1,443,000-1,466,000",
"tracks": [
{ "trackId": "ecoli_pggb_depth", "height": 60 },
{
"trackId": "ecoli_ava",
"type": "MultiWaySyntenyDisplay",
"domain": ["NCTC86", "CFT073", "Sakai", "IAI39"],
"height": 340
}
]
}
]
}
}jbrowse set-default-session --session - << 'EOF'
{
"name": "E. coli all-vs-all multi-way track",
"views": [
{
"type": "LinearGenomeView",
"assembly": "K12",
"loc": "chr:1,443,000-1,466,000",
"tracks": [
{ "trackId": "ecoli_pggb_depth", "height": 60 },
{
"trackId": "ecoli_ava",
"type": "MultiWaySyntenyDisplay",
"domain": ["NCTC86", "CFT073", "Sakai", "IAI39"],
"height": 340
}
]
}
]
}
EOFThe gap in the graph genome view
Sequence absent from the alignment is absent from the PAF; in the graph the
island is a segment, and each strain's walk goes through it or detours around
it. The E. coli tutorials build that minigraph graph, and the
graph genome view plugin opens a window
of it beside the alignment. The ringed segment, s502, is the long node
carrying the island.
The lower band is blank across the island, which is a substitution: each strain carries an island of its own there, the phenylacetate operon and a prophage on K-12, a set of nleG effector genes on Sakai.
Launching a stacked view at one locus
Drag-select a region and pick Launch → Linear synteny view. With the all-vs-all track as the dialog's dataset, JBrowse finds every assembly aligning to that region and opens a row for each. The dialog lists them top to bottom and lets you reorder them; ribbons are drawn between neighbouring rows only, which is why IAI39 sits directly below K-12 in the whole-genome figure above.
Right-clicking a single alignment offers three routes. Linear synteny view with Sakai (or whichever strain the alignment names) opens the one pair that alignment describes. Linear synteny view, all assemblies here is the same multi-strain dialog. Open Sakai at the matching region opens that strain on its own coordinates at the matching stretch, which is also what a graph node offers.
A launched view is a few kilobases wide, where the CIGAR minimap2 -c wrote
matters: each insertion and deletion is drawn where it falls. The color key
appears by itself for the modes that have one and closes from its own button,
and CIGAR indels in the settings menu switches between colored, transparent
and none.
Checking a gap against the PAF
Print the Sakai side of every Sakai/K-12 alignment near the stx2 island, taking
the coordinates from whichever column Sakai landed in, since -X emits each
pair once in either direction:
awk -F'\t' -v OFS='\t' '
$1 ~ /^Sakai#/ && $6 ~ /^K12#/ { print $3, $4; next }
$1 ~ /^K12#/ && $6 ~ /^Sakai#/ { print $8, $9 }
' all_vs_all.paf | sort -n | awk '$1 < 1300000 && $2 > 1200000'1207288 1207877
1210882 1246166
1251954 1252260
1274685 1275548The second line is the shared backbone the ribbon draws. Past a short scrap nothing aligns again until the fourth, so stx2A and stx2B fall in a stretch with no K-12 counterpart.
Reproduce it end to end
build_ecoli_pangenome_synteny.sh
runs everything on this page, download and preparation included:
curl -fO https://raw.githubusercontent.com/GMOD/jbrowse-components/main/scripts/build_ecoli_pangenome_synteny.sh
bash build_ecoli_pangenome_synteny.sh # builds ./ecoli_pangenome_build/jbrowse2
npx --yes serve ecoli_pangenome_build/jbrowse2 # then open the printed URLThe script writes a config.json with the five assemblies, the gene tracks, the
synteny track, and a default session opening the stacked view. It needs the
tools under Prerequisites.
For a whole-genome pangenome, swap the add-track step for the make-pif +
MultiGenomeIndexedPAFAdapter path from
Large files.
See also
- Pangenome (pggb)
- Synteny from gene symbols (44 E. coli genomes)
- Pangenome (Minigraph-Cactus)
- Synteny visualization (pairwise minimap2)
- Synteny from an ortholog table (grape, peach, cacao)
- Dotplot view
- Synteny track
- MultiGenomePAFAdapter
- MultiGenomeIndexedPAFAdapter
- PIF (Pairwise Indexed Format)
- jbrowse-anywidget
- JBrowseR
Feedback on this tutorial is welcome: contact us.