Synteny from an ortholog table (grape, peach, cacao)
The tutorials target the JBrowse v5 beta, and the v4.3.0 release
on the download page lacks some of what they show. To install the
beta of JBrowse Web, run npm install -g @jbrowse/cli@next, then
jbrowse create jbrowse2 --branch v5.0.0-beta.21. Desktop beta builds for Linux and
macOS are on the releases page;
the Windows build waits on code signing.
MCScan, from the jcvi toolkit, lines up orthologous genes across more than two genomes at once, into one wide table with a column per species. We:
- load the
.blocksfile as a synteny track and stack grape, peach and cacao as rows of one synteny view, zoomed to a conserved block - tell direct gene pairs from transitive ones
- open one grape locus against all the genomes as a multi-way track, one lane per genome
- redraw each lane in the coordinates of its genome, and restack the genomes around a locus
Prerequisites
- a JBrowse to open the files in: Desktop takes a local file by path, Web through Add track
- jcvi with the LAST aligner
- Or any other ortholog table, converted as in building a table for MCScanBlocksAdapter
- the NCBI
datasetsCLI - gffread
samtools
# Debian/Ubuntu
sudo apt install samtools gffread last-align python-is-python3
# or conda, which has every tool above
conda install -c bioconda -c conda-forge samtools gffread ncbi-datasets-cli \
last python
# or Linux x86_64 binaries for the tools apt lacks
wget -P ~/.local/bin \
https://ftp.ncbi.nlm.nih.gov/pub/datasets/command-line/v2/linux-amd64/datasets \
https://ftp.ncbi.nlm.nih.gov/pub/datasets/command-line/v2/linux-amd64/dataformat
chmod +x ~/.local/bin/datasets ~/.local/bin/dataformat
# any platform
pip install jcviThe data
Seven RefSeq assemblies, one per species, each fetched by accession with the
datasets CLI.
You don't need these files to follow the sessions, which read hosted copies. The track configs name local files that the build script writes, so run it first, or follow the jcvi steps on a GFF3 and a FASTA per genome of your own.
| What it is | File |
|---|---|
| Grape | GCF_030704535.1_ASM3070453v1/ |
| Peach | GCF_000346465.2_Prunus_persica_NCBIv2/ |
| Cacao | GCF_000208745.1_Criollo_cocoa_genome_V2/ |
| Arabidopsis | GCF_000001735.4_TAIR10.1/ |
| Poplar | GCF_000002775.5_P.trichocarpa_v4.1/ |
| Tomato | GCF_036512215.1_SLM_r2.1/ |
| Citrus | GCF_000493195.1_Citrus_clementina_v1.0/ |
What a .blocks file is
A .blocks file is a tab-delimited table. Each row is an orthologous group and
each column a genome, with . where a genome has no member. The file has no
header row, so blockAssemblies names its columns by position:
grape01 peach01 cacao01
grape02 peach02 cacao02
grape03 . .A real cell holds the gene id from the annotation (rna-XM_007225519.2 for
NCBI); the table has no coordinates, so one .bed per genome places each id.
How a .blocks table holds a duplicated gene
Two conventions place a second copy of a gene
mcscan writes a column per chain of synteny blocks, so a grape gene syntenic
to two peach regions fills a second peach column (--iter caps the chains). At
--iter=2:
grape01 peach01 peach01b
grape02 peach02 .
grape03 . .Name that column peach in blockAssemblies with peach.bed beside it, and
the track draws both; assemblyNames on the track still names peach once.
The other convention repeats the grape id, one row per copy:
grape01 peach01 cacao01
grape02 peach02a cacao02
grape02 peach02b cacao02
grape03 . .OrthoFinder's converter, orthologs2jbrowse orthofinder, writes this shape by
default; the MCScanX converter keeps the best-scoring copy.
Producing grape.blocks and the BEDs with jcvi
grape.blocks and the BEDs come from jcvi
and the LAST aligner.
for sp in grape peach cacao; do
gffread "$sp.gff3" -g "$sp.fa" -x "$sp.cds.fa"
# --key=ID joins the two files; --primary_only keeps one transcript per gene
python -m jcvi.formats.gff bed --type=mRNA --key=ID --primary_only \
"$sp.gff3" -o "$sp.bed"
python -m jcvi.formats.fasta format "$sp.cds.fa" "$sp.cds"
done--key=ID writes the GFF3 id of each mRNA into BED column 4, matching the CDS
name gffread wrote. On an NCBI annotation, --key=transcript_id and
--key=Name leave jcvi a key it cannot resolve, and it writes a placeholder id
like mrna_494685 that matches no CDS name.
Then catalog orthologs against grape, MCScan each pair, and join. The grape and
peach catalog step is the one the MCScan anchors page ran, and it writes
grape.peach.lifted.anchors beside the .anchors file that page loads:
for sp in peach cacao; do
# --no_strip_names keeps the ids matching the BEDs above; --no_dotplot skips
# jcvi's own PDF, which needs LaTeX
python -m jcvi.compara.catalog ortholog --no_strip_names --no_dotplot grape "$sp"
# --iter=1 keeps one block per grape gene, so one mate per genome
python -m jcvi.compara.synteny mcscan grape.bed "grape.$sp.lifted.anchors" \
--iter=1 -o "grape.$sp.i1.blocks"
done
# each per-pair table lists grape then the mate, so the join emits the grape
# column twice; cut -f1,2,4 keeps it once, in the order blockAssemblies and
# bedLocations list
python -m jcvi.formats.base join grape.peach.i1.blocks grape.cacao.i1.blocks \
--noheader | cut -f1,2,4 > grape.blocksThe commands above build the two per-pair tables the stack uses, peach and cacao
against grape. The build script runs the same steps for arabidopsis, poplar,
tomato and citrus too, joins all six tables and keeps column 1 and every even
column after it, which writes the seven-column grape.blocks.
Building a table for MCScanBlocksAdapter
converts MCScanX, OrthoFinder, Ensembl Compara and reciprocal-best-hit output to
the same shape, and the adapter reads the .blocks and BED files plain or
gzipped.
Setting up the grape, peach and cacao assemblies
Grape, peach and cacao are each a genome hub on genomes.jbrowse.org under the RefSeq accession the table was built from. The assembly entry and gene track of each row come from there:
# a GenArk hub path is the accession cut into threes
curl -fO https://jbrowse.org/hubs/genark/GCF/000/346/465/GCF_000346465.2/config.jsonThe build keeps each hub entry, labels the row, and adds the short name as an
alias, so a session can still say peach:
{
"name": "GCF_000346465.2",
"displayName": "peach",
"aliases": ["peach"],
"sequence": {
"type": "ReferenceSequenceTrack",
"trackId": "GCF_000346465.2-ReferenceSequenceTrack",
"adapter": {
"type": "TwoBitAdapter",
"uri": "https://hgdownload.soe.ucsc.edu/hubs/GCF/000/346/465/GCF_000346465.2/GCF_000346465.2.2bit",
"chromSizes": "https://hgdownload.soe.ucsc.edu/hubs/GCF/000/346/465/GCF_000346465.2/GCF_000346465.2.chrom.sizes.txt"
}
},
"refNameAliases": {
"adapter": {
"type": "RefNameAliasAdapter",
"refNameColumnHeaderName": "ucsc",
"uri": "https://hgdownload.soe.ucsc.edu/hubs/GCF/000/346/465/GCF_000346465.2/GCF_000346465.2.chromAlias.txt"
}
}
}refNameColumnHeaderName makes the UCSC names canonical, so the first peach
chromosome is chrG1 where the BED has NC_034009.1.
Loading the blocks file with MCScanBlocksAdapter
One track backs each pair of adjacent rows. blockAssemblies names every column
in order, and bedLocations gives the matching BED per column; the adapter
needs no assemblyNames unless you mean to narrow it:
Goes in the tracks array of config.json. See Tracks.
{
"type": "SyntenyTrack",
"trackId": "grape_peach_cacao_blocks",
"name": "Grape / peach / cacao (MCScan blocks)",
"assemblyNames": ["GCF_030704535.1", "GCF_000346465.2", "GCF_000208745.1"],
"adapter": {
"type": "MCScanBlocksAdapter",
"uri": "grape.blocks.gz",
"blockAssemblies": [
"GCF_030704535.1",
"GCF_000346465.2",
"GCF_000208745.1"
],
"bedLocations": ["grape.bed.gz", "peach.bed.gz", "cacao.bed.gz"]
}
}Edits config.json in the current folder. See JBrowse CLI with Desktop.
jbrowse add-track-json '{
"type": "SyntenyTrack",
"trackId": "grape_peach_cacao_blocks",
"name": "Grape / peach / cacao (MCScan blocks)",
"assemblyNames": ["GCF_030704535.1", "GCF_000346465.2", "GCF_000208745.1"],
"adapter": {
"type": "MCScanBlocksAdapter",
"uri": "grape.blocks.gz",
"blockAssemblies": [
"GCF_030704535.1",
"GCF_000346465.2",
"GCF_000208745.1"
],
"bedLocations": ["grape.bed.gz", "peach.bed.gz", "cacao.bed.gz"]
}
}'In JBrowse Desktop, or in any running JBrowse Web session, open a view on this track’s assembly, then File → Open track..., choose Add track from pasted JSON, and paste:
{
"type": "SyntenyTrack",
"trackId": "grape_peach_cacao_blocks",
"name": "Grape / peach / cacao (MCScan blocks)",
"assemblyNames": ["GCF_030704535.1", "GCF_000346465.2", "GCF_000208745.1"],
"adapter": {
"type": "MCScanBlocksAdapter",
"uri": "grape.blocks.gz",
"blockAssemblies": [
"GCF_030704535.1",
"GCF_000346465.2",
"GCF_000208745.1"
],
"bedLocations": ["grape.bed.gz", "peach.bed.gz", "cacao.bed.gz"]
}
}grape.blocks.gz, grape.bed.gz, peach.bed.gz, cacao.bed.gz are relative to a config.json. Replace each with its URL or its path on this computer.
That shows it in the linear view. For the synteny view, open Add → Linear synteny view, pick the track under Quick start, and click Launch.
The hosted
config.json holds the
seven-column version of this track under the same trackId, named Grape vs
peach, cacao, arabidopsis, poplar, tomato, citrus (MCScan blocks). Every
session and figure below runs on that one. Its assemblyNames lists only grape,
peach and cacao, the genomes with an assembly; the other four columns draw as
multi-way lanes from the table and their BED alone.
Stacking grape, peach and cacao
Choose Add → Linear synteny view and pick grape_peach_cacao_blocks; the
all-vs-all tutorial walks the
dialog. As a defaultSession, the same stack ordered peach-cacao-grape:
Goes at the top level of config.json, replacing any defaultSession there. See Default session.
{
"defaultSession": {
"name": "Grape / Peach / Cacao multi-way synteny",
"views": [
{
"type": "LinearSyntenyView",
"displayName": "Peach - Cacao - Grape (MCScan blocks)",
"views": [
{ "assembly": "GCF_000346465.2" },
{ "assembly": "GCF_000208745.1" },
{ "assembly": "GCF_030704535.1" }
],
"tracks": ["grape_peach_cacao_blocks"],
"encoding": { "color": "reference" },
"autoDiagonalize": true,
"collapseEmptyRows": true
}
]
}
}Replaces any defaultSession in config.json in the current folder. See JBrowse CLI with Desktop.
jbrowse set-default-session --session - << 'EOF'
{
"name": "Grape / Peach / Cacao multi-way synteny",
"views": [
{
"type": "LinearSyntenyView",
"displayName": "Peach - Cacao - Grape (MCScan blocks)",
"views": [
{ "assembly": "GCF_000346465.2" },
{ "assembly": "GCF_000208745.1" },
{ "assembly": "GCF_030704535.1" }
],
"tracks": ["grape_peach_cacao_blocks"],
"encoding": { "color": "reference" },
"autoDiagonalize": true,
"collapseEmptyRows": true
}
]
}
EOFOpen this session in JBrowse Web ↗
Needs JBrowse Desktop 5.0 or newer. If nothing opens, copy the link and paste it into Desktop's Open JBrowse Web link..., on the start screen or under File → Session.
https://jbrowse.org/code/jb2/main/?config=https%3A%2F%2Fjbrowse.org%2Fdemos%2Fgrape_peach_cacao%2Fconfig.json&session=json-%7B%22session%22%3A%7B%22name%22%3A%22Grape+%2F+Peach+%2F+Cacao+multi-way+synteny%22%2C%22views%22%3A%5B%7B%22type%22%3A%22LinearSyntenyView%22%2C%22displayName%22%3A%22Peach+-+Cacao+-+Grape+%28MCScan+blocks%29%22%2C%22views%22%3A%5B%7B%22assembly%22%3A%22GCF_000346465.2%22%7D%2C%7B%22assembly%22%3A%22GCF_000208745.1%22%7D%2C%7B%22assembly%22%3A%22GCF_030704535.1%22%7D%5D%2C%22tracks%22%3A%5B%22grape_peach_cacao_blocks%22%5D%2C%22encoding%22%3A%7B%22color%22%3A%22reference%22%7D%2C%22autoDiagonalize%22%3Atrue%2C%22collapseEmptyRows%22%3Atrue%7D%5D%7D%7D&sessionName=JBrowse+docs+exampletracksnames the track once, and it backs both bands because itsassemblyNameslist every genomecollapseEmptyRowsgives each trackless row a bare scale barencoding.coloris a color scale, afieldmapped to colors; thereferencefield colors both bands by the chromosomes of the shared middle row
Direct and transitive pairs
Every row of grape.blocks starts from a grape gene, as in any jcvi MCScan
table, so the bands of a stack differ in where their ribbons come from:
- A band touching grape draws pairs straight from the table (direct pairs).
- A peach-cacao band draws a ribbon only through a grape gene both genomes share, so each of its ribbons is transitive, and no ribbon is drawn for a row where grape has no member.
- The stack above puts grape at the bottom, so its peach-cacao band is the transitive one. A peach, grape, cacao stack makes both bands direct.
The adapter also loads an all-against-all table, such as an OrthoFinder one, where a row holds orthologs inferred across all genomes at once.
Zooming to one block conserved in every genome
We'll stack peach over grape over cacao. Then:
- Navigate the rows to
chrG7:18,565,100-18,700,100,chr11:787,000-914,000andchr9:3,898,700-4,003,700. - Turn on each genome's NCBI RefSeq - RefSeq All track with Filter by → Genes → Only genes.
- Pick Color by value → Syntenic depth on the palette button in the view header.
One grape locus against all the genomes
grape.blocks has seven columns, and the hosted track names all seven in
blockAssemblies and bedLocations, so it draws every mate (the ortholog in
each other genome) at once in a linear genome view on grape:
- Navigate to
chr11:778,000-866,000and turn on grape's NCBI RefSeq - RefSeq All track. - Turn on Grape vs peach, cacao, arabidopsis, poplar, tomato, citrus (MCScan blocks), which draws every mate in one pileup.
- Pick Group by → Mate assembly to facet the track into one lane per genome.
Lanes in per-genome coordinates
In the one-locus view above, every gene sits at its grape coordinate. Display types → Multi-way synteny display redraws the lanes, each in the coordinates of one genome:
- the display fits each lane to the genome it shows, over the orthologs in the window
- one ribbon per ortholog group joins adjacent lanes, bridging past an empty one, red where the two lanes run the same way and blue where one is reversed
- any track whose features have a
mateper assembly feeds the same lanes, including an OrthoFinder table or an all-vs-all PAF
Each lane's gene models come from a gene track on that lane's assembly, and gene
spans from the table are outlined where there is none. We'll add one for grape;
each other genome gets the same track under its accession. Swap the uri for
your own GFF3, bgzipped and tabix-indexed (prep), with
the same refNames as the assembly:
Goes in the tracks array of config.json. See Tracks.
{
"type": "FeatureTrack",
"trackId": "grape_genes",
"name": "grape genes",
"assemblyNames": ["GCF_030704535.1"],
"adapter": {
"type": "Gff3TabixAdapter",
"uri": "grape.sorted.gff3.gz"
}
}Edits config.json in the current folder. See JBrowse CLI with Desktop.
jbrowse add-track grape.sorted.gff3.gz \
--trackId grape_genes \
--name "grape genes" \
--assemblyNames GCF_030704535.1 \
--load copyIn JBrowse Desktop, or in any running JBrowse Web session, open a view on this track’s assembly, then File → Open track... and, in Add a track from file or URL, enter:
- Main file:
grape.sorted.gff3.gz
JBrowse reads the adapter and track type off the file name. Then fill in:
- Track name:
grape genes - Assembly:
GCF_030704535.1
Click Add.
grape.sorted.gff3.gz is relative to a config.json. Replace it with its URL or its path on this computer.
The multi-way lanes as a defaultSession (the track ids come from the hosted
config, so name your own trackId instead):
Goes at the top level of config.json, replacing any defaultSession there. See Default session.
{
"defaultSession": {
"name": "Grape multi-way synteny track",
"views": [
{
"type": "LinearGenomeView",
"assembly": "GCF_030704535.1",
"loc": "chr11:778,000-866,000",
"tracks": [
{
"trackId": "GCF_030704535.1-ncbiRefSeq",
"type": "LinearBasicDisplay",
"filter": ["geneLike"],
"displayMode": "compact"
},
{
"trackId": "grape_peach_cacao_blocks",
"type": "MultiWaySyntenyDisplay",
"rows": {
"sort": [
"GCF_000346465.2",
"GCF_000208745.1",
"poplar",
"citrus",
"arabidopsis",
"tomato"
]
},
"height": 340
}
]
}
]
}
}Replaces any defaultSession in config.json in the current folder. See JBrowse CLI with Desktop.
jbrowse set-default-session --session - << 'EOF'
{
"name": "Grape multi-way synteny track",
"views": [
{
"type": "LinearGenomeView",
"assembly": "GCF_030704535.1",
"loc": "chr11:778,000-866,000",
"tracks": [
{
"trackId": "GCF_030704535.1-ncbiRefSeq",
"type": "LinearBasicDisplay",
"filter": ["geneLike"],
"displayMode": "compact"
},
{
"trackId": "grape_peach_cacao_blocks",
"type": "MultiWaySyntenyDisplay",
"rows": {
"sort": [
"GCF_000346465.2",
"GCF_000208745.1",
"poplar",
"citrus",
"arabidopsis",
"tomato"
]
},
"height": 340
}
]
}
]
}
EOFOpen this session in JBrowse Web ↗
Needs JBrowse Desktop 5.0 or newer. If nothing opens, copy the link and paste it into Desktop's Open JBrowse Web link..., on the start screen or under File → Session.
https://jbrowse.org/code/jb2/main/?config=https%3A%2F%2Fjbrowse.org%2Fdemos%2Fgrape_peach_cacao%2Fconfig.json&session=json-%7B%22session%22%3A%7B%22name%22%3A%22Grape+multi-way+synteny+track%22%2C%22views%22%3A%5B%7B%22type%22%3A%22LinearGenomeView%22%2C%22assembly%22%3A%22GCF_030704535.1%22%2C%22loc%22%3A%22chr11%3A778%2C000-866%2C000%22%2C%22tracks%22%3A%5B%7B%22trackId%22%3A%22GCF_030704535.1-ncbiRefSeq%22%2C%22type%22%3A%22LinearBasicDisplay%22%2C%22filter%22%3A%5B%22geneLike%22%5D%2C%22displayMode%22%3A%22compact%22%7D%2C%7B%22trackId%22%3A%22grape_peach_cacao_blocks%22%2C%22type%22%3A%22MultiWaySyntenyDisplay%22%2C%22rows%22%3A%7B%22sort%22%3A%5B%22GCF_000346465.2%22%2C%22GCF_000208745.1%22%2C%22poplar%22%2C%22citrus%22%2C%22arabidopsis%22%2C%22tomato%22%5D%7D%2C%22height%22%3A340%7D%5D%7D%5D%7D%7D&sessionName=JBrowse+docs+exampleEach lane header gives the lane's range, with [rev] where its gene order runs
against grape's, and, on a lane zoomed out past grape, its span as a multiple of
the grape window. rows.sort in the session above sets the lane order, peach
and cacao first. The
multi-way synteny display guide
covers the headers, lane order and lane menus.
Zooming the lanes to a few genes
Zoom in until each ribbon connects one gene to its ortholog, fanning where copy number differs.
- Hover a ribbon to highlight its ortholog group down every lane.
- Click a ribbon to open the details for that pair, and click empty canvas to clear the outline.
Restacking the genomes around a locus
Two routes open a stacked synteny view from the lanes:
- from the lane track, Advanced → Launch → Linear synteny view (visible region)... in its track menu offers a row to every aligning genome
- from the scale bar, drag-select a locus and pick Launch → Linear synteny view. The dialog opens a row per genome, grape already placed between peach and cacao for the reference-in-the-middle layout, with arrows to reorder them
Reproduce it end to end
build_grape_peach_cacao_synteny.sh
runs everything above and writes a config.json with the hub assemblies, gene
tracks, synteny track and default session:
- Download the seven RefSeq genomes and annotations, and write a BED and a CDS
file per genome keyed on the mRNA's GFF3
ID, so the two join. - Catalog orthologs against grape and MCScan each pair with
--iter=1, one block per grape gene. - Join the six per-pair tables on the grape column into one seven-column
grape.blocks. - Write the config from the grape, peach and cacao hubs, with the other four genomes as lanes of the blocks track.
BLOCKS_ONLY_SPECIES lists the extra lanes; a genome added there needs an NCBI
accession with an annotation, and gets a lane but no row of its own.
curl -fO https://raw.githubusercontent.com/GMOD/jbrowse-components/main/scripts/build_grape_peach_cacao_synteny.sh
bash build_grape_peach_cacao_synteny.sh
npx --yes serve grape_peach_cacao_build # then open config.json at the printed URLSee also
- Tutorial: Synteny visualization (all-vs-all minimap2)
- Tutorial: Synteny visualization (OrthoFinder orthogroups)
- Tutorial: Synteny visualization (a polyploid against itself)
- Tutorial: Synteny visualization (pairwise minimap2)
- Tutorial: Synteny on genomes.jbrowse.org
- User guide: Linear synteny view
- Config guide: Synteny track
- Config guide: Grouping and lane order
- Reference: MCScanBlocksAdapter
- Reference: MultiWaySyntenyDisplay
Feedback on this tutorial is welcome: contact us.