RepeatMasker as one row per class
The tutorials target the JBrowse v5 beta, and the v4.3.0 release
on the download page lacks some of what they show. To install the
beta of JBrowse Web, run npm install -g @jbrowse/cli@next, then
jbrowse create jbrowse2 --branch v5.0.0-beta.22. Desktop beta builds for Linux and
macOS are on the releases page;
the Windows build waits on code signing.
RepeatMasker annotates a genome's repeats and names each copy's class (SINE, LINE, LTR, DNA transposon, simple repeat). The classes spread along a genome differently: over a window of 17q21, SINE copies fill it end to end while LINE copies come in clusters. A RepeatMasker track draws every class in one packed row of colored blocks, where that difference is hard to read. We:
- switch hg38's RepeatMasker track to a multi-row feature display, one labelled row per class, from the track menu
- pin the row order and colors in a track config
- convert your own RepeatMasker
.outinto the BED the display reads, sort it with the JBrowse CLI and point the track config at it
Every genome at genomes.jbrowse.org, and any UCSC or GenArk hub config, has a RepeatMasker track to try the first step on.
Prerequisites
- a JBrowse to paste the tracks into (Web or Desktop)
- htslib (
bgzip,tabix), for indexing your own output samtools, for serving your own RepeatMasker output and Reproduce it end to end, which runs the same conversionnode, for the JBrowse CLI, which both of those need too
# Debian/Ubuntu
sudo apt install samtools tabix nodejs
# or conda, which has every tool above
conda install -c bioconda -c conda-forge samtools htslib nodejs
# any platform
npm install -g @jbrowse/cli@nextThe data
The figures read UCSC's RepeatMasker track for hg38, rehosted on jbrowse.org.
Nothing to download: the track configs read their files by URL. The
build script takes a genome of your own, as its
FASTA and RepeatMasker's .out.
Switching hg38's RepeatMasker track to one row per class
We'll open hg38 on genomes.jbrowse.org and switch the track:
- Turn on RepeatMasker and type
chr17:45,700,000-45,750,000into the location box. - Choose Display types → Multi-row display in the track menu. The display
rows on the features'
typeunless told otherwise, and RepeatMasker's table has notype, so it draws every repeat in one row and opens One row per. - Pick
repClassin the box. Dismissed, the picker comes back from the One row per... chip at the head of the track, or from One row per → Attribute... in the track menu.
The picker lists every attribute the features in view carry, with how many rows
each would make, repFamily among them.
Where UCSC and GenArk hubs store the repeat class
The multi-row display's rows assigns one row per value of a field, and on a
GenArk genome the switch above needs that field spelled out, because UCSC
golden-path and GenArk hubs store the repeat class in different places:
- A UCSC golden-path assembly ships a BED whose header names its columns,
repClassamong them. That column is a feature attribute, sorowsis"repClass". - A GenArk assembly ships a
bigRmskBed, whose autoSql has no class column. The name contains the class as a suffix,L1HS#LINE/L1, sorowsderives it with a jexl expression,"jexl:split(split(feature.name,'#')[1],'/')[0]"; Multi-row feature track works through it.
In both cases JBrowse builds the rows from the values in the window, so a window with no satellite repeats has no satellite row.
Pinning row order and colors in a track config
A track config can open the track in the partitioned view, and two settings keep
the rows stable as the window's class list changes. rows.sort lists the
classes that come first, in order, and a class it leaves out follows the listed
ones. rows.colors pins a color on the label bar of each class it names.
Goes in the tracks array of config.json. See Tracks.
{
"type": "FeatureTrack",
"trackId": "rmsk_hg38_rows",
"name": "RepeatMasker by class",
"assemblyNames": ["hg38"],
"adapter": {
"type": "BedTabixAdapter",
"uri": "https://jbrowse.org/ucsc/hg38/rmsk.bed.gz",
"csi": true
},
"displays": [
{
"type": "LinearMultiRowFeatureDisplay",
"rows": {
"field": "repClass",
"sort": [
"SINE",
"LINE",
"LTR",
"DNA",
"Simple_repeat",
"Low_complexity"
],
"axis": { "grid": true },
"colors": {
"SINE": "#e41a1c",
"LINE": "#377eb8",
"LTR": "#4daf4a",
"DNA": "#984ea3",
"Simple_repeat": "#ff7f00",
"Low_complexity": "#a65628"
}
}
}
]
}Edits config.json in the current folder. See JBrowse CLI with Desktop.
jbrowse add-track-json '{
"type": "FeatureTrack",
"trackId": "rmsk_hg38_rows",
"name": "RepeatMasker by class",
"assemblyNames": ["hg38"],
"adapter": {
"type": "BedTabixAdapter",
"uri": "https://jbrowse.org/ucsc/hg38/rmsk.bed.gz",
"csi": true
},
"displays": [
{
"type": "LinearMultiRowFeatureDisplay",
"rows": {
"field": "repClass",
"sort": [
"SINE",
"LINE",
"LTR",
"DNA",
"Simple_repeat",
"Low_complexity"
],
"axis": { "grid": true },
"colors": {
"SINE": "#e41a1c",
"LINE": "#377eb8",
"LTR": "#4daf4a",
"DNA": "#984ea3",
"Simple_repeat": "#ff7f00",
"Low_complexity": "#a65628"
}
}
}
]
}'In JBrowse Desktop, or in any running JBrowse Web session, open a view on this track’s assembly, then File → Open track..., choose Add track from pasted JSON, and paste:
{
"type": "FeatureTrack",
"trackId": "rmsk_hg38_rows",
"name": "RepeatMasker by class",
"assemblyNames": ["hg38"],
"adapter": {
"type": "BedTabixAdapter",
"uri": "https://jbrowse.org/ucsc/hg38/rmsk.bed.gz",
"csi": true
},
"displays": [
{
"type": "LinearMultiRowFeatureDisplay",
"rows": {
"field": "repClass",
"sort": [
"SINE",
"LINE",
"LTR",
"DNA",
"Simple_repeat",
"Low_complexity"
],
"axis": { "grid": true },
"colors": {
"SINE": "#e41a1c",
"LINE": "#377eb8",
"LTR": "#4daf4a",
"DNA": "#984ea3",
"Simple_repeat": "#ff7f00",
"Low_complexity": "#a65628"
}
}
}
]
}Open this track in JBrowse Web ↗
Needs JBrowse Desktop 5.0 or newer. If nothing opens, copy the link and paste it into Desktop's Open JBrowse Web link..., on the start screen or under File → Session.
https://jbrowse.org/code/jb2/main/?config=https%3A%2F%2Fjbrowse.org%2Fucsc%2Fhg38%2Fconfig.json&session=spec-%7B%22sessionTracks%22%3A%5B%7B%22type%22%3A%22FeatureTrack%22%2C%22trackId%22%3A%22rmsk_hg38_rows%22%2C%22name%22%3A%22RepeatMasker+by+class%22%2C%22assemblyNames%22%3A%5B%22hg38%22%5D%2C%22adapter%22%3A%7B%22type%22%3A%22BedTabixAdapter%22%2C%22uri%22%3A%22https%3A%2F%2Fjbrowse.org%2Fucsc%2Fhg38%2Frmsk.bed.gz%22%2C%22csi%22%3Atrue%7D%2C%22displays%22%3A%5B%7B%22type%22%3A%22LinearMultiRowFeatureDisplay%22%2C%22rows%22%3A%7B%22field%22%3A%22repClass%22%2C%22sort%22%3A%5B%22SINE%22%2C%22LINE%22%2C%22LTR%22%2C%22DNA%22%2C%22Simple_repeat%22%2C%22Low_complexity%22%5D%2C%22axis%22%3A%7B%22grid%22%3Atrue%7D%2C%22colors%22%3A%7B%22SINE%22%3A%22%23e41a1c%22%2C%22LINE%22%3A%22%23377eb8%22%2C%22LTR%22%3A%22%234daf4a%22%2C%22DNA%22%3A%22%23984ea3%22%2C%22Simple_repeat%22%3A%22%23ff7f00%22%2C%22Low_complexity%22%3A%22%23a65628%22%7D%7D%7D%5D%7D%5D%2C%22views%22%3A%5B%7B%22type%22%3A%22LinearGenomeView%22%2C%22assembly%22%3A%22hg38%22%2C%22loc%22%3A%22chr17%3A45%2C700%2C000-45%2C750%2C000%22%2C%22tracks%22%3A%5B%22rmsk_hg38_rows%22%5D%7D%5D%7D&sessionName=JBrowse+docs%3A+RepeatMasker+by+classThis view in a notebook, with jbrowse-anywidget. Swap a track's uri for your file.
from jbrowse_anywidget import JBrowseApp, fetch_hub
config = fetch_hub("https://jbrowse.org/ucsc/hg38/config.json")
app = JBrowseApp(
assemblies=config["assemblies"],
tracks=config.get("tracks", []) + [
{
"type": "FeatureTrack",
"trackId": "rmsk_hg38_rows",
"name": "RepeatMasker by class",
"assemblyNames": [
"hg38"
],
"adapter": {
"type": "BedTabixAdapter",
"uri": "https://jbrowse.org/ucsc/hg38/rmsk.bed.gz",
"csi": True
},
"displays": [
{
"type": "LinearMultiRowFeatureDisplay",
"rows": {
"field": "repClass",
"sort": [
"SINE",
"LINE",
"LTR",
"DNA",
"Simple_repeat",
"Low_complexity"
],
"axis": {
"grid": True
},
"colors": {
"SINE": "#e41a1c",
"LINE": "#377eb8",
"LTR": "#4daf4a",
"DNA": "#984ea3",
"Simple_repeat": "#ff7f00",
"Low_complexity": "#a65628"
}
}
}
]
}
],
views=[
{
"type": "LinearGenomeView",
"assembly": "hg38",
"loc": "chr17:45,700,000-45,750,000",
"tracks": [
"rmsk_hg38_rows"
]
}
],
)
app # display the widgetThis view in your React 19 app, with @jbrowse/react-app2 and @jbrowse/core. Add your own files to tracks.
import { use } from 'react'
import { JBrowse } from '@jbrowse/react-app2'
import '@jbrowse/react-app2/styles.css'
import { addRelativeUris } from '@jbrowse/core/util/addRelativeUris'
const configUrl = "https://jbrowse.org/ucsc/hg38/config.json"
const configPromise = fetch(configUrl)
.then(res => res.json())
.then(config => {
addRelativeUris(config, new URL(configUrl))
return config
})
export default function Figure() {
const config = use(configPromise)
return (
<JBrowse
assemblies={config.assemblies}
tracks={[...(config.tracks ?? []), ...[
{
"type": "FeatureTrack",
"trackId": "rmsk_hg38_rows",
"name": "RepeatMasker by class",
"assemblyNames": [
"hg38"
],
"adapter": {
"type": "BedTabixAdapter",
"uri": "https://jbrowse.org/ucsc/hg38/rmsk.bed.gz",
"csi": true
},
"displays": [
{
"type": "LinearMultiRowFeatureDisplay",
"rows": {
"field": "repClass",
"sort": [
"SINE",
"LINE",
"LTR",
"DNA",
"Simple_repeat",
"Low_complexity"
],
"axis": {
"grid": true
},
"colors": {
"SINE": "#e41a1c",
"LINE": "#377eb8",
"LTR": "#4daf4a",
"DNA": "#984ea3",
"Simple_repeat": "#ff7f00",
"Low_complexity": "#a65628"
}
}
}
]
}
]]}
aggregateTextSearchAdapters={config.aggregateTextSearchAdapters}
configuration={config.configuration}
views={[
{
"type": "LinearGenomeView",
"assembly": "hg38",
"loc": "chr17:45,700,000-45,750,000",
"tracks": [
"rmsk_hg38_rows"
]
}
]}
/>
)
}jbrowse-img draws this view as an SVG, no browser needed. Swap --config or a track for your own.
cat > tracks.json <<'JSON'
[
{
"type": "FeatureTrack",
"trackId": "rmsk_hg38_rows",
"name": "RepeatMasker by class",
"assemblyNames": [
"hg38"
],
"adapter": {
"type": "BedTabixAdapter",
"uri": "https://jbrowse.org/ucsc/hg38/rmsk.bed.gz",
"csi": true
},
"displays": [
{
"type": "LinearMultiRowFeatureDisplay",
"rows": {
"field": "repClass",
"sort": [
"SINE",
"LINE",
"LTR",
"DNA",
"Simple_repeat",
"Low_complexity"
],
"axis": {
"grid": true
},
"colors": {
"SINE": "#e41a1c",
"LINE": "#377eb8",
"LTR": "#4daf4a",
"DNA": "#984ea3",
"Simple_repeat": "#ff7f00",
"Low_complexity": "#a65628"
}
}
}
]
}
]
JSON
npx @jbrowse/img --config https://jbrowse.org/ucsc/hg38/config.json \
--trackConfigs tracks.json \
--assembly hg38 --loc chr17:45,700,000-45,750,000 \
--track rmsk_hg38_rows \
--out figure.svgFor a PNG, --out figure.png (needs rsvg-convert: apt install librsvg2-bin, brew install librsvg).
The rows come from the classes in the file, so a class rows.colors leaves out
still gets a row, as Unknown does in the figures, painted in the default block
color. scale.unknown in rows.color sets one color for every unnamed row.
The track opens with the first display listed, so the multi-row entry makes rows
the default. A bare { "type": "LinearBasicDisplay" } ahead of it keeps the
packed form as the default and leaves the rows one menu click away.
Serving your own RepeatMasker output
A hub's RepeatMasker BED has the repClass column the display partitions on.
RepeatMasker's .out file instead has a single class/family field, LINE/L1
for a repeat with both and a bare Simple_repeat for one whose family is its
class. Splitting that field in two, under a header naming the columns, turns the
.out into a file of the same shape as the one the track above reads:
# the header names the columns, so `rows: "repClass"` can name a column
# outside the BED spec
{
printf '#genoName\tgenoStart\tgenoEnd\tname\tstrand\trepFamily\trepClass\tswScore\tmilliDiv\n'
awk 'BEGIN { OFS = "\t" }
# a data row is the one starting with a bare integer, its SW score
$1 ~ /^[0-9]+$/ {
# "LINE/L1" splits; "Simple_repeat" does not, and UCSC repeats the class
# as the family for exactly those rows. Positions are 1-based inclusive
# and the strand column spells minus "C".
n = split($11, cf, "/")
print $5, $6 - 1, $7, $10, ($9 == "C") ? "-" : "+", (n > 1) ? cf[2] : cf[1], cf[1], $1, int($2 * 10 + 0.5)
}' repeats.out
} > rmsk.bed
# `sort-bed` is `sort -k1,1 -k2,2n` under LC_ALL=C, header kept on top
jbrowse sort-bed rmsk.bed | bgzip > rmsk.bed.gz
tabix -f -C -p bed rmsk.bed.gzServe rmsk.bed.gz and its .csi together. In the track config above, put its
path or URL in uri and your assembly's name in assemblyNames.
Reproduce it end to end
build_repeatmasker_classes.sh
turns your RepeatMasker .out into a runnable JBrowse. The script:
- Splits the
.out'sclass/familyfield into a class column and a family column. - Writes a BED whose header names the columns UCSC's does, then bgzips and indexes it.
- Downloads JBrowse and adds the track with the multi-row display selected.
curl -fO https://raw.githubusercontent.com/GMOD/jbrowse-components/main/scripts/build_repeatmasker_classes.sh
bash build_repeatmasker_classes.sh genome.fa repeats.out # builds ./repeatmasker_build/jbrowse2
npx --yes serve repeatmasker_build/jbrowse2 # then open the printed URLgenome.fa is the FASTA RepeatMasker was run against and repeats.out is its
.out; either may be gzipped.
See also
- Tutorial: Basic usage of genomes.jbrowse.org
- Tutorial: ChromHMM chromatin states
- User guide: Multi-row feature track
- Docs: Cookbook
Feedback on this tutorial is welcome: contact us.