Low-mappability regions (SMN)
The tutorials target the JBrowse v5 beta, and the v4.3.0 release
on the download page lacks some of what they show. To install the
beta of JBrowse Web, run npm install -g @jbrowse/cli@next, then
jbrowse create jbrowse2 --branch v5.0.0-beta.20. Desktop beta builds for Linux and
macOS are on the releases page;
the Windows build waits on code signing.
A pileup looks the same whether its reads belong at a locus or merely landed there. Where two copies of a sequence are near-identical, an aligner cannot tell which copy a read came from, so the reads, the coverage and any call made from them are unreliable there. We check a locus for this at the SMN1/SMN2 duplication on chromosome 5. We:
- load hg38's Umap mappability as a quantitative track drawn at its bin minimum, beside gnomAD coverage, GIAB's problematic regions and long-read SV calls
- add a public 1000 Genomes CRAM as an alignments track colored by mapping quality
- open a synteny view of UCSC's hg38-to-T2T-CHM13 liftOver chains over the duplication
- repeat the check at a locus of your own beside a control window
Prerequisites
- a JBrowse to open the hosted hg38 config in (Web or Desktop)
samtoolsandbigWigToBedGraph, for the counts at your own locus, and the tools listed under Reproduce it end to end for the script
The data
The hosted tracks are files already wired into the genomes.jbrowse.org hg38 config. The reads come from the 1000 Genomes high-coverage short-read release and its ONT long-read release (Gustafson et al. 2024).
You don't need these files to follow the page: the track configs below read the hosted results. The build script streams them to take its counts. Applying the mappability check to another locus runs the same check on your own locus and reads.
| What it is | File |
|---|---|
| GM18501 ONT long reads aligned to GRCh38, which the script counts; the bucket serves no CORS headers, so it cannot load as a track | GM18501-ONT-hg38-R9-LSK110-guppy-sup-5mC.phased.bam |
| The same sample aligned to T2T-CHM13 | GM18501-ONT-chm13-R9-LSK110-guppy-sup-5mC.phased.bam |
Read by URL (no download needed)
| What it is | File |
|---|---|
| Umap k100 multi-read mappability | k100.Umap.MultiTrackMappability.bw |
| GnomAD v3 mean genome coverage | gnomad.coverage.mean.bw |
| GIAB's low-mappability and segmental-duplication regions | alllowmapandsegdupregions.bb |
| ENCODE's blacklist | encBlacklist.bb |
| The GRC's exclusion list | grcExclusions.bb |
| UCSC's problematic-regions comments | comments.bb |
| The DGV merged CNV catalogue | dgvMerged.bb |
| The 1000 Genomes ONT long-read SV callset | 1kgOnt.bb |
| NA12878 at 30x, GRCh38, the read track added to the session | NA12878.final.cram |
SMN1 and SMN2: a near-identical gene duplication
SMN1 and SMN2 sit about 900 kb apart on chromosome 5 and are roughly 99.9%
identical across their ~28 kb. Spinal muscular atrophy turns on the copy number
of SMN1, so which copy a read came from is the clinical question. An aligner
given a 150 bp read from either copy has two equally good places to put it and
reports mapping quality 0 (MAPQ 0). MAPQ is
-10 log10 Pr{mapping position is wrong}
(SAM specification), so MAPQ 0
means the chosen position is about as likely wrong as right.
Mappability, coverage and read tracks across the SMN1 and SMN2 region
Turn these tracks on from the track selector of the hosted hg38 config at genomes.jbrowse.org, or add them with the configs below to any JBrowse that has the hg38 assembly. UCSC publishes the mappability and problematic-region tracks for hg38 only:
- Multi-read mappability - Umap M100, the fraction of overlapping 100-mers
at each position that are unique in the genome, computed from the reference
alone. The file omits positions with no unique 100-mer, so the track goes
blank there.
- Resolution → Bin summary → Minimum, with Show min and max unchecked, draws the worst position in each bin
- Y axis... → Range pins the axis at 0 to 1
- In the 2.5 Mb frame, at about a kilobase per pixel, even Minimum sits on the floor everywhere, so read this track in the narrower read view
- gnomAD v3 Genome Coverage - Mean Coverage, averaged over tens of thousands of genomes. gnomAD drops reads that do not place uniquely before averaging, so this track falls wherever the Umap track is blank.
- GIAB Problematic Regions - LowMap+SegDup, the sequence GIAB leaves out of its benchmark.
- Long-read SVs - 1KG Vienna ONT SVs, the long-read callset.
- NCBI RefSeq - RefSeq Curated, the gene track the figures draw.
The Umap config holds the Minimum score mode and the 0 to 1 axis:
Goes in the tracks array of config.json. See Tracks.
{
"type": "QuantitativeTrack",
"trackId": "hg38-umap100Quantitative",
"name": "Multi-read mappability - Umap M100",
"assemblyNames": ["hg38"],
"adapter": {
"type": "BigWigAdapter",
"uri": "https://hgdownload.soe.ucsc.edu/gbdb/hg38/hoffmanMappability/k100.Umap.MultiTrackMappability.bw"
},
"displays": [
{
"type": "LinearWiggleDisplay",
"displayId": "hg38-umap100Quantitative-LinearWiggleDisplay",
"mark": { "extent": "none" },
"encoding": {
"y": {
"field": "minScore",
"scale": { "domainMin": 0, "domainMax": 1 }
}
}
}
]
}jbrowse add-track-json '{
"type": "QuantitativeTrack",
"trackId": "hg38-umap100Quantitative",
"name": "Multi-read mappability - Umap M100",
"assemblyNames": ["hg38"],
"adapter": {
"type": "BigWigAdapter",
"uri": "https://hgdownload.soe.ucsc.edu/gbdb/hg38/hoffmanMappability/k100.Umap.MultiTrackMappability.bw"
},
"displays": [
{
"type": "LinearWiggleDisplay",
"displayId": "hg38-umap100Quantitative-LinearWiggleDisplay",
"mark": { "extent": "none" },
"encoding": {
"y": {
"field": "minScore",
"scale": { "domainMin": 0, "domainMax": 1 }
}
}
}
]
}'In JBrowse Desktop, or in any running JBrowse Web session, open a view on this track’s assembly, then File → Open track..., choose Add track from pasted JSON, and paste:
{
"type": "QuantitativeTrack",
"trackId": "hg38-umap100Quantitative",
"name": "Multi-read mappability - Umap M100",
"assemblyNames": ["hg38"],
"adapter": {
"type": "BigWigAdapter",
"uri": "https://hgdownload.soe.ucsc.edu/gbdb/hg38/hoffmanMappability/k100.Umap.MultiTrackMappability.bw"
},
"displays": [
{
"type": "LinearWiggleDisplay",
"displayId": "hg38-umap100Quantitative-LinearWiggleDisplay",
"mark": { "extent": "none" },
"encoding": {
"y": {
"field": "minScore",
"scale": { "domainMin": 0, "domainMax": 1 }
}
}
}
]
}Goes in the tracks array of config.json. See Tracks.
{
"trackId": "hg38-gnomad3MeanCoverage",
"name": "gnomAD v3 Genome Coverage - Mean Coverage",
"uri": "https://hgdownload.soe.ucsc.edu/gbdb/hg38/gnomAD/coverage/v3-genome/gnomad.coverage.mean.bw",
"assemblyNames": ["hg38"]
}jbrowse add-track https://hgdownload.soe.ucsc.edu/gbdb/hg38/gnomAD/coverage/v3-genome/gnomad.coverage.mean.bw \
--trackId hg38-gnomad3MeanCoverage \
--name "gnomAD v3 Genome Coverage - Mean Coverage" \
--assemblyNames hg38In JBrowse Desktop, or in any running JBrowse Web session, open a view on this track’s assembly, then File → Open track... and, in Add a track from file or URL, enter:
- Main file:
https://hgdownload.soe.ucsc.edu/gbdb/hg38/gnomAD/coverage/v3-genome/gnomad.coverage.mean.bw
Click Next. JBrowse reads the adapter and track type off the file name. Then fill in:
- Track name:
gnomAD v3 Genome Coverage - Mean Coverage - Assembly:
hg38
Click Add.
Goes in the tracks array of config.json. See Tracks.
{
"trackId": "hg38-alllowmapandsegdupregions",
"name": "GIAB Problematic Regions - LowMap+SegDup",
"uri": "https://hgdownload.soe.ucsc.edu/gbdb/hg38/problematic/GIAB/alllowmapandsegdupregions.bb",
"assemblyNames": ["hg38"]
}jbrowse add-track https://hgdownload.soe.ucsc.edu/gbdb/hg38/problematic/GIAB/alllowmapandsegdupregions.bb \
--trackId hg38-alllowmapandsegdupregions \
--name "GIAB Problematic Regions - LowMap+SegDup" \
--assemblyNames hg38In JBrowse Desktop, or in any running JBrowse Web session, open a view on this track’s assembly, then File → Open track... and, in Add a track from file or URL, enter:
- Main file:
https://hgdownload.soe.ucsc.edu/gbdb/hg38/problematic/GIAB/alllowmapandsegdupregions.bb
Click Next. JBrowse reads the adapter and track type off the file name. Then fill in:
- Track name:
GIAB Problematic Regions - LowMap+SegDup - Assembly:
hg38
Click Add.
Goes in the tracks array of config.json. See Tracks.
{
"trackId": "hg38-lrSv1kgOnt",
"name": "Long-read SVs - 1KG Vienna ONT SVs",
"uri": "https://hgdownload.soe.ucsc.edu/gbdb/hg38/lrSv/1kgOnt.bb",
"assemblyNames": ["hg38"]
}jbrowse add-track https://hgdownload.soe.ucsc.edu/gbdb/hg38/lrSv/1kgOnt.bb \
--trackId hg38-lrSv1kgOnt \
--name "Long-read SVs - 1KG Vienna ONT SVs" \
--assemblyNames hg38In JBrowse Desktop, or in any running JBrowse Web session, open a view on this track’s assembly, then File → Open track... and, in Add a track from file or URL, enter:
- Main file:
https://hgdownload.soe.ucsc.edu/gbdb/hg38/lrSv/1kgOnt.bb
Click Next. JBrowse reads the adapter and track type off the file name. Then fill in:
- Track name:
Long-read SVs - 1KG Vienna ONT SVs - Assembly:
hg38
Click Add.
Then add the NA12878 reads. The CRAM needs its .crai beside it and decodes
against hg38; for your own sample, swap the uri. In
grammar-of-graphics terms, the track's
encoding.color is a color scale on the mapq field:
Goes in the tracks array of config.json. See Tracks.
{
"trackId": "na12878_qc_reads",
"name": "NA12878, 30x Illumina (1000 Genomes)",
"uri": "https://s3.amazonaws.com/1000genomes/1000G_2504_high_coverage/data/ERR3239334/NA12878.final.cram",
"assemblyNames": ["hg38"],
"displayDefaults": { "encoding": { "color": "mapq" } }
}jbrowse add-track https://s3.amazonaws.com/1000genomes/1000G_2504_high_coverage/data/ERR3239334/NA12878.final.cram \
--trackId na12878_qc_reads \
--name "NA12878, 30x Illumina (1000 Genomes)" \
--assemblyNames hg38 \
--displayDefaults '{"encoding":{"color":"mapq"}}'In JBrowse Desktop, or in any running JBrowse Web session, open a view on this track’s assembly, then File → Open track..., choose Add track from pasted JSON, and paste:
{
"trackId": "na12878_qc_reads",
"name": "NA12878, 30x Illumina (1000 Genomes)",
"uri": "https://s3.amazonaws.com/1000genomes/1000G_2504_high_coverage/data/ERR3239334/NA12878.final.cram",
"assemblyNames": ["hg38"],
"displayDefaults": { "encoding": { "color": "mapq" } }
}Open this track in JBrowse Web ↗
Needs JBrowse Desktop 5.0 or newer. If nothing opens, copy the link and paste it into Desktop's Open JBrowse Web link..., on the start screen or under File → Session.
https://jbrowse.org/code/jb2/main/?config=https%3A%2F%2Fjbrowse.org%2Fucsc%2Fhg38%2Fconfig.json&session=spec-%7B%22sessionTracks%22%3A%5B%7B%22trackId%22%3A%22na12878_qc_reads%22%2C%22name%22%3A%22NA12878%2C+30x+Illumina+%281000+Genomes%29%22%2C%22uri%22%3A%22https%3A%2F%2Fs3.amazonaws.com%2F1000genomes%2F1000G_2504_high_coverage%2Fdata%2FERR3239334%2FNA12878.final.cram%22%2C%22assemblyNames%22%3A%5B%22hg38%22%5D%2C%22displayDefaults%22%3A%7B%22encoding%22%3A%7B%22color%22%3A%22mapq%22%7D%7D%7D%5D%2C%22views%22%3A%5B%7B%22type%22%3A%22LinearGenomeView%22%2C%22assembly%22%3A%22hg38%22%2C%22loc%22%3A%22chr5%3A70%2C889%2C000-70%2C989%2C000%22%2C%22tracks%22%3A%5B%22na12878_qc_reads%22%5D%7D%5D%7D&sessionName=JBrowse+docs%3A+NA12878%2C+30x+Illumina+%281000+Genomes%29Open chr5:69,200,000-71,700,000 for the whole region, and a second view at
chr5:70,850,000-71,500,000 for the reads.
GIAB flags two long intervals, much larger than the gene, and the gnomAD track stays low across both. The long-read SV track is empty across them. In the read view, the reads stay at MAPQ 0 until well past the end of SMN1, where the Umap track steps up at the same coordinate as the gnomAD coverage.
Zoom the read view to the SMN cassette, chr5:70,889,000-70,989,000.
Dark blue is MAPQ 0; yellow is MAPQ 30 and above. Advanced → Edit plot... on
the reads track shows this encoding.color as text and applies edits live.
Does T2T-CHM13 resolve the SMN duplication?
T2T-CHM13, the telomere-to-telomere assembly, is finished on this chromosome. UCSC's hg38-to-CHM13 liftOver forms several long, overlapping chains over the region, some of them reversed:
tabix https://jbrowse.org/ucsc/hg38/liftOver/hg38ToHs1.over.pif.gz \
tchr5:69200000-71700000A synteny view between the two assemblies draws each chain as a ribbon. The hosted hg38 config has the track and loads hs1 with it. On another JBrowse, load the hs1 assembly and the chain file, whose query is the genome it lifts to:
Goes in the assemblies array of config.json. See Assemblies.
{
"name": "hs1",
"uri": "https://hgdownload.soe.ucsc.edu/goldenPath/hs1/bigZips/hs1.2bit"
}jbrowse add-assembly https://hgdownload.soe.ucsc.edu/goldenPath/hs1/bigZips/hs1.2bit \
--name hs1In JBrowse Desktop, Open new genome on the start screen (or File → Open genome... in a session), then Open from a URL and paste:
https://hgdownload.soe.ucsc.edu/goldenPath/hs1/bigZips/hs1.2bitJBrowse reads the format off the file name. Then fill in:
- Genome name:
hs1
Goes in the tracks array of config.json. See Tracks.
{
"type": "SyntenyTrack",
"trackId": "hg38_to_hs1_liftOver",
"name": "hg38 to Human (hs1) liftOver",
"assemblyNames": ["hs1", "hg38"],
"adapter": {
"type": "PairwiseIndexedPAFAdapter",
"uri": "https://jbrowse.org/ucsc/hg38/liftOver/hg38ToHs1.over.pif.gz",
"csi": true,
"queryAssembly": "hs1",
"targetAssembly": "hg38"
}
}jbrowse add-track-json '{
"type": "SyntenyTrack",
"trackId": "hg38_to_hs1_liftOver",
"name": "hg38 to Human (hs1) liftOver",
"assemblyNames": ["hs1", "hg38"],
"adapter": {
"type": "PairwiseIndexedPAFAdapter",
"uri": "https://jbrowse.org/ucsc/hg38/liftOver/hg38ToHs1.over.pif.gz",
"csi": true,
"queryAssembly": "hs1",
"targetAssembly": "hg38"
}
}'In JBrowse Desktop, or in any running JBrowse Web session, open a view on this track’s assembly, then File → Open track..., choose Add track from pasted JSON, and paste:
{
"type": "SyntenyTrack",
"trackId": "hg38_to_hs1_liftOver",
"name": "hg38 to Human (hs1) liftOver",
"assemblyNames": ["hs1", "hg38"],
"adapter": {
"type": "PairwiseIndexedPAFAdapter",
"uri": "https://jbrowse.org/ucsc/hg38/liftOver/hg38ToHs1.over.pif.gz",
"csi": true,
"queryAssembly": "hs1",
"targetAssembly": "hg38"
}
}That shows it in the linear view. For the synteny view, open Add → Linear synteny view, pick the track under Quick start, and click Launch.
The gene order is the same in both assemblies, SMN2 first and then SMN1. The crossing chains join each GRCh38 copy to both CHM13 copies, so CHM13 has the same duplication, with the two genes closer together, and a read from either copy still has two places to go.
Applying the mappability check to another locus
The same check runs on any hg38 locus and your own reads, read against a control window: depth alone can look the same at a duplicated locus and an ordinary one, and the share of reads at MAPQ 0 separates them.
- Turn on the four tracks above, with the same Umap settings
- Add your reads with the
uriin the read track config swapped for your BAM or CRAM, colored by mapping quality - Open a second window of the same width, from the same sample and outside every flagged interval, beside the first
In numbers, the comparison is three samtools counts per window, with -q a
minimum MAPQ:
samtools view -c "$CRAM" chr5:70,900,000-71,000,000 # every read
samtools view -c -q 1 "$CRAM" chr5:70,900,000-71,000,000 # placed at all
samtools view -c -q 60 "$CRAM" chr5:70,900,000-71,000,000 # placed uniquelyThe Umap track's value over a window is one command over the same bigWig:
# positions with no unique 100-mer are absent from the file, so the share of
# the span that has any value is the number to read
bigWigToBedGraph -chrom=chr5 -start=70049000 -end=70077000 \
k100.Umap.MultiTrackMappability.bw stdout |
awk -v s=70049000 -v e=70077000 \
'{cov += $3 - $2} END {printf "%.1f%% has a value\n", 100 * cov / (e - s)}'Run both on the control window too, at the same width.
Reproduce it end to end
scan_mappability_qc.sh
measures the SMN region from the files the figures draw. It asks whether a read
can be placed at a locus several ways, and reads each answer against a control:
- Over SMN1, SMN2 and two controls, it measures how much of each window the Umap track has a value for, gnomAD's mean depth, and the share of NA12878's reads at MAPQ 0. The near control is a window of the same width at the 5' end of BDP1, past both flagged intervals on the same chromosome, so sample, library and chromosome all match; ACTB on chromosome 7 shows the near control is ordinary.
- It lists which problematic-region tracks flag the region, and bins the gnomAD track across it to find where the depth recovers.
- It counts GM18501's long-read records over SMN1 at MAPQ 0 and MAPQ 60, once on each reference. The release aligned that sample to both with the same pipeline, so the two counts compare.
- It counts each SV catalogue's calls inside the region and in a flank of about the same width on either side, and across chr5 the share of calls whose midpoint falls in a flagged region.
The script needs kent tools (bigWigInfo, bigWigToBedGraph, bigBedToBed),
bedtools, samtools, curl and awk.
curl -fO https://raw.githubusercontent.com/GMOD/jbrowse-components/main/scripts/scan_mappability_qc.sh
bash scan_mappability_qc.shThe script measures a locus added to its LOCI list the same way.
See also
- Tutorial: Structural variants (1000 Genomes)
- Tutorial: Pangenome (HPRC) part 1: the graph's alleles and the haplotypes that have them
- Tutorial: Synteny on genomes.jbrowse.org
- Tutorial: A grammar of graphics over a BAM (NA12878 insert size)
- User guide: Alignments track
Citations
- Li et al. (2009).
The Sequence Alignment/Map format and SAMtools,
and the current
SAM specification, which
defines MAPQ as
-10 log10 Pr{mapping position is wrong}. - Li et al. (2008). Mapping short DNA sequencing reads and calling variants using mapping quality scores, which introduced that estimator.
- Karimzadeh et al. (2018). Umap and Bismap: quantifying genome and methylome mappability, the source of the k100 mappability track.
- Gustafson et al. (2024). High-coverage nanopore sequencing of samples from the 1000 Genomes Project to build a comprehensive catalog of human genetic variation, the source of both the long-read SV callset in the wide figure and the long reads the script counts on GRCh38 and T2T-CHM13.
Feedback on this tutorial is welcome: contact us.