GPU displays

TL;DR: Build a display that renders via WebGPU/WebGL2 with a required Canvas2D fallback: define data types, write a .slang shader, implement a GPU and a Canvas2D renderer behind one factory, wire an MST model with installUpload, and render through DisplayChrome.

Architecture overview#

JBrowse GPU displays follow a three-layer model:

The model keeps two autoruns running at all times (owned by RenderLifecycleMixin, installed by installUpload):

• One upload autorun fires when any rpcDataMap entry or the backend changes; it diffs the map against what it last sent and calls backend.uploadRegion() only for regions that moved. That diff keeps a streaming whole-genome fetch at O(N) uploads instead of O(N²).
• The render autorun fires when renderTick bumps (after every upload) or when frame-level state like scroll position changes; it calls backend.renderBlocks().

The backend is a HAL (Hardware Abstraction Layer) that dispatches to WebGPU, WebGL2, or Canvas2D at runtime. Your renderer talks to the HAL, never to WebGPU or WebGL2 directly. See the architecture spec for the full lifecycle and packages/render-core/CLAUDE.md for HAL invariants.

For real references, plugins/gwas/src/LinearManhattanDisplay/ is the simplest per-region streamed case. plugins/canvas/src/LinearBasicDisplay/ is the fullest (four shader passes) but uses the whole-map laidOutDataMap form for cross-region layout, so start from Manhattan when your regions are independent. Plotting features in a custom display lists the rest.

Files to create#

The same example-plugins/score-example/ the Canvas2D guide builds, with the [GPU only] rows added:

src/
  index.ts                       the plugin class; installs the display, the RPC method and the feature panel
  LinearScoreDisplay/
    configSchema.ts              config slots (color, scoreColumn)
    index.ts                     registers the display type
    model.ts                     MST model: rpcDataMap, renderState, fetchNeeded, startRenderingBackend
    components/
      Canvas2DScoreRenderer.ts   extends Canvas2DPerRegionRenderingBackend; the SVG-export path too
      GpuScoreRenderer.ts        [GPU only] extends GpuPerRegionRenderingBackend; packs instances, writes uniforms
      ScoreDisplayComponent.tsx  React: DisplayChrome wrapping the canvas
      ScoreRendererFactory.ts    the factory DisplayChrome calls; picks GPU or Canvas2D
      drawScore.ts               pure draw function over a Ctx2D
      scoreTypes.ts              ScoreRenderState and the backend type
      shaders/
        score.slang              [GPU only] vertex + fragment for one pass; compiled by gen:shaders
  ScoreFeaturePanel/
    index.tsx                    adds a panel to the feature details widget
  ScoreRPC/
    GetScoreData.ts              worker: fetch features from the adapter, then pack
    buildScoreResult.ts          pure packer, unit-tested without a worker
    index.ts                     registers the RPC method
    rpcTypes.ts                  ScoreRegionData and the RPC arg types

Step 1: Define data types#

import type { ScoreRegionData } from '../../ScoreRPC/rpcTypes.ts'
import type { PerRegionRenderingBackend } from '@jbrowse/render-core/perRegionRenderingBackend'

// Recomputed cheaply every frame without fetching. Carries the canvas
// dimensions (required by the base class to size the backing store) plus the
// one setting the draw path reads.
export interface ScoreRenderState {
  canvasWidth: number
  canvasHeight: number
  color: string
}

export type ScoreRenderingBackend = PerRegionRenderingBackend<
  ScoreRegionData,
  ScoreRenderState
>

Step 2: Write the shaders#

Create a .slang file. JBrowse uses a Slang-derived shader language that compiles to both WGSL (WebGPU) and GLSL (WebGL2). Modules are referenced by bare name (import hpmath;), not file path; the shared helpers live in packages/render-core/src/shaders/ (hpmath for the high-precision genomic→pixel transform, colorPack for unpacking packed colors). The example declares its uniforms inline; if several passes share a struct, put it in a sibling module (scoreUniforms.slang, starting module scoreUniforms; with a public struct).

// Score display: one box per feature. The box spans start->end horizontally and
// its height is score (0..1) x canvasHeight, grown up from the bottom. A single
// uniform ABGR color fills every box. This is the minimal per-region GPU pass
// used by the "GPU displays" developer guide.
//! targets: wgsl, glsl

import hpmath;
import colorPack;

public static const uint VERTS_PER_INSTANCE = 6u;

struct ScoreInstance {
  uint  startBp : ATTR0;
  uint  endBp   : ATTR1;
  float score   : ATTR2;
};

struct Uniforms {
  // hpmath genomic->clip transform (hi, lo, +/-clippedLengthBp)
  float3 bpRangeX;
  float  zero;
  float  canvasWidth;
  float  canvasHeight;
  uint   color;
};
[[vk::binding(1, 0)]] ConstantBuffer<Uniforms> u;

float bpToClipX(uint bp, Uniforms u) {
  return hpToClipX(hpSplitUint(bp), u.bpRangeX, u.zero);
}

struct VsOut {
  float4 position : SV_Position;
  float4 color    : COLOR0;
};

[shader("vertex")]
VsOut vs_main(ScoreInstance inst, uint vid : SV_VertexID) {
  // quadLocal maps the 6 vertices to the corners of a unit box: x/y each 0 or 1.
  float2 local = quadLocal(vid);

  float x1 = bpToClipX(inst.startBp, u);
  float x2 = bpToClipX(inst.endBp, u);
  // widen a sub-pixel feature so a 1bp box still paints (reversal-safe: reversal
  // is baked into bpRangeX's negated length, so x2 < x1 on reversed blocks).
  x2 = extendToMinWidthX(x1, x2, 1.0, u.canvasWidth);
  float x = local.x < 0.5 ? x1 : x2;

  float barHeightPx = clamp(inst.score, 0.0, 1.0) * u.canvasHeight;
  // local.y: 0 = top of the box, 1 = bottom (canvas bottom edge).
  float yPx = (u.canvasHeight - barHeightPx) + local.y * barHeightPx;

  VsOut o;
  o.position = float4(x, yPxToClipY(yPx, u.canvasHeight), 0.0, 1.0);
  o.color = unpackRGBA(u.color);
  return o;
}

[shader("fragment")]
float4 fs_main(VsOut fragIn) : SV_Target {
  return fragIn.color;
}

Run the codegen after every edit. In your own repo:

pnpm add -D @jbrowse/shader-tools
npx jbrowse-build-shaders

It scans from the project root for *.slang, fetches a pinned slangc on first use, and writes each *.generated.ts next to its source (hpmath / colorPack resolve from your installed @jbrowse/render-core). Inside this repo the same tool is pnpm gen:shaders.

One .slang file with entry points produces up to three modules, and which one you import from decides what your users download. A bundler treats a namespace import (import * as shader from './score.generated.ts') as using every export, so a module is included or excluded whole — whatever the smallest eager consumer of a module wants, the always-loaded chunk pays for all of it.

So a display model reading one threshold reaches for the .consts. module, not the shader module that re-exports it.

What lands in score.generated.ts, and what a plugin imports from it. It re-exports the other two modules, so the table below is the union of all three:

Only what a given shader needs is emitted: no compute entry point for a render-only shader, and an *_OFFSET_* map only for the typed-array views its fields actually take.

Genomic positions travel as absolute uint attributes; convert them with the bpToClipX wrapper above and nothing else. The bpHi/bpLo split it hides exists because float32 can't represent every base past ~16.7 Mbp, and it stays confined to that one line; in TypeScript outside uniform writes, use plain bp - bpStart.

Step 3: GPU renderer#

The base class GpuPerRegionRenderingBackend owns the per-frame scaffold: resize, beginFrame/endFrame, and the per-block scissor/viewport clip. It also owns the upload, driven by the regionPasses you declare — each pass carries the function that packs its instance buffer, so there is one place a pass is named and no instance count to keep in agreement with the bytes.

You implement one method: drawRegion, which writes uniforms and issues the draw pass for one already-clipped block.

import { cssColorToABGR } from '@jbrowse/core/util/colorBits'
import { writeBpRangeUniforms } from '@jbrowse/render-core/blockClipUtils'
import { GpuPerRegionRenderingBackend } from '@jbrowse/render-core/perRegionRenderingBackend'
import { slangPass } from '@jbrowse/render-core/slangPass'

import * as shader from './shaders/score.generated.ts'

import type { ScoreRegionData } from '../../ScoreRPC/rpcTypes.ts'
import type { ScoreRenderState } from './scoreTypes.ts'
import type { BlockClipResult } from '@jbrowse/render-core/blockClipUtils'
import type { GpuHal } from '@jbrowse/render-core/hal'
import type { RenderBlock } from '@jbrowse/render-core/renderBlock'

const PASS = 'score'
const U = shader.UNIFORM_OFFSET_F32
const UU = shader.UNIFORM_OFFSET_U32

// A pass is its shader plus the function that fills its instance buffer. Six
// vertices per instance = two triangles, so the boxes need a triangle-list
// topology. Exported so the factory can hand the pass list to the HAL.
//
// You write no upload: the base class packs every pass in `regionPasses` and
// hands the bytes to the HAL, taking the instance count from the buffer's own
// length. Nothing to keep in agreement, and an empty pack releases the buffer.
export const SCORE_PASSES = [
  {
    ...slangPass({ id: PASS, mod: shader }),
    // the generated packInstances interleaves the parallel arrays into the
    // GL_ATTRIBUTES layout, no manual DataView offsets
    pack: (data: ScoreRegionData) =>
      shader.packInstances(
        { startBp: data.starts, endBp: data.ends, score: data.scores },
        data.numFeatures,
      ),
  },
]

export class GpuScoreRenderer extends GpuPerRegionRenderingBackend<
  ScoreRegionData,
  ScoreRenderState
> {
  private uniformF32: Float32Array
  private uniformU32: Uint32Array
  protected regionPasses = SCORE_PASSES

  constructor(hal: GpuHal) {
    // the base allocates the reusable this.uniformData scratch buffer
    super(hal, shader.UNIFORMS_SIZE_BYTES)
    this.uniformF32 = new Float32Array(this.uniformData)
    this.uniformU32 = new Uint32Array(this.uniformData)
  }

  protected drawRegion(
    block: RenderBlock,
    clip: BlockClipResult,
    _region: ScoreRegionData,
    state: ScoreRenderState,
  ) {
    // fills the hp-split genomic->clip transform (and negates it on reversal)
    writeBpRangeUniforms(this.uniformF32, U.bpRangeX, clip, block.reversed)
    this.uniformF32[U.zero] = 0
    this.uniformF32[U.canvasWidth] = state.canvasWidth
    this.uniformF32[U.canvasHeight] = state.canvasHeight
    this.uniformU32[UU.color] = cssColorToABGR(state.color)
    this.hal.writeUniforms(this.uniformData)
    this.hal.drawPass(PASS, block.displayedRegionIndex)
  }
}

For a real, complete example of this shape see plugins/gwas/src/LinearManhattanDisplay/GpuManhattanRenderer.ts.

Step 4: Canvas2D renderer (required)#

Implement the same interface using ctx.fillRect etc. Canvas2D is the floor every display must ship: SVG export runs the Canvas2D path, and the GPU shader is the optional accelerator layered on top. This renderer also runs when WebGPU and WebGL2 are both unavailable.

It is written once and unchanged here: Plotting features, Step 4 builds drawScore.ts and Canvas2DScoreRenderer.ts in full. The only difference on this path is the factory in Step 5 below, which now has a GPU backend to prefer.

Step 5: RenderingBackend factory#

import { createRenderingBackend } from '@jbrowse/render-core/createRenderingBackend'

import { Canvas2DScoreRenderer } from './Canvas2DScoreRenderer.ts'
import { GpuScoreRenderer, SCORE_PASSES } from './GpuScoreRenderer.ts'
import { UNIFORMS_SIZE_BYTES } from './shaders/score.generated.ts'

import type { ScoreRenderingBackend } from './scoreTypes.ts'

// createRenderingBackend tries the GPU HAL first (WebGPU, then WebGL2) and
// falls back to Canvas2D when no GPU device is available. It's async (it awaits
// device creation), so this returns a Promise; DisplayChrome awaits it.
export function ScoreRenderer(canvas: HTMLCanvasElement) {
  return createRenderingBackend<ScoreRenderingBackend>(canvas, {
    passes: SCORE_PASSES,
    uniformByteSize: UNIFORMS_SIZE_BYTES,
    createGpuBackend: hal => new GpuScoreRenderer(hal),
    createCanvas2DBackend: c => new Canvas2DScoreRenderer(c),
  })
}

Step 6: MST model#

Compose MultiRegionDisplayMixin (which includes RenderLifecycleMixin and the fetch autoruns), store the worker output in an rpcDataMap, and wire the render lifecycle with installUpload. This is the per-region streamed upload pattern from the architecture spec's upload patterns, the right shape when each region's data is independent (no cross-region layout coupling).

The model is identical to the Canvas2D one: Plotting features, Step 3 builds it in full (rpcDataMap, rpcProps, renderState, fetchNeeded), and none of it changes when a shader appears. The one action worth reading again here is the render wiring:

// called once by DisplayChrome when the backend is created. Streams each
// region into the backend and draws every frame from renderState. This is
// the only part of the model that knows a backend exists, and it is
// identical whether that backend is the GPU or the Canvas2D one.
startRenderingBackend(backend: ScoreRenderingBackend) {
  installUpload(self, backend, {
    cells: () => self.rpcDataMap,
    render: (b, regions) => {
      if (regions.size === 0) {
        return false // keep the loading overlay up until data lands
      }
      b.renderBlocks(self.renderBlocks, regions, self.renderState)
      return true
    },
  })
},

One installer wires the render lifecycle for you. installUpload remembers what it last sent for each key and uploads only what changed, so N regions streaming in cost N uploads rather than N². The key is whatever your map is keyed by: a displayedRegionIndex here, a sibling display's sharedBackendKey on a canvas several displays share, or a slot name (oneCell('data', payload)) on a display that holds one payload for the whole view. Only displays that lay features into Y-rows across regions (LinearBasicDisplay, alignments) hand it a whole-map computed instead of the raw rpcDataMap.

An encode that needs more than the region's own data — a color scheme, a scale — declares it as inputs, and a change there re-encodes every loaded region. Reading it inside encode instead does not work: the helper invalidates on inputs and on the region's own data, and on nothing else.

Three settings buckets (see the rpcProps() / gpuProps() pattern):

• rpcProps() refetches in the worker, so scroll and zoom must stay out of it.
• renderState is recomputed per frame and refetches nothing.
• gpuProps() takes a setting that needs a main-thread buffer re-encode but no refetch — a color, a scale.

Step 7: React component#

The same component the Canvas2D path uses, unchanged — Plotting features, Step 5 shows it in full. DisplayChrome creates the HAL via useRenderingBackend, calls model.startRenderingBackend(backend) once the backend is live, and hands back the canvasRef to attach to your <canvas>. Nothing in it knows whether the factory resolved to a GPU or a Canvas2D backend, and its render-prop child keeps it agnostic to how many canvases a display draws.

Step 8: Register the display#

In your plugin's install(), register the display type pointing at your model factory and React component (see Custom track and display types for the full registration pattern).

The WebGL2 context ceiling#

One display owns one backend canvas, and WebGL2Hal takes its own WebGL2 context with no pooling. Browsers cap how many a page may hold — 16 on Chrome — and past that, eviction and re-acquisition cascade and wedge the main thread rather than degrading. A single ordinary view reaches it: 17 GPU tracks on one linear genome view. So budget contexts as one per open GPU track.

Chromosomes are free on this axis — a whole-genome view of one track is still one canvas, with one GPU buffer per displayedRegionIndex.

View-level lazy mount and bounded auto-recovery in useRenderingBackend bound the problem; tracks inside a mounted view are not virtualized, so the ceiling stays reachable.

WebGPU has no per-canvas cap, because gpuDevice.ts shares one device across displays. The trade is its mirror image: one device.lost takes down every display at once. That makes triage easy — one track broke points at WebGL2, every track broke at once points at WebGPU.

What each backend refuses to allocate is Memory and retention.

Key invariants#

• All worker output uses absolute genomic uint32 coordinates, not region-relative. float32 cannot hold 3 Gbp; use uint32 for positions crossing the worker boundary.
• rpcProps must not contain fetch results. SettingsInvalidate watches rpcProps(); putting derived cell data there creates an infinite fetch loop.
• Shader uniforms use CSS pixels: don't scale canvas_width/canvas_height by devicePixelRatio before writing them.
• Never edit *.generated.ts. Always edit .slang and run pnpm gen:shaders; CI enforces this with git diff --exit-code.
• Renderers stay stateless. Don't cache per-region data on the renderer class (private regions = new Map()); the model's rpcDataMap is the single source of truth and is passed into renderBlocks. Delegate GPU buffer lifecycle to hal.pruneRegions(active).
• Render the canvas through DisplayChrome, never by calling useRenderingBackend in your own component.

The What NOT to do section of the architecture spec is the full quick-scan list.

See also#

• Data flow, end to end
• Optimizations
• Memory and retention
• Data fetching pipeline
• RPC and worker system
• Custom track and display types
• SVG export
• GPU_CONTEXT_BUDGET.md — the WebGL2 context ceiling one display spends against, what reaches it, and the four fixes already measured and eliminated