Skip to main content

Codec fixture guide

SpatialData codec fixtures live under test-fixtures/codecs/ and are generated separately from the versioned blobs() fixtures. The reference fixtures are small OME-Zarr/SpatialData images using:

  • imagecodecs_jpeg2k (registered JP2K codec id)
  • experimental.openjph_htj2k (OpenJPH WASM HTJ2K codec id)

Generated stores include:

  • jpeg2k.zarr — small multiscale Mandelbrot ([1,1,1,64,64])
  • mandelbulb.zarr — multi-dimensional Mandelbulb volume ([2,1,8,128,128], HTJ2K, one plane per t/z chunk)
  • htj2k.zarr / htj2k-demo.zarr — HTJ2K smoke and encode-demo stores (when the WASM encoder is available)

Older fixtures may use experimental.imagecodecs_htj2k; the frontend decodes both.

Try it in the browser​

From the repo root:

pnpm test:fixtures:generate:codecs
pnpm --filter @spatialdata/vis dev

Open http://127.0.0.1:5173/codec.

The route uses SpatialCanvas, which activates the bundled codec worker through @spatialdata/vis automatically in browser contexts. Under the hood, vis calls an idempotent ensureCodecWorkers() helper, which uses zarrextra/workers and @fideus-labs/fizarrita with a custom codec worker. That worker registers JP2K and experimental HTJ2K support inside the worker before decode. Use the codec selector on the page to switch between /test-fixtures/codecs/jpeg2k.zarr, /test-fixtures/codecs/mandelbulb.zarr, and /test-fixtures/codecs/htj2k-demo.zarr. When HTJ2K is selected, the demo loads a single htj2k-demo.zarr store containing mandelbrot_lossless, mandelbrot_balanced, and mandelbrot_small image layers (512×512 Mandelbrot, two pyramid levels). Use the SpatialCanvas layer controls to toggle between presets. Manifests are at /test-fixtures/codecs/jpeg2k.manifest.json, /test-fixtures/codecs/mandelbulb.manifest.json, and /test-fixtures/codecs/htj2k-encode-demo.manifest.json.

The vis dev script starts the fixture server on port 38473 and proxies /test-fixtures through Vite on 5173. Override the fixture server port with SPATIALDATA_FIXTURE_PORT if needed.

Python reference writer​

The publishable package (spatialdata-js-util) exposes recompression only. Codec test fixtures are generated by repo-local scripts under python/spatialdata-js-util/scripts/.

Useful commands:

pnpm test:fixtures:generate:codecs
uv run --directory python/spatialdata-js-util spatialdata-js-util images inspect ../../test-fixtures/codecs/jpeg2k.manifest.json

Fixture provenance​

Dev fixtures stamp root metadata with an experimental marker rather than a fake spatialdata library version:

"spatialdata_attrs": {
  "experimental_codec_writer": "0.1.0",
  "note": "codec test fixture; not a spatialdata.write() artifact"
}

The frontend image schema treats spatialdata_attrs as optional on rasters and does not use version for OME format detection. We intend to propose a standardized provenance field to the spatialdata community; feedback welcome.

The sidecar manifest records shape, dtype, chunk shape, codec id, package versions, and representative decoded/encoded checksums. The fixture scripts validate representative chunks after writing so JS tests can compare decoded samples against Python.

Whole-object recompression​

For larger or real examples, use the recompress command. It copies the source SpatialData store, rewrites configured image rasters with JP2K or experimental HTJ2K, and writes labels with Blosc/zstd by default.

uv run --directory python/spatialdata-js-util spatialdata-js-util images recompress input.sdata.zarr output-jp2k.zarr --image-key morphology_focus --preset balanced --chunks auto --overwrite

Custom HTJ2K quality (instead of a preset name):

uv run --directory python/spatialdata-js-util spatialdata-js-util images recompress \
  input.sdata.zarr output-htj2k.zarr \
  --image-key morphology_focus \
  --codec experimental.openjph_htj2k \
  --quality 0.001 \
  --chunks auto \
  --sibling \
  --overwrite

--quality requires --image-key and --codec experimental.openjph_htj2k. Lower values preserve more detail. With --sibling, the output image is named like morphology_focus:htj2k_q0.001.

For repeatable per-raster settings, use JSON:

{
  "default_image": { "codec": "imagecodecs_jpeg2k", "preset": "lossless", "chunks": "auto" },
  "images": {
    "morphology_focus": { "preset": "balanced" },
    "morphology_focus_htj2k_tuned": {
      "codec": "experimental.openjph_htj2k",
      "quality": 0.001,
      "chunks": "auto"
    },
    "he_image": { "preset": "small" },
    "fast_preview": {
      "codec": "experimental.openjph_htj2k",
      "preset": "lossless",
      "chunks": "auto"
    }
  },
  "default_labels": { "codec": "blosc", "clevel": 5 }
}

JP2K and HTJ2K share preset names (lossless, balanced, small). HTJ2K encode uses OpenJPH WASM (experimental.openjph_htj2k) via HTJ2KEncoder.setQuality(reversible, quality). The quality argument is a float quantization factor (lower = higher fidelity, larger output) — not JP2K-style 0–100. The step is relative to the dtype's full range, so HTJ2K presets are expressed as a multiple of the input's LSB and resolve from the raster's dtype: balanced: 2 LSB, small: 5 LSB — 0.0078125/0.01953125 for uint8, 0.000030518/0.000076294 for uint16. A step below one LSB encodes larger than lossless for a bit-identical image, so the presets stay clear above that floor; see python/spatialdata-js-util/docs/htj2k-wasm-encode-design.md. Override with CLI --quality 0.001 or per-image JSON "quality": 0.001 (an absolute step, not an LSB multiple; implies lossy unless "reversible": true). A future codec-demo UI may add interactive q exploration on sample regions.

generate_codec_fixtures.py --experimental-htj2k also writes htj2k-quality-sweep.manifest.json, encoding the same Mandelbrot plane at several qualities so you can confirm compression responds on detail-rich imagery, and htj2k-encode-demo.manifest.json with three browser-viewable multiscale image layers in one store (htj2k-demo.zarr: mandelbrot_lossless, mandelbrot_balanced, mandelbrot_small) at 512×512 with 64×64 chunks. The main htj2k.zarr fixture remains a small 64×64 Mandelbrot raster for fast CI smoke tests. mandelbulb.zarr exercises multi t/z HTJ2K decode in JS (images/mandelbulb, shape [2, 1, 8, 128, 128], chunks (1, 1, 1, 128, 128)).

Browser-targeted image codec output is limited to uint8, int8, uint16, and int16. Labels are not image-codec-compressed in v1; they stay lossless through Blosc/zstd because integer IDs and wider dtypes such as uint32 are not considered supported by the current JavaScript decoder paths.

Useful Xenium morphology experiment:

uv run --directory python/spatialdata-js-util spatialdata-js-util images recompress /path/to/input.zarr /path/to/output.zarr --image-key morphology_focus --preset balanced --chunks auto --overwrite

When serving recompressed stores for browser experiments, disable HTTP caching or use a fresh output path after each rewrite. http-server defaults to cacheable responses, so overwriting a Zarr store in place can leave the browser with stale array metadata that points at the old codec while the chunk bytes have changed.

bunx http-server --cors -c-1 /private/tmp

Repo scripts may set UV_CACHE_DIR=.tmp/uv-cache for sandbox or CI cache isolation, but normal user-facing uv run commands do not need it.

JS codec flow​

zarrextra exposes:

  • registerJpeg2kCodec() for the registered imagecodecs_jpeg2k id (Node/CI).
  • enableWorkerChunkDecode() from zarrextra/workers for browser apps (fizarrita worker pool + custom codec worker with JP2K and experimental HTJ2K registration).
  • registerExperimentalHtj2kCodec() for OpenJPH HTJ2K decode (experimental.openjph_htj2k and legacy experimental.imagecodecs_htj2k) in Node/CI smoke tests.
  • loadOmeZarrMultiscalesFromStore() for loading multiscales from a zarr.Readable store without going back through Viv's URL loader.

@spatialdata/vis now prefers raster element stores (images/<key> and labels/<key>) and keeps Viv URL loading as a compatibility fallback. Browser apps using vis do not need to call enableWorkerChunkDecode() directly; the shared renderer path calls ensureCodecWorkers() once. Lower-level browser apps that use zarrextra without vis should still call enableWorkerChunkDecode() before loading JP2K or HTJ2K chunks.