koni_jxl 0.1.4

Pure Dart JPEG XL (JXL) codec — decode .jxl files to pixels and encode lossless .jxl, with zero native dependencies on every Dart/Flutter platform.

Changelog

0.1.4

Lossless encoding — now competitive with cjxl, smaller than PNG

A series of never-worse compression levers (each tried against a real-assembly baseline and kept only when it codes smaller) moved the modular encoder from "correct" to genuinely competitive. On the 97-image public burkardt PNG set it now totals 60% of the source PNGs (beating PNG on 87/97) and 102.4% of cjxl -e7 (beating cjxl on 19/97).

• Predictor selection from the learned tree's training entropy. Instead of running both the gradient and weighted-predictor pipelines to completion, the encoder now decides from each learned tree's training entropy and finishes only the winner when it's a clear win — −17% to −26% encode time, output bit-identical.
• Per-leaf predictor selection. Each MA-tree leaf independently keeps the image predictor or switches gradient↔weighted wherever that codes its own pixels smaller (gray_screentone −24.6%; the decoder already supports a per-leaf predictor, so no format change).
• Deeper, gated LZ77 matcher (chain depth 256, lazy lookahead) tried only when LZ77 already beats plain, kept alongside the old matcher so it's provably never-worse: gray_screentone −46.5%, screentone the winning mode.
• Palette above 256 colours (RGB, up to 4096) and a single-channel grayscale palette gated on value sparsity — flat UI graphics −48%, and bilevel/fractal grayscale (previously the worst gap to cjxl) transformed: dla now beats cjxl, sierpinski/math_emporium went 290%/229% → 125%/109% of cjxl -e7.
• Cross-channel context for RCT colour (the tree conditions Co/Cg on the prior channel and on channel index): colour content ~−2%, illustration/UI ties cjxl.
• Per-image hybrid-uint tokenization ((4,1,0) vs (4,2,0), keep smaller): screentone/line-art −2% to −4%.

Decoding

• Embedded ICC profiles are now applied as an output colour transform for matrix/TRC RGB profiles (color/icc_transform.dart), not just decoded and exposed. Conformance-verified against ref.png — the progressive case goes from rmse 12.6 to 0.082. Grayscale/CMYK/LUT-CLUT profiles still fall back to sRGB.
• Spot-colour extra channels are composited onto the image (out = mix·spotRGB + (1−mix)·out), verified against the spot conformance case (rmse 115.9 → 0.43).
• Fixed the default VarDCT quant weights for DCT 256×128 / 128×256 — an inherited jxlatte transcription bug (channels 1/2 used the square base weights instead of the rectangular ones), a latent lossy-fidelity gap at the default distance. Confirmed inherited via the standard djxl/jxlatte three-way compare.
• Reduced-resolution decode extended to Modular/lossless responsive (Squeeze) files — decodes the low-frequency pyramid alone and box-downsamples to 1:8 (~3.6× faster on a 1024×1536 responsive file) — and to progressive-DC (separate LF frame) VarDCT files (2.7–6.8× faster).
• EPF pass 0 is now Float32x4-SIMD (both gray and colour double-cross kernels), the last scalar EPF kernel: an 11 MP triple-pass progressive photo dropped from ~4.8 s to ~2.0 s full decode (pass 0 itself 3.0 s → 0.4 s).
• Chained multi-layer blend modes (blendmodes): investigated and confirmed a libjxl-version deviation koni tracks faithfully (djxl 0.11.2 itself deviates from ref.png nearly identically); koni matches djxl/jxlatte to within 8-bit rounding. Not a koni bug; gated against djxl rather than ref.png.

0.1.3

Decoding

• Float (HDR) sample decode is now supported. Images with floating-point samples previously threw JxlUnsupportedException; they now decode. This root-causes a long-open lossless_pfm conformance failure to a spurious 32-bit truncation (.toSigned(32)) that had been applied to prediction() and the weighted predictor's internals — libjxl computes those in 64-bit (pixel_type_w), narrowing to 32-bit only at genuine storage points, never mid-expression. Verified bit-exact against the lossless_pfm conformance reference: all 750,000 float values across 3 channels, zero mismatches. Encoding float samples remains unimplemented.
• JxlDecoder.decode(bytes, {targetWidth, targetHeight}) — reduced- resolution decode for callers that only need a smaller image (a manga reader's oversized-page safety cap, a thumbnail). For a single-frame VarDCT image with no patches, splines, noise or format-level upsampling, and a target no larger than the format's built-in 1:8-scale DC image, this decodes only the DC data — every AC coefficient is skipped entirely — and box-filters that down to the exact target: ~11x faster on a 6000x9000 synthetic test image decoded at a 512px cap (see tool/bench_downscale.dart). Every other case (animated, Modular/ lossless, patches/splines/noise present, or a target finer than 1:8 scale) decodes the image fully and box-downsamples the result — always correct, just without the CPU/memory saving. Output never upscales past native size (matches ui.ResizeImage's contract). JxlStreamingDecoder.decodePreview()'s DC-image assembly is now shared with this path (render/dc_image.dart), unchanged in behavior.

Encoding

• Restored the never-worse-than-baseline guarantee for the opt-in bespoke transforms (VardctL0Config.enableBespokeTransforms, off by default). The 0.1.2 16x16-selection rework regressed this: with bespoke transforms enabled, an over-selected layout could be committed before the 16x16-tier decision ran, producing up to +4% larger output than the flags-off baseline on one real manga page. The transform-layout decision now always feeds the flags-off baseline cascade into the same real- assembly candidate pool it min-selects from, so the chosen output is provably never larger than either the baseline or 0.1.2. Default (flags- off) output is bit-identical to 0.1.2; every bespoke combination improved on the 144-encode real-manga sweep (e.g. +bespoke -0.20% → -0.42%).

0.1.2

Decoding

• Fixed 7 Java-int-truncation overflow bugs in modular (lossless) prediction and entropy reconstruction: the weighted predictor's eSum masking (the original crash) and its subpred/n3 truncation, the simple predictor's averaging/gradient cases, MA-tree property computation (including cross-channel gradient), the core per-pixel decode loop, and hybrid-uint reconstruction's final mask. Found while auditing float-sample support (packed values approach the full ±2^31 range, unlike ordinary 8/16-bit samples) and verified against jxlatte/libjxl source; includes a genuine bug inherited from jxlatte, where the hybrid-integer extra-bit count now correctly wraps via n &= 31 (matching libjxl) instead of rejecting anything over 32.
• Fixed 32-bit bitwise-op truncation on the dart2js web target. dart2js compiles int to a JS double and coerces every bitwise operator through JS's ToInt32/ToUint32, silently corrupting the bit reader, the weighted predictor (decode and encode), VLC/hybrid-integer sentinels, and XorShiro's 64-bit noise constants whenever a value or shift amount left 32-bit range — latent on dart2wasm (real 64-bit ints, this project's actual web target), live on dart2js. Found with a new differential oracle comparing native vs. dart2js output on the corpus.
• 18-24% faster real-manga decode (~0.3s → ~0.25s): skip redundant integer divisions in the AC coefficient context computation for the (common, for JPEG-transcoded manga content) single-block case, and replace six full-block-list scans with a precomputed per-LF-group index. Lossless decode is also 2-4% faster generally from a matching fix to MaTree.compactify().

Encoding

• Fixed 16x16 transform-size selection, now on by default. Replaced the old pre-quantization coefficient-magnitude proxy — which over-selected 16x16 on manga content (+20% screentone, +31% line art) — with a real bootstrap-frozen bit-rate estimate plus a whole-image real-assembly safety net, so enabling this can never produce a larger file than leaving it off. VardctL0Config.enableVariableTransforms now defaults to true: 4-27% smaller with better RMSE on photographic content across distance 0.5-8.0, never worse on screentone/line-art; narrows the gap to cjxl -e1 from 1.52x-2.79x to 1.18x-1.82x on the benchmark corpus. A further cascade refinement (candidate scoring now uses a live, incrementally-updated neighbor-prediction grid instead of a frozen bootstrap snapshot) shrinks output up to a further ~4% on multi-level transform-size configurations.
• All 27 VarDCT transform types now exist and are djxl-verified correct (up from 8x8/16x16 only): every remaining square size (32x32 through 256x256, maxTransformSize), all 12 rectangular types (enableRectangularTransforms), and all 9 bespoke types — DCT4x4, DCT2x2, Hornuss, DCT4x8/DCT8x4, AFV0-3 (enableBespokeTransforms). This completes the full-format-coverage goal tracked since 0.1.1's L3 milestone. A real-manga ROI evaluation across 144 encodes of real chapter pages found real but small wins (best combination: -0.86% at 6.1x baseline encode time) — every one of these knobs stays off by default; existence and default-on-ness remain separate questions. See ROADMAP.md/doc/spec_notes.md for the full numbers.

0.1.1

API

• looksLikeJxl(bytes) — cheap signature sniff (bare FF 0A codestream or the ISOBMFF container box) for routing bytes of mixed image formats to the right decoder, e.g. inside a custom Flutter ImageProvider. Never throws.

Encoding

• Lossy (VarDCT) — L0 milestone. JxlEncoder.encodeLossy produces a real VarDCT stream: 8x8-DCT-only, uniform quantization mirroring the decoder's own dequantization formulas, chroma-from-luma pre-subtraction, filters off, single group (width/height multiples of 8, up to 256x256). This is a correctness-first milestone (see ROADMAP.md) — quality and compactness come with later phases.
• Lossy (VarDCT) — L1 milestone. encodeLossy gained a distance: parameter, the real HF coefficient context model (in place of L0's single shared histogram — meaningfully smaller files on busier images), and multi-group support (up to 2048x2048, still a single LF group; see ROADMAP.md for the multi-LfGroup gap).
• Lossy (VarDCT) — L2 milestone. Adaptive per-block quantization (~65-70% RMSE reduction on smooth/gradient content, where fixed quantization causes visible banding), a custom per-frequency quant weight table that removes distance's previous quality floor (now monotonic down to distance = 0.05 in testing, vs. plateauing around 0.5-0.8 before), and a global (whole-image) chroma-from-luma fit. See ROADMAP.md for the remaining per-region CfL upgrade.
• Lossy (VarDCT) — L3 milestone. Two opt-in additions, both off by default: Gaborish + edge-preserving filtering (VardctL0Config.enableFilters), and adaptive per-region 8x8/16x16 transform size selection (enableVariableTransforms). Both are real, djxl-verified working capabilities that help smooth/photographic content but were measured to regress manga's dominant content types (screentone, line art) by a wide margin, so both default off — see doc/spec_notes.md for the full numbers.
• Lossy (VarDCT) — per-region chroma-from-luma. Upgraded L2's global-only chroma-from-luma fit to the spec's real per-64x64-region granularity (HfMetadata's xFromY/bFromY), on by default: ~26% RMSE reduction at roughly the same file size on content with genuinely different color relationships across regions, vs. ~1% size overhead on content with no real regional color variation to exploit.
• Lossy (VarDCT) — multi-LF-group support. encodeLossy no longer caps at 2048x2048: images of any size now split into multiple LF groups as needed, matching the format's own structure. The AC entropy coding path required no changes — groups were already numbered independent of LF groups end-to-end — so this only needed splitting DC/HfMetadata into per-LF-group sections and restructuring the TOC.
• Lossy (VarDCT) — L4 milestone. encodeLossy now accepts any positive width/height, not just multiples of 8 (padded internally via edge replication, true size written to the header — see doc/spec_notes.md). New encodeJxlLossyFromRgba/ encodeJxlLossyFromUiImage Flutter helpers (koni_jxl_flutter, alpha dropped). New real-corpus lossy round-trip gate (test/encode/encoder_lossy_corpus_test.dart) and a cjxl-comparison benchmark (tool/bench_lossy_vs_cjxl.dart) — this encoder currently produces files 1.5-5x larger than cjxl -e1 at matched distance, expected given no rate-distortion search and only 2 of 27 transform types, now measured concretely rather than assumed. VardctL0Config is now exported from the public API.
• Lossy (VarDCT) — DC gradient prediction. DC (LF) coefficients were being encoded with zero spatial prediction — over half this encoder's total output size on real photo content, more than the AC coefficients. Now uses the same clamped-gradient predictor (predictor 5) the lossless encoder already uses, cutting DC size 49-75% and total file size 25-27% on the corpus's two RGB test images, roughly halving the size gap vs cjxl -e1.
• Lossy (VarDCT) — weighted predictor for DC. DC coefficients now also try the self-correcting weighted predictor (predictor 6) alongside the clamped-gradient predictor, keeping whichever compresses smaller — a further ~5% reduction on real photo content where WP wins, no change where gradient already wins.
• Lossy (VarDCT) — RD-hfMult search (VardctL0Config.enableRdHfMult, off by default). A genuine per-block rate-distortion search replacing the crude 3-bucket adaptive-quantization heuristic: real weighted- squared-error distortion, a real Huffman-code-length-based rate estimate (EntropyCodes.tokenBitLengths(), new), correctness-verified against djxl in every configuration tried. Calibration (tool/calibrate_rd_lambda.dart) found no single trade-off constant both beats the heuristic on real photo content and preserves its smooth-gradient banding protection — a genuine modeling limit (plain weighted MSE can't see banding sensitivity the way a real perceptual metric would), not a bug, documented in doc/spec_notes.md.
• ANS (rANS) is now a per-image lossless entropy candidate alongside prefix codes, and can carry LZ77 matches (plain/LZ77 x prefix/ANS, smallest actual output wins).
• Learned per-image context tree replaces the fixed 7-context MA tree: a greedy entropy-minimizing split search over decoder properties, up to 64 contexts.
• Weighted predictor is now a second per-image predictor candidate (alongside clamped gradient), with its own property set including the WP max-error signal.

0.1.0

First release. A pure-Dart JPEG XL codec with zero native dependencies, verified against libjxl (djxl) and the official conformance suite.

Decoding

• Lossless (Modular) still images decode bit-exact vs libjxl: all predictors including the self-correcting weighted predictor, RCT / palette (incl. delta palette) / squeeze transforms, patches, reference frames, all blend modes, alpha, 8/16-bit, EXIF orientation.
• Lossy (VarDCT) still images decode within ~1 RMSE of libjxl: all 27 transform types, adaptive quantization, chroma-from-luma, XYB color, Gaborish and edge-preserving filters, upsampling, noise synthesis, YCbCr with chroma subsampling.
• Animation — JxlDecoder.decodeAnimation returns all visible frames with per-frame durations, timecodes and loop count.
• Splines, progressive DC (LF) frames and multi-pass AC.
• Streaming — JxlStreamingDecoder decodes incrementally arriving bytes: header info, buffering progress, a 1:8 DC preview once the DC sections (or a progressive-DC frame) are available, then the final image.
• Header-only JxlInfo.parse and embedded ICC profile decoding.

Encoding

• Lossless — JxlEncoder.encodeLossless / encodeLossless16 (interleaved 8/16-bit gray/RGB with optional alpha) and encodeImage (JXL→JXL transcode). Modular with a fixed gradient-context per-image learned context tree over either the clamped-gradient or self-correcting weighted predictor, palette / YCoCg RCT, and the smallest of four entropy modes ({plain, LZ77} x {prefix, ANS}) chosen by actual coded size. Every output is verified bit-exact through this decoder and djxl in tests.

Robustness & performance

• All decode surfaces throw only JxlException on malformed input (mutation-fuzz verified); JxlLimits bounds header-driven allocations.
• Float32x4 SIMD across the lossy pipeline (fused 8×8 and batched large-block inverse DCT, dequantization, XYB inverse, Gaborish, EPF); native on AOT targets, emulated on the web. A 1536×2200 lossless page decodes in ~60–410 ms; typical lossy pages in ~0.3–0.5 s.

Tools

• jxl_info, jxl_dec and jxl_enc command-line utilities.

Not yet supported

Decoding throws JxlUnsupportedException (with the feature name) for spot-color rendering, JPEG bitstream reconstruction, and float (HDR) sample formats. Encoding is lossless-only.