snappy_neo 0.1.0

Pure Dart implementation of Google's Snappy compression: fast block-format compress and decompress, with a zero-copy Bytes API powered by blit.

snappy_neo

A pure Dart implementation of Google's Snappy compression format: fast, allocation-light, and with no native dependencies.

• Block format (the "raw" format used by Parquet, RPC payloads and key-value stores): Snappy.compress, Snappy.decompress, Snappy.compressInto, Snappy.decompressInto and Snappy.uncompressedLength.
• SnappyCodec for dart:convert.
• Zero-copy Bytes API backed by blit.
• Portable: Dart VM, AOT executables and the web (the codec avoids 64-bit integer semantics and is covered by dart test -p chrome).
• Interoperable: on the Dart VM and in AOT builds the output is byte-for-byte identical to the reference implementation, and every release is cross-checked in CI against the Rust snap crate in both directions. The JavaScript build uses a JS-safe hash, so it may emit a different (still valid, similarly sized) encoding.

Status

0.1.0 implements the block format. The framing format (stream identifier, chunked payloads, masked CRC32C) is planned for 0.1.1.

Usage

import 'package:snappy_neo/snappy_neo.dart';

final compressed = Snappy.compress(bytes);       // Uint8List
final restored = Snappy.decompress(compressed);  // Uint8List

Write into a buffer you own (no allocation inside the call):

final output = Uint8List(Snappy.maxCompressedLength(bytes.length));
final written = Snappy.compressInto(bytes, output);

dart:convert integration:

import 'dart:convert';
import 'package:snappy_neo/snappy_neo.dart';

const codec = SnappyCodec();
final encoded = codec.encode(utf8.encode('hello'));
final text = utf8.decode(codec.decode(encoded));

Zero-copy Bytes

import 'package:blit/bytes.dart';
import 'package:snappy_neo/blit.dart';

final payload = Bytes.view(backingStore, offset, length); // no copy
final compressed = payload.snappyCompress();              // Bytes, no copy on read
final restored = compressed.snappyDecompress();

API

Member	Description
Snappy.maxCompressedLength(int)	Upper bound for an input of that size
Snappy.uncompressedLength(List<int>)	Parses the stream header only
Snappy.compress(List<int>)	Compresses into a new Uint8List
Snappy.compressInto(List<int>, Uint8List)	Compresses into your buffer, returns bytes written
Snappy.decompress(List<int>)	Decompresses into a new Uint8List
Snappy.decompressInto(List<int>, Uint8List)	Decompresses into your buffer, returns bytes written
SnappyCodec / SnappyEncoder / SnappyDecoder	dart:convert integration

Invalid streams throw FormatException; a destination buffer that is too small throws ArgumentError.

Performance

Measured on one Windows x64 core with the same 100–180 KiB corpus used by the CI chart (MiB/s, higher is better; the Rust column is the reference snap crate running the same files):

corpus	Dart compress	Rust compress	Dart decompress	Rust decompress
alice29.txt (text)	175	514	770	1357
asyoulik.txt (text)	180	620	776	1394
fireworks.jpeg	5085	23518	17879	32966
geo.protodata	693	2829	1219	5822
html	411	2390	1019	4229
kppkn.gtb	223	721	537	1288
paper-100k.pdf	1937	10018	5551	17172

The chart is regenerated by CI after every push to master from benchmark_test runs (Dart JIT and AOT) plus the Rust oracle, rendered with benchcat. Numbers come from the ambient CI runner, so treat them as a trend rather than a specification; a busy developer machine can move a single run by a factor of two either way.

Reproduce locally:

dart run benchmark_test --output jsonl -c jit bench/dart/snappy_bench.dart > build/bench/dart-jit.jsonl
dart run tool/bench_report.dart build/bench/dart-jit.jsonl testdata/corpus build/bench/dart-jit.json

Correctness

• Hand-built format vectors for every tag type, literal/offset boundary and malformed-stream class, plus roundtrips over deterministic random, text, binary and mixed data from 0 bytes to several MiB.
• test/golden_test.dart decodes streams produced by the Rust snap crate.
• test/interop_test.dart runs the reference implementation in both directions and asserts byte-identical compression (requires SNAPPY_ORACLE, built from bench/rust).

Sample data in testdata/corpus comes from the Snappy project and is covered by its BSD-3-Clause test-data license (testdata/corpus/LICENSE).

License

BSD-3-Clause. Snappy itself is Apache-2.0; this package is an independent pure Dart implementation of the format.