blit 0.1.1

Zero-copy Bytes/BytesMut and high-performance random-access file I/O for Dart and Flutter, with native FFI mmap and Web OPFS.

blit

Zero-copy Bytes / BytesMut and fast random-access file I/O for Dart and Flutter. Native I/O is pure dart:ffi; Web I/O uses OPFS. No C compiler, native build hooks or native assets.

Why blit

• Zero-copy Bytes — a (buffer, offset, length) view; slice/sublist are O(1).
• TypedData bridge — Bytes.fromTypedData and TypedData.asBytes() wrap Uint8List, ByteData, Int32List, etc. without copying. BytesLike is a small zero-copy boundary interface for other byte libraries.
• BytesMut — growable buffer; freeze() / splitTo() / splitOff() return shared views with copy-on-write on later mutation.
• Exact Uint64 — two 32-bit limbs; works on VM, dart2js and dart2wasm without Uint64List or ByteData.getUint64.
• Reader/writer cursors — BytesReader / BytesWriter provide Buf/BufMut-style fixed-width access.
• Native file engines — mmap, isolate pool, Windows IOCP and Linux io_uring.
• Web backend — OPFS chunks + double-slot manifest, with a chunk-worker pool and an optional cross-origin-isolated SAB ring.

Runtime dependencies are package:ffi and the pure-Dart package:io_uring (used only by the Linux ring backend).

Install

dart pub add blit

Requires Dart ^3.13.5. A current Flutter SDK includes a compatible Dart SDK.

Quick start

Bytes and views

import 'dart:typed_data';
import 'package:blit/blit.dart';

final backing = Uint8List.fromList([1, 2, 3, 4]);
final bytes = Bytes.fromUint8List(backing);
final slice = bytes.slice(1, 3); // O(1), shares backing

backing[1] = 99;
print(slice[0]); // 99

Typed data interop is zero-copy:

final bytes = Bytes.fromTypedData(byteData); // Uint8List / ByteData / Int32List
final typed = bytes.asUint8List();            // zero-copy view

Other byte libraries can implement BytesLike:

abstract interface class BytesLike {
  int get length;
  Uint8List asUint8List();
}

Convert at the boundary, then stay on Bytes / Uint8List in hot loops.

Encoding

final mut = BytesMut(16)
  ..putUint32(0x01020304)
  ..putUint64(const Uint64(0x11223344, 0x55667788));
final bytes = mut.freeze();

print(bytes.getUint32(0).toRadixString(16)); // 1020304
print(bytes.getUint64(4));                   // 1234605616436508552

Random-access files

import 'dart:typed_data';
import 'package:blit/blit.dart';

final file = await openBytesFile('/tmp/data.bin');
try {
  await file.writeAt(0, Uint8List.fromList([1, 2, 3, 4]));
  await file.append(Uint8List.fromList([5, 6]));

  final out = Uint8List(6);
  await file.readAt(0, out);
} finally {
  await file.close();
}

openBytesFile is the portable async API. The synchronous companion lives in package:blit/bytes_ffi.dart for native-only code.

Native engines

Import native-only APIs from package:blit/bytes_ffi.dart.

mmap

final file = openMmapFileSync('/tmp/sparse.bin');
file.truncate(32 << 30); // sparse by default on Windows
final region = file.map(offset: 30 << 30, length: 1 << 20);

region.bytes[0] = 42;
region.prefetch(offset: 0, length: 1 << 20);
region.flush();
region.close();
file.close();

Mapped windows support prefetch() / discard() hints. File-level helpers include punchHole (Windows/Linux), cloneRangeFrom (Windows), preallocate and adviseRead (macOS), and cloneFileSync (macOS clonefile, copy fallback). Unsupported filesystems throw; fall back to zeroize or a copy.

Isolate pool

final pool = await ParallelBytesFile.open(
  '/tmp/data.bin',
  maxWorkers: 8,
  maxQueue: 64,
);

final chunks = await pool.readMany([0, 1 << 20, 2 << 20], 4096);
await pool.close();

maxQueue bounds queued + in-flight work. readMany merges exactly adjacent ranges. Read-only files up to mmapReadCacheLimit (default 1 GiB) are mapped once per worker for hot random reads; set the limit to 0 to disable. Mapping is virtual: cold pages still fault from disk.

Windows

IocpFile exposes overlapped I/O with queue depth 1–128:

final f = IocpFile.openSync('/tmp/data.bin', queueDepth: 32);
final data = f.readManySync(offsets, 4096);
f.close();

MapViewOfFile3 placeholders can swap mapped windows without an unmap/remap gap:

final placeholder = file.createPlaceholder(1 << 20);
final view = placeholder.map(otherFile);
view.close();
placeholder.close();

Direct I/O uses FILE_FLAG_NO_BUFFERING; largePages: true opts into MEM_LARGE_PAGES scratch buffers when SeLockMemoryPrivilege is available.

Linux io_uring

UringBytesFile is a pure-Dart io_uring backend built on package:io_uring. Use the batch methods for queue depth:

final f = await openUringBytesFile('/tmp/data.bin', entries: 64);
try {
  await f.writeMany(offsets, blocks);
  final reads = await f.readMany(offsets, 4096);
  await f.flush();
} finally {
  await f.close();
}

entries must be at least the batch size. The backend is Linux-only.

Direct I/O

For streaming files larger than RAM, open with direct: true (Windows FILE_FLAG_NO_BUFFERING, Linux/Android O_DIRECT, macOS/iOS F_NOCACHE). Offsets and lengths must be 4 KiB aligned. Direct I/O is intended for sequential copy/import/export, not 4 KiB random access.

Web backend

final file = await openBytesFile('data/big.bin');
await file.writeAt(0, header);
await file.flush();

On Web, BytesFile is backed by OPFS chunk files and a double-slot manifest. Logical length is metadata-only until data is written, so large sparse files do not zero-fill the whole range.

OPFS sync handles are dedicated-worker-only and exclusive per file. Chunks are distributed across a small worker pool; one logical file serializes its writes so the manifest stays single-writer. In a cross-origin-isolated page (COOP: same-origin + COEP: require-corp), small reads/writes use a SharedArrayBuffer ring; larger payloads and non-isolated pages fall back to postMessage transfer.

Platform support

Capability	Windows	Linux	macOS	Android	iOS	Web
Bytes / BytesMut / Uint64	✅	✅	✅	✅*	✅*	✅
BytesFile (dart:io)	✅	✅	✅	⚠️	⚠️	—
mmap (bytes_ffi.dart)	✅	✅	✅	⚠️	⚠️	—
IocpFile	✅	—	—	—	—	—
UringBytesFile	—	✅	—	—	—	—
ParallelBytesFile	✅	✅	✅	⚠️	⚠️	—
OPFS backend	—	—	—	—	—	✅

* Android/iOS are not built or run by CI; the pure-Dart core is portable, but mobile file paths are unverified. CI covers Dart on Ubuntu/macOS/Windows and Chrome on Ubuntu.

Large and sparse files

Meaning	Cost
Large logical length	metadata-only (truncate)
Large sparse file	metadata until written
Real data	disk/cache-bound; write only what you need

On Windows, a non-sparse truncate(32 GiB) followed by a far-offset write can zero-fill the gap. MmapFile.truncate(..., sparse: true) is the default and avoids it. OPFS uses chunk files for the same reason.

Performance

The benchmark card above compares Dart with the C counterpart in benchmark/c/blit_bench.c. Both implement the same six single-thread operations: 8 MiB sequential read/write, 4 KiB random pread/pwrite, and 4 KiB random mmap read/write. Dart-only pool, large-file and IOCP rows are kept as single-series rows; they have no C counterpart by design.

The card is generated from the benchmark/ suite with bench-svg. Raw JSONL/SVG artifacts are produced by CI; the chart is deterministic and reflects the latest default branch run.

Random 4 KiB I/O is latency-bound; the isolate pool hides it with concurrency. mmap pays page faults on cold random access but is memory-speed once hot. pwrite numbers are page-cache writes, not durability.

Examples

• example/bytes_example.dart: core APIs and native file I/O.
• example/bytes_web_example.dart: Web-safe core for dart2js/dart2wasm.
• flutter_example/: repository-only Flutter example.

Development

dart pub get
dart analyze
dart test
dart run example/bytes_example.dart

Chrome tests cover OPFS and Web fuzz. CI runs Dart on Ubuntu/macOS/Windows and Chrome on Ubuntu. On push/manual branch runs, a Windows benchmark job renders the SVG with bench-svg, uploads JSONL/SVG artifacts and commits the chart with [skip ci]. PR runs stop after tests. workflow_dispatch accepts benchmark_gb and BYTES_BENCH_GB controls local random-file benchmarks.

Limitations

• OPFS sync handles are worker-only and exclusive per file; the Web backend is serialized per file. Safari/Firefox are feature-detect targets, not locally validated.
• Android/iOS are not built or run by CI; native mobile file paths are unverified.
• IocpFile is Windows-only; cold-device behavior above 4 GiB is not locally verified.
• Zeroize is memset-level; there is no compiler-proof SecureZeroMemory without a C shim.
• ParallelBytesFile does not order overlapping writes; await the previous write or use flush as a barrier.
• The API is pre-1.0; breaking changes may occur.

License

MIT. See LICENSE.