blit 0.2.1
blit: ^0.2.1 copied to clipboard
Zero-copy Bytes/BytesMut and high-performance random-access file I/O for Dart and Flutter, with async file I/O, native FFI mmap, and Web storage.
Changelog #
0.2.1 #
Added #
UnsafeBytes.alloc/free: fixed-length external memory outside the Dart GC.- Native
malloc/freeand WebWebAssembly.Memoryarena backends. - Raw int API:
unsafeAlloc,unsafeRead*,unsafeWrite*,unsafeFree. AsyncFile.readAtUnsafe/writeAtUnsafewith explicit length.
0.2.0 #
Added #
AsyncFile,BlitFile,FileOptions, and theBlitExceptionhierarchy.blitsimple API:open,read,write,append,copy,exists,delete.NativeBuffer,NativeBufferPool, andNativeMmapFileinpackage:blit/blit_native.dart.readMany/writeManycoalesce contiguous requests; clustered writes use one read-modify-write; readMany uses concurrent backends.AsyncFileNativepooled read/write viaNativeBuffer.- Web
openIndexedDb; OPFS falls back to IndexedDB. - Cooperative
CancelToken;exclusivecreate;flushWith. close/deletedrain pending operations before closing.directmmap selection andPerformanceProfileworker tuning.- Split public entries:
blit.dart,blit_io.dart,blit_buffer.dart,blit_web.dart,blit_native.dart.
0.1.7 #
Added #
- AsyncIO utilities in
package:blit/blit_io.dart:ReadBuf,IoSlice,AsyncRead,AsyncWrite,AsyncBufRead,AsyncReadExt,AsyncWriteExt. BufReader,BufWriter,readLine,readerStream,copy.Uint8ListReader,BytesAsyncReader,Uint8ListWriter,BytesMutAsyncWriter,AsyncFileReader,AsyncFileWriter.
0.1.5 #
0.1.4 #
Changed #
- Benchmark chart rendering moved to
benchcat(benchcat.toml); CI usesunk-404/benchcat@masterinstead ofbenchmark_svg.
Fixed #
- Shared
io_uringloops are now explicitly closable throughUringBytesFile.closeSharedLoopPool; the Linux test suite closes its pool intearDownAll. This prevents an idleio_uring-reaperisolate from keepingdart test/ CI alive after the shared-loop tests finish.
0.1.3 #
Changed #
- Requires
io_uring^0.1.1, which providesUringLoopPool/UringLoopLease.
Fixed #
BytesMut.splitTo/splitOffnow mark the source frozen, so the returned views survive laterBytesMutmutations (copy-on-write), matching the README.
Performance #
- Added
BytesMut.spareView+advanceand unchecked put variants for uint8/uint16/uint32/uint64/float64/bytes/slice/repeated/uint;Bytesgained unchecked uint64/float32/float64 reads. - Windows
IocpFilereusesOVERLAPPEDblocks and its single-request scratch buffer instead of allocating per request. - Linux
UringBytesFileregisters a small fixed-buffer set and usesREAD_FIXED/WRITE_FIXED, falling back to normal ops when unsupported. - Added
openUringBytesFile(..., sharedLoop: true)using theio_uring0.1.1UringLoopPool/UringLoopLease; fixed buffers are unregistered before a lease is returned. Leases are exclusive, so this reuses reaper isolates for sequential open/close workloads rather than multiplexing concurrent files. - Added a core Bytes/BytesMut/reader/writer benchmark suite, including a raw
Uint8Listbaseline, merged into the main benchmark card.
Added #
MmapFile.readAtPointer/writeAtPointerfor caller-owned native buffers; these bypass the typed-data scratch copy (and the transparent mmap cache).
0.1.2 #
Performance #
- Buffered
MmapFile.readAt/writeAtnow use a transparent whole-file mmap cache by default;mmapCache: falsekeeps pool workers and direct-I/O files on the syscall path. File size and FFI argument storage are reused instead of queried/allocated per call. - Added an isolate-local
PooledNativeAllocatorfor uninitialized, reusable scratch blocks. UringBytesFilereusesUringBuffers across single and batched operations.
0.1.1 #
Added #
- TypedData interop:
Bytes.fromTypedData,TypedData.asBytes(),Bytes.fromBytesLike, and theBytesLikeboundary interface;BytesandBytesMutimplementBytesLikewithout copying. MmapFile.punchHole(WindowsFSCTL_SET_ZERO_DATA, Linuxfallocate(PUNCH_HOLE)), WindowscloneRangeFrom(FSCTL_DUPLICATE_EXTENTS_TO_FILE), macOSpreallocate(F_PREALLOCATE) /adviseRead(F_RDADVISE), andcloneFileSync(clonefileon macOS/iOS, copy fallback elsewhere).- Linux
UringBytesFile/openUringBytesFile: pure-Dartpackage:io_uringSQ/CQ rings, batchedreadMany/writeMany, noliburingor C shim. - Windows
MmapPlaceholderviaVirtualAlloc2+MapViewOfFile3(MEM_RESERVE_PLACEHOLDER/MEM_REPLACE_PLACEHOLDER), for gap-free window replacement on Windows 10 1803+. - Windows direct-I/O
largePages: trueopt-in (MEM_LARGE_PAGESscratch, privilege-checked with normal-page fallback). ParallelBytesFile.maxQueuebackpressure for queued + in-flight jobs.- Web OPFS worker-per-chunk pool:
OpfsWorkerPoolroutes chunkitoworkers[i % N](default 4), keeps the manifest single-writer, and caches each chunk handle in its owning worker. - Optional Web SAB ring for small
read/writecalls whencrossOriginIsolatedandAtomics.waitAsyncare available: 8 闂?64 KiB slots,Atomicswakeups, postMessage fallback for larger payloads and non-isolated pages. - Benchmark SVG rendering now uses
bench-svg(bench-svg.toml+ theunk-404/benchmark_svgaction) instead of a repo-local SVG generator. - Benchmark chart compares Dart with a C counterpart implementing the same single-thread 8 MiB sequential and 4 KiB random/mmap operations; Dart-only engine rows use bench-svg placeholder tracks.
0.1.0 #
First development release.
Added #
- Zero-copy
Bytesviews (slice/sublistO(1)) andBytesMutwithfreeze/splitTo/splitOffcopy-on-write semantics. - Exact
Uint64(hi/lo limbs) for VM, dart2js and dart2wasm, including arithmetic, bitwise, shifts,parseandtoRadixString. BytesReader/BytesWriterfixed-width cursors with BE/LE support.BytesFile/SyncBytesFileand the nativedart:iobackend.- Native mmap engine: Windows
CreateFileMappingW, POSIXmmap/pread; sparse files, zero-copy windows, zeroize, read-only mappings. - Windows
IocpFile: overlapped I/O + IOCP, QD 1闂?28 batch reads/writes. MappedRegion.prefetch()/discard(): WindowsPrefetchVirtualMemory(discard is a no-op for file-backed maps), POSIXmadvise(MADV_WILLNEED / MADV_DONTNEED).Bytes.getUint8/16/32UncheckedandBytesReader.readUint8/16/32/64Uncheckedhot-path methods (caller checks bounds once).- Direct I/O:
openMmapFileSync(..., direct: true)/ParallelBytesFile.open(..., direct: true)via WindowsFILE_FLAG_NO_BUFFERING+VirtualAlloc, Linux/AndroidO_DIRECT, and macOS/iOSF_NOCACHE; aligned scratch buffers and 4 KiB-aligned streaming reads/writes. ParallelBytesFile: isolate worker pool (default up to 32), automatic FIFO dispatch, batch 32,readMany/writeMany, watermark barriers, adjacent-range coalescing, and a read-only full-file mmap cache for hot files (default limit 1 GiB).- Web OPFS backend: chunk files, double-slot manifest, CRC-32, dedicated worker, capability/quota probing.
- Benchmark suite (
benchmark_test, JIT/AOT + JSONL), a generatedassets/benchmarks.svg, and Dart/Web CI workflows. - Tests: differential fuzz, lifetime/error paths, 100k-op stress, 32 GiB sparse.
Fixed during development #
putUint64little-endian limb encoding and atomic failure on short buffers.ParallelBytesFilebarrier deadlock when queued work already existed.- Windows read-only mapping
zeroizeguard and empty-filemapconsistency.