ChunkedBackend class

Generic chunked storage layer.

Wraps a backing StorageBackend (any platform — local files, IndexedDB, a remote store, in-memory) and stores each logical key as N+1 records, with the chunk records tagged by a write generation:

{key}__manifest      → { generation, chunkCount, totalSize,
                         contentType, metadata }
{key}__g{G}__c0      → { body: Uint8List (≤ chunkSize) }
{key}__g{G}__c1      → { body: Uint8List (≤ chunkSize) }
...
{key}__g{G}__c{N-1}  → { body: Uint8List (≤ chunkSize) }

The chunking algorithm lives here; the storage primitives live in the backing backend. This separation keeps the algorithm platform-independent (testable on any VM) and means individual backends don't reinvent it.

What this layer guarantees

• writeStream is true streaming — buffers one chunkSize chunk at a time, flushes, then writes the manifest LAST.
• readStream yields chunk-by-chunk — O(chunkSize) memory.
• head/exists/list use the manifest only — no body reads.
• readRange decodes only the chunks that overlap the requested range.
• A crash mid-FIRST-write leaves the manifest absent → head/exists/list/read all behave as if the key never existed.
• A crash (or stream failure) mid-OVERWRITE leaves the previous object fully intact: each write lands its chunks under a fresh generation, swaps the manifest last, and only then reclaims the previous generation's chunks. Without generations, an overwrite would clobber the old chunk records in place and a mid-write failure would serve a torn mix of old and new bytes.
• Orphaned chunks from a failed write share the generation number the NEXT write of that key will use, so they're overwritten and reclaimed naturally; a failed first write is reaped by the defensive sweep in delete/deletePrefix.

Atomicity caveats

The backing backend is not assumed to provide multi-key transactions. Two callers writing to the SAME key concurrently can interleave each other's chunk writes; that's a caller-side serialization concern (the StorageBackend interface already notes this on every backend).

materialize

Not implemented at this layer because how to materialize an arbitrarily-large object as a platform-local handle is platform-specific (filesystem path on native, Blob URL on web). Subclasses or wrapping decorators provide it.

Implemented types

• StorageBackend

Mixed-in types

• DisposeGuard

Implementers

• IndexedDbBackend

Constructors

ChunkedBackend({required StorageBackend backing, int chunkSize = defaultChunkSize})

Wrap backing, splitting every object into chunkSize-byte records.

Properties

backing → StorageBackend

The backing backend — exposed for subclasses that need to add platform-specific operations (e.g. materialize) on top.

no setter

chunkSize → int

Plaintext bytes per chunk record.

final

disposeLabel → String

Short class name used in the post-dispose error, e.g. 'FileSystemBackend'.

no setter, override

hashCode → int

The hash code for this object.

no setter, inherited

isDisposed → bool

True once markDisposed has run.

no setter, inherited

runtimeType → Type

A representation of the runtime type of the object.

no setter, inherited

Methods

checkNotDisposed() → void

Throws StateError when this object has been disposed. Call as the first statement of every public method.

inherited

chunkKeyFor(String key, int generation, int index) → String

The backing-record key of chunk index in generation of key. For subclasses whose materialize reads chunk records directly — pair with manifestFor to get the current generation.

copy(String sourceKey, String destKey) → Future<void>

Copy an object, including its metadata.

override

delete(String key) → Future<void>

Delete a single object.

override

deletePrefix(String prefix) → Future<void>

Delete every object whose key starts with prefix (batch delete).

override

dispose() → Future<void>

Release the resources this backend itself constructed (database connections, HTTP clients). Idempotent — safe to call twice. Every method except dispose throws StateError afterwards.

override

exists(String key) → Future<bool>

Check if a key exists. Convenience for (await head(key)) != null.

override

head(String key) → Future<ObjectInfo?>

Get object metadata without reading the body (HeadObject).

override

list(String prefix) → Future<List<ObjectInfo>>

List every object whose key starts with prefix, with full metadata.

override

manifestFor(String key) → Future<ChunkedManifest?>

The manifest for key, or null when the object doesn't exist. For subclasses whose materialize reads chunk records directly.

markDisposed() → void

Record disposal. Idempotent.

inherited

materialize(String key, {bool decrypt = true, bool exclusive = false}) → Future<MaterializedFile>

Make a stored object available as a platform-local accessible resource.

override

noSuchMethod(Invocation invocation) → dynamic

Invoked when a nonexistent method or property is accessed.

inherited

read(String key) → Future<Uint8List>

Read entire contents as bytes. Convenience wrapper over readStream.

override

readRange(String key, {required int start, required int length}) → Future<Uint8List>

Read a byte range.

override

readStream(String key) → Stream<List<int>>

Read as a byte stream. Primary read path.

override

toString() → String

A string representation of this object.

inherited

updateMetadata(String key, {String? contentType, Map<String, String> metadata = const {}}) → Future<void>

Update metadata for an existing object without touching its bytes.

override

write(String key, Uint8List bytes, [WriteOptions options = const WriteOptions()]) → Future<void>

Write bytes. Convenience wrapper over writeStream.

override

writeStream(String key, Stream<List<int>> byteStream, [WriteOptions options = const WriteOptions()]) → Future<void>

Write a byte stream with optional write options.

override

Operators

operator ==(Object other) → bool

The equality operator.

inherited

Constants

chunkPrefix → const String

Per-chunk record marker. The chunk index follows it.

defaultChunkSize → const int

Default chunk size — 64 KiB.

generationPrefix → const String

Generation marker in a chunk record's key. The write generation follows it; __c{index} follows the generation.

manifestSuffix → const String

Suffix for the per-key manifest record. Distinct from the chunk markers, short, obvious in any storage browser.