memory_gate library

Memory-gated concurrency for upload operations.

Limits the total bytes held in-flight during concurrent uploads by checking available system memory and reserving slots. Modeled after internxt-dart's MemoryGate.

Classes

ChunkSemaphore

A counting semaphore that bounds the number of concurrent in-flight chunk transfers. Paired with MemoryGate (which bounds the bytes in flight), it caps chunk concurrency by both count and memory — the Step 1 model.

MemoryGate

Constants

kDefaultFileConcurrency → const int

File-level (batch) concurrency (Step 2). W = whole FILES transferred at once in a batch directory upload/download — the bigger real-world win when syncing many files. Each file ALSO runs Step 1 chunk concurrency internally, so the dangerous quantity is the PRODUCT (W files × N chunks each). One shared ChunkSemaphore caps that product across the whole batch: every chunk transfer (sequential OR concurrent path) takes one permit before the network call and releases it after, so total in-flight is bounded regardless of how W and the per-file degree combine. (The per-chunk MemoryGate still bounds bytes; this bounds the cross-file count.)

kGlobalMaxInflightChunks → const int

Total chunks allowed in flight across ALL files × their chunks. At ~2 MB live per chunk this is a ~16 MB ceiling — matters on mobile (CrispCloud).

Functions

runWithConcurrency<T>(Iterable<T> items, int concurrency, Future<void> action(T item)) → Future<void>

Run action over items with at most concurrency in flight at once. Mirrors internxt-dart's runWithConcurrency: a ChunkSemaphore gates how many item futures are active; the rest queue. This is the file-level (Step 2) batch primitive — each whole-file transfer is one item, and the per-file chunk concurrency (Step 1) composes underneath. Completes once every item has finished. If action throws for an item, that error propagates out of the returned future (callers that must not abort the batch should catch inside action and return a sentinel instead).