ModelMemoryEstimate class
What loading a model costs, split into the parts that scale differently.
Constructors
- ModelMemoryEstimate({required int weightsBytes, required int kvBytesPerToken, required int fixedOverheadBytes, int? trainedContextTokens})
-
Creates an estimate.
const
Properties
- fixedOverheadBytes → int
-
Compute-graph and output-buffer overhead, independent of context.
final
- hashCode → int
-
The hash code for this object.
no setterinherited
- kvBytesPerToken → int
-
KV-cache cost per context token (f16 K+V across all layers).
final
- runtimeType → Type
-
A representation of the runtime type of the object.
no setterinherited
- trainedContextTokens → int?
-
The context length the model was trained for, when the GGUF reports
it. Contexts beyond it degrade quality, so planners clamp to it.
final
- weightsBytes → int
-
Weight cost — the GGUF file size(s). The file is mmapped, but with
full GPU offload on unified memory it is effectively resident.
final
Methods
-
bytesForContext(
int contextTokens) → int -
Total bytes to run with a
contextTokens-sized window. -
maxContextForBudget(
int budgetBytes) → int -
The largest context that fits in
budgetBytes, or0when even an empty context does not fit. -
noSuchMethod(
Invocation invocation) → dynamic -
Invoked when a nonexistent method or property is accessed.
inherited
-
toString(
) → String -
A string representation of this object.
inherited
Operators
-
operator ==(
Object other) → bool -
The equality operator.
inherited