gpuActivationFor function
- required LlmModelFamily? family,
- required bool? declaresActivation,
- required int maxNumTokens,
- required bool android,
The activation type a GPU load runs at when the caller set none: float32
for Gemma 4 with a context above gemma4Fp16MaxContextTokens (its float16
rotary positions stop being distinct there, LiteRT-LM #4001), and for a
FunctionGemma bundle that declares no prefer_activation_type (converted
before litetune 0.1.4: at float16 it answers with <pad> or nothing).
family is the bundle's llm_model_type; declaresActivation whether it
declares a type (null: unread). Null leaves the bundle's own.
On Android (android) Gemma 4 keeps the bundle's float16 at any context,
with GpuActivationReason.gemma4LongContextAndroid above
gemma4Fp16MaxContextTokens so the engine says so: float32 fails there on
Adreno 8xx with this native build. An explicit float32 is still honoured.
Implementation
({ActivationDataType? type, GpuActivationReason? reason}) gpuActivationFor({
required LlmModelFamily? family,
required bool? declaresActivation,
required int maxNumTokens,
required bool android,
}) => switch (family) {
LlmModelFamily.gemma4
when maxNumTokens > gemma4Fp16MaxContextTokens && android =>
(type: null, reason: GpuActivationReason.gemma4LongContextAndroid),
LlmModelFamily.gemma4 when maxNumTokens > gemma4Fp16MaxContextTokens => (
type: ActivationDataType.float32,
reason: GpuActivationReason.gemma4LongContext,
),
LlmModelFamily.functionGemma when declaresActivation == false => (
type: ActivationDataType.float32,
reason: GpuActivationReason.functionGemmaWithoutKey,
),
_ => (type: null, reason: null),
};