applyThinkingTokenBudget function

  1. @visibleForTesting
void applyThinkingTokenBudget(
  1. LiteRtLmBindings b,
  2. Pointer<LiteRtLmConversationOptionalArgs> optionalArgs, {
  3. required bool thinking,
  4. required int? budget,
})

Gives one send's optionalArgs a thinking token budget of budget when the send thinkings and a budget is set; otherwise touches nothing, so a send without a budget reaches LiteRT-LM exactly as before.

LiteRT-LM applies the budget at decode time to the conversation's first channel (thought on the Gemma 4 bundles); a bundle without that channel ignores it. The args keep a copy of the config, which is deleted here.

Implementation

@visibleForTesting
void applyThinkingTokenBudget(
  LiteRtLmBindings b,
  Pointer<LiteRtLmConversationOptionalArgs> optionalArgs, {
  required bool thinking,
  required int? budget,
}) {
  if (!thinking || budget == null) return;
  // A failure here throws before any generation starts.
  final config = createLiteRtLm(
    'litert_lm_thinking_config_create',
    b.litert_lm_thinking_config_create,
    b.lastError,
  );
  try {
    checkLiteRtLmSetter(
      'litert_lm_thinking_config_set_enable_thinking',
      b.litert_lm_thinking_config_set_enable_thinking(config, true),
      b.lastError,
    );
    checkLiteRtLmSetter(
      'litert_lm_thinking_config_set_thinking_token_budget',
      b.litert_lm_thinking_config_set_thinking_token_budget(config, budget),
      b.lastError,
    );
    checkLiteRtLmSetter(
      'litert_lm_conversation_optional_args_set_thinking_config',
      b.litert_lm_conversation_optional_args_set_thinking_config(
        optionalArgs,
        config,
      ),
      b.lastError,
    );
  } finally {
    b.litert_lm_thinking_config_delete(config);
  }
}