createSession abstract method

Future<InferenceModelSession> createSession({
  1. double? temperature,
  2. int? randomSeed,
  3. int? topK,
  4. double? topP,
  5. String? loraPath,
  6. bool? enableVisionModality,
  7. bool? enableAudioModality,
  8. String? systemInstruction,
  9. bool enableThinking = false,
  10. List<Tool> tools = const [],
  11. int? maxOutputTokens,
})
inherited

Creates a new InferenceModelSession for generation.

temperature, randomSeed, topK, topP — sampling. A value you set is used as is; one no engine can sample with throws an ArgumentError. An unset one comes from the model itself when the engine can read it (a native .litertlm bundle's sampler, an ONNX model's genai_config.json), otherwise from the family's published defaults (SamplingParams.forModelType), otherwise from SamplingParams.fallback. Engines that cannot take a value say so once, and two depart from this: ONNX leaves unset fields to its config, and built-in AI sends temperature: 0.8, topK: 1. Pass topK: 1 or temperature: 0 for greedy decoding. On .litertlm the sampler is fixed by the engine's first generation (LiteRT-LM#2080), so a later session with different values needs the model reloaded. loraPath — optional path to LoRA model. enableVisionModality — enable vision modality for multimodal models. enableAudioModality — enable audio modality for Gemma 3n E4B models.

maxOutputTokens — optional cap on how many tokens this session generates per response. This is the GENERATION length, distinct from the model's maxTokens (the whole CONTEXT WINDOW — input + output — passed to createModel/getActiveModel). To make the model produce a short reply, set maxOutputTokens (e.g. 100) and leave maxTokens at the default; do NOT lower maxTokens to 100 — .litertlm models require a context window of at least 1024 and will fail to allocate tensors below it (#318). Currently honored on the .litertlm (FFI) path; the MediaPipe .task path has no session-level output cap and ignores it (a log line is emitted).

Implementation

Future<InferenceModelSession> createSession({
  double? temperature,
  int? randomSeed,
  int? topK,
  double? topP,
  String? loraPath,
  bool? enableVisionModality, // Add vision modality support
  bool? enableAudioModality, // Add audio modality support (Gemma 3n E4B)
  String? systemInstruction,
  bool enableThinking =
      false, // Enable thinking mode (Gemma 4 via extraContext)
  List<Tool> tools =
      const [], // Native tool calling (Gemma 4 → SDK tools_json)
  int? maxOutputTokens, // Cap GENERATED tokens (not the context window)
});