flutter_gemma_litertlm 1.0.1

LiteRT-LM (.litertlm) on-device inference engine for flutter_gemma via dart:ffi (5 native platforms) + web. Opt-in InferenceEngineProvider.

example/README.md

flutter_gemma_litertlm example

flutter_gemma_litertlm is an opt-in inference engine for flutter_gemma. It runs .litertlm models via dart:ffi on the 5 native platforms (and via @litert-lm/core on web). Register the engine once at startup, then use the unchanged inference API.

import 'package:flutter/widgets.dart';
import 'package:flutter_gemma/flutter_gemma.dart';
import 'package:flutter_gemma_litertlm/flutter_gemma_litertlm.dart';

Future<void> main() async {
  WidgetsFlutterBinding.ensureInitialized();

  // Opt into the LiteRT-LM engine (handles ModelFileType.litertlm).
  await FlutterGemma.initialize(
    inferenceEngines: [LiteRtLmEngine()],
  );

  // Install a .litertlm model (downloads + sets it active).
  await FlutterGemma.installModel(
    modelType: ModelType.gemmaIt,
    fileType: ModelFileType.litertlm,
  ).fromNetwork('https://example.com/gemma3-1b-it.litertlm').install();

  // Create a model + session and generate.
  final model = await FlutterGemma.getActiveModel(maxTokens: 1024);
  final session = await model.createSession();
  await session.addQueryChunk(const Message(text: 'Hello!', isUser: true));
  final reply = await session.getResponse();
  print(reply);

  await session.close();
  await model.close();
}

Pass LiteRtLmEngine() alongside other engines (e.g. MediaPipeEngine from flutter_gemma_mediapipe) if your app uses both .litertlm and .task models. Web inference is an early preview — see the package README for the web/index.html handshake. A full runnable app lives in the flutter_gemma example.