flutter_edge_ai_litertlm 1.8.7

LiteRT-LM (.litertlm) on-device inference engine for flutter_edge_ai via dart:ffi (5 native platforms) + web. Opt-in InferenceEngineProvider. Also ships the LiteRT C API embedding backend (LiteRtEmbed [...]

example/README.md

flutter_edge_ai_litertlm example

flutter_edge_ai_litertlm is an opt-in inference engine for flutter_edge_ai. It runs .litertlm models via dart:ffi on the 5 native platforms (and via @litert-lm/core on web). Register the engine once at startup, then use the unchanged inference API.

import 'package:flutter/widgets.dart';
import 'package:flutter_edge_ai/flutter_edge_ai.dart';
import 'package:flutter_edge_ai_litertlm/flutter_edge_ai_litertlm.dart';

Future<void> main() async {
  WidgetsFlutterBinding.ensureInitialized();

  // Opt into the LiteRT-LM engine (handles ModelFileType.litertlm).
  await FlutterEdgeAi.initialize(
    inferenceEngines: [LiteRtLmEngine()],
  );

  // Install a .litertlm model (downloads + sets it active).
  await FlutterEdgeAi.installModel(
    modelType: ModelType.gemmaIt,
    fileType: ModelFileType.litertlm,
  ).fromNetwork('https://example.com/gemma3-1b-it.litertlm').install();

  // Create a model + session and generate.
  final model = await FlutterEdgeAi.getActiveModel(maxTokens: 1024);
  final session = await model.createSession();
  await session.addQueryChunk(const Message(text: 'Hello!', isUser: true));
  final reply = await session.getResponse();
  print(reply);

  await session.close();
  await model.close();
}

Pass LiteRtLmEngine() alongside other engines (e.g. MediaPipeEngine from flutter_edge_ai_mediapipe) if your app uses both .litertlm and .task models. Web inference is an early preview — see the package README for the web/index.html handshake. A full runnable app lives in the flutter_edge_ai example.

Embeddings

As of 1.5.0 this package also ships the LiteRT C API embedding backend (LiteRtEmbeddingBackend, Gecko / EmbeddingGemma .tflite — moved here from flutter_edge_ai_embeddings, which now supplies the tokenizers this backend asks core for):

import 'package:flutter_edge_ai_embeddings/flutter_edge_ai_embeddings.dart';
import 'package:flutter_edge_ai_litertlm/flutter_edge_ai_litertlm.dart';

await FlutterEdgeAi.initialize(
  embeddingBackends: [LiteRtEmbeddingBackend()],
  embeddingTokenizers: [GemmaEmbeddingTokenizers()],
);