ZeticMLange Flutter

Flutter SDK for running ZeticMLange on-device AI models on Android and iOS.

ZeticMLange loads models deployed from the ZETIC dashboard and runs inference through the native Android and iOS runtimes. The Flutter package exposes Dart APIs for general model inference, supported Hugging Face models, and LLM token generation.

Features

  • Run ZeticMLange models on-device from Flutter.
  • Load models by personal key, model name, and optional version.
  • Load first-party models with a rotatable application runtime key and typed activation errors.
  • Run tensor-based inference with Dart Tensor values.
  • Stream generated tokens from on-device LLM models.
  • Inject retrieved chunks or use a local on-device RAG pipeline with LLM models.
  • Select model mode, quantization, target, accelerator type, and cache policy.
  • Use the same Dart API surface on Android and iOS.

Platform support

Platform Minimum
Android API 24+
iOS iOS 16.6+
Dart 3.11.5+
Flutter 3.35.0+

Installation

Add the package to your Flutter app:

dependencies:
  zetic_mlange: ^1.9.1

Then install dependencies:

flutter pub get

Repository configuration

The Android plugin adds no repositories to your projects. It resolves its native dependencies from the repositories your app already declares, which for a flutter create app are google() and mavenCentral() in android/build.gradle.kts. If you centralize repositories in settings.gradle.kts instead, declare both there: mavenCentral() serves com.zeticai.mlange:*, and google() is required for com.google.android.play:ai-delivery and androidx.security:security-crypto, which are not published to Maven Central.

Flutter's own Gradle plugin adds download.flutter.io to rootProject.allprojects, so dependencyResolutionManagement { repositoriesMode = FAIL_ON_PROJECT_REPOS } fails for any Flutter app regardless of this package. Use PREFER_SETTINGS. See docs/agp9-consumer-compatibility.md for the reproduction and the supported AGP matrix.

Flutter 3.44 and AGP 9.0.x

ZeticMLange is verified with Flutter 3.44.x, AGP 9.0.x, Gradle 9.1, JDK 17, and compileSdk 34. Add this setting to android/gradle.properties when using that toolchain:

android.newDsl=false

This is required by Flutter 3.44's Gradle plugin under AGP 9.0.x; it is not a package-specific setting. Keep repositories in settings.gradle.kts and use PREFER_SETTINGS: FAIL_ON_PROJECT_REPOS is incompatible with Flutter's own repository injection. The published ZeticMLange plugin does not add repositories. AGP 9.2 and later require a newer Flutter, so they are outside the Flutter 3.44.x support scope. See the AGP 9 consumer compatibility guide for the full configuration and caveat.

On iOS, declare the platform in ios/Podfile, because CocoaPods otherwise assigns iOS 13.0 and refuses to resolve a pod that requires iOS 16.6:

platform :ios, '16.6'

This package requires the ZeticMLange native runtime to be available in the host app. Follow the Flutter setup guide for Android and iOS integration:

Basic inference

import 'dart:typed_data';

import 'package:zetic_mlange/zetic_mlange.dart';

Future<Float32List> runModel({
  required String personalKey,
  required String modelName,
  required Float32List inputValues,
}) async {
  final model = await ZeticMLangeModel.create(
    personalKey: personalKey,
    name: modelName,
  );

  try {
    final input = Tensor.float32List(
      inputValues,
      shape: const [1, 3, 224, 224],
    );

    final outputs = model.run([input]);
    return outputs.first.asFloat32List();
  } finally {
    model.close();
  }
}

LLM generation

import 'package:zetic_mlange/zetic_mlange.dart';

Future<void> generateText({
  required String personalKey,
  required String modelName,
}) async {
  final llm = await ZeticMLangeLLMModel.create(
    personalKey: personalKey,
    name: modelName,
    initOption: const LLMInitOption(nCtx: 4096),
  );

  try {
    llm.run('Explain on-device AI in one paragraph.');

    while (true) {
      final next = llm.waitForNextToken();
      if (next.isFinished) {
        break;
      }
      print(next.token);
    }

    llm.cleanUp();
  } finally {
    llm.close();
  }
}

Runtime-key activation

Dashboard-issued runtime keys bind model activation to an SDK application. Use createWithRuntimeKey for new integrations; the existing personal-key APIs remain available for compatibility.

ZeticMLangeModel? model;
try {
  model = await ZeticMLangeModel.createWithRuntimeKey(
    runtimeKey: runtimeKey,
    name: 'owner/model',
  );
  final activation = model.runtimeActivation!;
  print('Activated ${activation.activationId}: ${activation.state}');
  final outputs = model.run(inputs);
} on MlangeActivationException catch (error) {
  switch (error.code) {
    case MlangeActivationErrorCode.sdkUpgradeRequired:
      // Upgrade the bundled Android/iOS SDK before retrying.
    case MlangeActivationErrorCode.activationAllowanceExhausted:
    case MlangeActivationErrorCode.paygDisabled:
    case MlangeActivationErrorCode.paygCapReached:
    case MlangeActivationErrorCode.paymentRequired:
    case MlangeActivationErrorCode.activationRequired:
    case MlangeActivationErrorCode.unknown:
      rethrow;
  }
} finally {
  model?.close();
}

ZeticMLangeLLMModel.createWithRuntimeKey provides the same activation and typed-error contract for LLMs. A first offline run without a server-issued activation receipt fails with activationRequired; cached execution after a successful activation is handled by the native SDK. Runtime-created models expose an immutable MlangeRuntimeActivation with an activation ID, completion state, and receipt status. Receipt tokens are intentionally never exposed to Dart. Models created with personal keys or local assets have runtimeActivation == null.

Hugging Face models

Supported Hugging Face models can be loaded with ZeticMLangeHFModel:

final model = await ZeticMLangeHFModel.create(
  'owner/repository',
  userAccessToken: userAccessToken,
  cacheHandlingPolicy: CacheHandlingPolicy.keepExisting,
);

try {
  final outputs = model.run(inputs);
} finally {
  model.close();
}

See the Hugging Face model guide for supported formats and deployment requirements.

LLM tool calling

Register tool callbacks before starting a tool-enabled turn. runWithTools() returns generated tokens as a Stream<String>. Callbacks can return an LLMToolResult immediately or a Future<LLMToolResult>. Calling run() while tools are registered throws.

llm.registerTool(
  const LLMToolSpec(
    name: 'weather',
    description: 'Looks up a city forecast.',
    parametersJson: '{"type":"object"}',
  ),
  (call) async => const LLMToolResult(content: '{"forecast":"sunny"}'),
);

await for (final token in llm.runWithTools('What is the weather?')) {
  print(token);
}

Lifecycle

ZeticMLangeModel, ZeticMLangeHFModel, and ZeticMLangeLLMModel expose close() and isClosed. Call close() when a model is no longer needed to release native runtime resources; repeated close() calls are safe no-ops. After isClosed is true, inference and token APIs throw MlangeException before invoking native FFI handles. ZeticMLangeLLMModel.deinit() remains as a deprecated compatibility alias for close().

RAG generation

External retrievers can pass ranked chunks through RagPipeline:

final rag = RagPipeline(
  retriever: myRetriever,
  llm: llm,
  profile: const RagProfile.qwen25(),
);

await for (final token in rag.respond(query: 'What does the SDK support?')) {
  append(token);
}

For end-to-end on-device retrieval, create a local RAG pipeline with decoder and embedder GGUF files, index documents, then use it as the retriever:

final localRag = await LocalRagPipeline.create(
  profile: const RagProfile.qwen25(backboneGgufPath: 'decoder.gguf'),
  embedderGgufPath: 'embedder.gguf',
);

await localRag.indexDocs(const [
  RagDocument(text: 'RAG injects retrieved context.', source: 'docs/rag.md'),
]);

final rag = RagPipeline(
  retriever: localRag,
  llm: llm,
  profile: const RagProfile.qwen25(backboneGgufPath: 'decoder.gguf'),
);

await for (final token in rag.respond(query: 'Explain RAG.')) {
  append(token);
}

Public API

The package exports:

  • ZeticMLangeModel
  • ZeticMLangeHFModel
  • ZeticMLangeLLMModel
  • RagPipeline, RagProfile, RetrievedChunk, RagDocument, LocalRagPipeline
  • Tensor
  • DataType, Target, APType, ModelMode
  • LLMModelMode, LLMTarget, LLMInitOption
  • LLMKVCacheCleanupPolicy, CacheHandlingPolicy
  • MlangeException
  • MlangeActivationException, MlangeActivationErrorCode

Documentation

Full documentation is available at docs.zetic.ai.

Useful starting points:

License

Apache-2.0. See LICENSE.

Development

Native SDK release updates

Verified native releases update this repository through the Update verified native SDK metadata workflow. It accepts only schema-v1 android-sdk-ready and ios-sdk-ready repository-dispatch events and creates or updates an App-owned draft pull request. It never merges or publishes the Flutter package.

Android payloads contain readiness_key, mlange_version, runtime_version, central_state, sorted first-party coordinates, and verification_run_url. The Android key is validated as lowercase SHA-256 provenance; SW-791 cannot be recomputed here because its stable input includes an undispatched Central deployment ID.

iOS payloads contain readiness_key, ios_version, the immutable public artifact_url, sha256, manifest_commit, and verification_run_url. The consumer recomputes the key from canonical compact JSON with sorted keys over artifact_url, ios_version, manifest_commit, and sha256.

Logical releases use deterministic branches:

  • feature/SW-812-android-sdk-<mlange_version>-<runtime_version>
  • feature/SW-812-ios-sdk-<ios_version>

Identical delivery updates the matching draft or exits successfully if main already contains the desired values and matching provenance. Lookup covers all pull-request states so a closed or conflicting logical release cannot create a duplicate draft. New drafts use the repository review template and assign shinil-zetic. Downgrades, conflicting same-version metadata, malformed contracts, and unexpected file sentinels fail before a branch or draft is created. Android aggregate and runtime versions are compared independently, and neither may move backward.

For a new deterministic branch, automation first creates a content-free ref at the selected main commit. GitHub's GraphQL createCommitOnBranch mutation then creates a GitHub-signed App commit and advances the ref atomically with an expected-head check; ref updates are never forced. If publication fails after the bootstrap, an identical event may recover that ref only while it remains an ancestor of current main with no branch-side content. A published orphan is recoverable only when GitHub reports a valid signature by the configured App bot and the single commit still has the exact readiness message, expected sentinel values, and an approved-only complete diff. Unrelated later changes on main are not copied into that branch.

The workflow mints a short-lived token from the organization Actions variable SDK_RELEASE_APP_ID and secret SDK_RELEASE_APP_PRIVATE_KEY. The GitHub App must be installed on mlange_flutter with repository contents and pull-request write permissions. Git and gh both use that App token; no user PAT is needed. The consumer also requires the dispatch sender to match the bot login derived from that App token and requires the target repository to be zetic-ai/mlange_flutter.

Run the deterministic updater tests without GitHub credentials:

python3 -m unittest discover -s scripts/tests -p 'test_*.py'

The tests use temporary Git repositories and recording GitHub boundaries. Live cross-repository dispatch and App-authenticated evidence belongs to SW-793.

Public native metadata updates fail closed while native-release-set.json is active. The legacy Android- or iOS-only updater cannot prove that independently delivered artifacts share one FFI contract, so it must not create a partial release-set update. SW-840 tracks the atomic cross-repository automation design required before the next public native release.

Internal native release sets

An internal Flutter build consumes native-release-set.json at the repository root. The file is added only after both native SDK artifacts are available. It pins a non-empty release-set ID, the FFI header commit and SHA-256, Android source/version/GitHub Packages repository and verification run, plus the iOS source/version/XCFramework URL/SHA-256 and verification run. Both platform consumers require every field and verify the checked-in FFI header against the declared SHA-256, so replacing only one platform's metadata is rejected instead of mixing native releases.

The Android artifact is private in GitHub Packages. Configure credentials outside this repository with either Gradle properties:

githubPackagesUsername=YOUR_GITHUB_LOGIN
githubPackagesPassword=YOUR_READ_PACKAGES_TOKEN

or the equivalent GITHUB_ACTOR and GITHUB_TOKEN environment variables. The token needs read:packages; never add it to this repository or to native-release-set.json. ZETIC_MLANGE_ANDROID_REPO and the existing iOS local overrides (ZETIC_MLANGE_IOS_REPO or ZETIC_MLANGE_IOS_XCFRAMEWORK) continue to take precedence for local SDK development.

These credentials are only for builds from this repository. The published pub.dev package ships no native-release-set.json, so it resolves com.zeticai.mlange:mlange from Maven Central and downloads its XCFramework from the public ZeticMLangeiOS release, and needs no GitHub Packages access. Both published pins live next to the code that uses them — publishedMlangeVersion in android/build.gradle.kts and published_ios_version plus its checksum in ios/zetic_mlange.podspec — and must be bumped together with a public native release.

Publishing to pub.dev

Run the publish guard before publishing:

python3 scripts/check_publish_config.py

It asks pub which files it would upload and fails if any of them would carry an internal configuration: native-release-set.json, the private GitHub Packages repository, an internal native version, a repository injection into the consumer's projects, or compiled Python bytecode. .pubignore keeps the internal release set, android/internal-repositories.gradle.kts and scripts/ out of the package; the guard is what verifies that it stayed that way. It also runs in CI.

FFI API Development Guide

To add or update FFI APIs exposed by the Android and iOS SDKs:

  1. Modify the FFI Contract: Update lib/src/ffi/mlange_ffi.h with your changes to the C-interface signatures.

  2. Generate Dart Bindings: Run ffigen to update lib/src/ffi/mlange_bindings.dart:

    dart run ffigen

    Note: You must have LLVM (Clang) installed locally.

  3. Update Dart FFI Logic: Modify lib/src/ffi/mlange_abi.dart to match the new signatures or structs.

  4. Local Development / Cross-Platform Sync: When developing Android or iOS SDKs alongside Flutter, you can sync/copy the local FFI header instead of fetching it from Nexus. Set the MLANGE_FLUTTER_DIR environment variable to point to your local mlange_flutter repository:

    export MLANGE_FLUTTER_DIR="/path/to/mlange_flutter"

    Then run ./fetch-deps.sh in the respective Android or iOS SDK repository.

Libraries

zetic_mlange