ZeticMLange Flutter

Flutter SDK for running ZeticMLange on-device AI models on Android and iOS.

ZeticMLange loads models deployed from the ZETIC dashboard and runs inference through the native Android and iOS runtimes. The Flutter package exposes Dart APIs for general model inference, supported Hugging Face models, and LLM token generation.

Features

  • Run ZeticMLange models on-device from Flutter.
  • Load models by personal key, model name, and optional version.
  • Load first-party models with a rotatable application runtime key and typed activation errors.
  • Run tensor-based inference with Dart Tensor values.
  • Stream generated tokens from on-device LLM models.
  • Inject retrieved chunks or use a local on-device RAG pipeline with LLM models.
  • Select model mode, quantization, target, accelerator type, and cache policy.
  • Download general and LLM model artifacts in the background and remove their managed cache.
  • Use the same Dart API surface on Android and iOS.

Platform support

Platform Minimum
Android API 24+
iOS iOS 16.6+
Dart 3.11.5+
Flutter 3.35.0+

Installation

Add the package to your Flutter app:

dependencies:
  zetic_mlange: ^1.11.1

Then install dependencies:

flutter pub get

Repository configuration

The Android plugin adds no repositories to your projects. Add the public ZeticMLange release repository to the repository list your app already uses, usually android/settings.gradle.kts:

dependencyResolutionManagement {
    repositoriesMode.set(RepositoriesMode.PREFER_SETTINGS)
    repositories {
        google()
        mavenCentral()
        ivy {
            name = "ZeticMLangeAndroidGitHubRelease"
            url = uri("https://github.com/zetic-ai/ZeticMLangeAndroid/releases/download")
            patternLayout {
                artifact("[revision]/[artifact]-[revision](-[classifier]).[ext]")
            }
            metadataSources {
                gradleMetadata()
            }
            content {
                includeGroup("com.zeticai.mlange")
                includeGroup("com.zeticai.mlange.backend")
            }
        }
    }
}

The GitHub Releases Ivy repository serves the Android SDK modules. Keep google() for com.google.android.play:ai-delivery and AndroidX dependencies, and mavenCentral() for their other public transitive dependencies. The plugin does not inject any repositories into the consuming app.

Flutter's own Gradle plugin adds download.flutter.io to rootProject.allprojects, so dependencyResolutionManagement { repositoriesMode = FAIL_ON_PROJECT_REPOS } fails for any Flutter app regardless of this package. Use PREFER_SETTINGS. See docs/agp9-consumer-compatibility.md for the reproduction and the supported AGP matrix.

Flutter 3.44 and AGP 9.0.x

ZeticMLange is verified with Flutter 3.44.x, AGP 9.0.x, Gradle 9.1, JDK 17, and compileSdk 34. Add this setting to android/gradle.properties when using that toolchain:

android.newDsl=false

This is required by Flutter 3.44's Gradle plugin under AGP 9.0.x; it is not a package-specific setting. Keep repositories in settings.gradle.kts and use PREFER_SETTINGS: FAIL_ON_PROJECT_REPOS is incompatible with Flutter's own repository injection. The published ZeticMLange plugin does not add repositories. AGP 9.2 and later require a newer Flutter, so they are outside the Flutter 3.44.x support scope. See the AGP 9 consumer compatibility guide for the full configuration and caveat.

On iOS, declare the platform in ios/Podfile, because CocoaPods otherwise assigns iOS 13.0 and refuses to resolve a pod that requires iOS 16.6:

platform :ios, '16.6'

This package requires the ZeticMLange native runtime to be available in the host app. Follow the Flutter setup guide for Android and iOS integration:

Basic inference

import 'dart:typed_data';

import 'package:zetic_mlange/zetic_mlange.dart';

Future<Float32List> runModel({
  required String personalKey,
  required String modelName,
  required Float32List inputValues,
}) async {
  final model = await ZeticMLangeModel.create(
    personalKey: personalKey,
    name: modelName,
  );

  try {
    final input = Tensor.float32List(
      inputValues,
      shape: const [1, 3, 224, 224],
    );

    final outputs = model.run([input]);
    return outputs.first.asFloat32List();
  } finally {
    model.close();
  }
}

Background model downloads

Schedule a download before the app is backgrounded, persist the returned handle.id, and reconstruct BackgroundDownloadHandle(handleId) after the app or UI has been recreated to observe or stop the durable native job. The same methods are available on ZeticMLangeLLMModel.

final handle = await ZeticMLangeModel.downloadInBackground(
  personalKey: personalKey,
  name: modelName,
);

final status = await ZeticMLangeModel.getBackgroundDownloadStatus(handle);
if (!status.isTerminal) {
  await ZeticMLangeModel.stopBackgroundDownload(handle);
}

final removal = await ZeticMLangeModel.removeDownloadedModel(name: modelName);
if (removal.isInUse) {
  // Close active model instances, then retry removal.
}

BackgroundDownloadStatus is the latest persisted transfer snapshot. Its progress is best-effort and is null when the SDK does not yet know the total byte count; it is not a live progress event stream.

On iOS, await downloadInBackground before entering the background. Also forward the host app's background URL-session lifecycle event to the native SDK so a relaunched app can complete verification and cache installation:

override func application(
  _ application: UIApplication,
  handleEventsForBackgroundURLSession identifier: String,
  completionHandler: @escaping () -> Void
) {
  ZeticMLangeModel.handleBackgroundURLSessionEvents(
    identifier: identifier,
    completionHandler: completionHandler
  )
}

LLM generation

import 'package:zetic_mlange/zetic_mlange.dart';

Future<void> generateText({
  required String personalKey,
  required String modelName,
}) async {
  final llm = await ZeticMLangeLLMModel.create(
    personalKey: personalKey,
    name: modelName,
    initOption: const LLMInitOption(nCtx: 4096),
  );

  try {
    llm.run('Explain on-device AI in one paragraph.');

    while (true) {
      final next = llm.waitForNextToken();
      if (next.isFinished) {
        break;
      }
      print(next.token);
    }

    llm.cleanUp();
  } finally {
    llm.close();
  }
}

Runtime-key activation

Dashboard-issued runtime keys bind model activation to an SDK application. Create one ZeticMLangeRuntime and use it to load general or LLM models. The existing personal-key APIs remain available for compatibility.

ZeticMLangeModel? model;
try {
  final runtime = ZeticMLangeRuntime(runtimeKey: runtimeKey);
  model = await runtime.loadModel(
    name: 'owner/model',
  );
  final outputs = model.run(inputs);
} on MlangeActivationException catch (error) {
  switch (error.code) {
    case MlangeActivationErrorCode.sdkUpgradeRequired:
      // Upgrade the bundled Android/iOS SDK before retrying.
    case MlangeActivationErrorCode.activationAllowanceExhausted:
    case MlangeActivationErrorCode.paygDisabled:
    case MlangeActivationErrorCode.paygCapReached:
    case MlangeActivationErrorCode.paymentRequired:
    case MlangeActivationErrorCode.activationRequired:
    case MlangeActivationErrorCode.unknown:
      rethrow;
  }
} finally {
  model?.close();
}

runtime.loadLLMModel(name: 'owner/llm') provides the same typed-error contract for LLMs. A first offline run without a server-issued activation receipt fails with activationRequired; cached execution after a successful activation is handled by the native SDK. Activation receipts and their lifecycle metadata are SDK implementation details and are not exposed through the Dart API.

Hugging Face models

Supported Hugging Face models can be loaded with ZeticMLangeHFModel:

final model = await ZeticMLangeHFModel.create(
  'owner/repository',
  userAccessToken: userAccessToken,
  cacheHandlingPolicy: CacheHandlingPolicy.keepExisting,
);

try {
  final outputs = model.run(inputs);
} finally {
  model.close();
}

See the Hugging Face model guide for supported formats and deployment requirements.

LLM tool calling

Register tool callbacks before starting a tool-enabled turn. runWithTools() returns generated tokens as a Stream<String> and invokes callbacks asynchronously. Calling run() while tools are registered throws.

llm.registerTool(
  const LLMToolSpec(
    name: 'weather',
    description: 'Looks up a city forecast.',
    parametersJson: '{"type":"object"}',
  ),
  (call) async => const LLMToolResult(content: '{"forecast":"sunny"}'),
);

await for (final token in llm.runWithTools('What is the weather?')) {
  print(token);
}

Lifecycle

ZeticMLangeModel, ZeticMLangeHFModel, and ZeticMLangeLLMModel expose close() and isClosed. Call close() when a model is no longer needed to release native runtime resources; repeated close() calls are safe no-ops. After isClosed is true, inference and token APIs throw MlangeException before invoking native FFI handles. ZeticMLangeLLMModel.deinit() remains as a deprecated compatibility alias for close().

RAG generation

External retrievers can pass ranked chunks through RagPipeline:

final rag = RagPipeline(
  retriever: myRetriever,
  llm: llm,
  profile: const RagProfile.qwen25(),
);

await for (final token in rag.respond(query: 'What does the SDK support?')) {
  append(token);
}

For end-to-end on-device retrieval, create a local RAG pipeline with decoder and embedder GGUF files, index documents, then use it as the retriever:

final localRag = await LocalRagPipeline.create(
  profile: const RagProfile.qwen25(backboneGgufPath: 'decoder.gguf'),
  embedderGgufPath: 'embedder.gguf',
);

await localRag.indexDocs(const [
  RagDocument(text: 'RAG injects retrieved context.', source: 'docs/rag.md'),
]);

final rag = RagPipeline(
  retriever: localRag,
  llm: llm,
  profile: const RagProfile.qwen25(backboneGgufPath: 'decoder.gguf'),
);

await for (final token in rag.respond(query: 'Explain RAG.')) {
  append(token);
}

Public API

The package exports:

  • ZeticMLangeModel
  • ZeticMLangeHFModel
  • ZeticMLangeLLMModel
  • RagPipeline, RagProfile, RetrievedChunk, RagDocument, LocalRagPipeline
  • Tensor
  • DataType, Target, APType, ModelMode
  • LLMModelMode, LLMTarget, LLMInitOption
  • LLMKVCacheCleanupPolicy, CacheHandlingPolicy
  • MlangeException
  • MlangeActivationException, MlangeActivationErrorCode

Documentation

Full documentation is available at docs.zetic.ai.

Useful starting points:

License

Apache-2.0. See LICENSE.

Development

Public native SDK metadata updates

The retained Update verified native SDK metadata (native-sdk-ready) workflow updates public native release metadata in this repository. It accepts only schema-v1 android-sdk-ready and ios-sdk-ready repository-dispatch events and creates or updates an App-owned draft pull request. It never merges or publishes the Flutter package, and it is separate from same-source monorepo development.

Android payloads contain readiness_key, mlange_version, runtime_version, central_state, sorted first-party coordinates, and verification_run_url. The Android key is validated as lowercase SHA-256 provenance; SW-791 cannot be recomputed here because its stable input includes an undispatched Central deployment ID.

iOS payloads contain readiness_key, ios_version, the immutable public artifact_url, sha256, manifest_commit, and verification_run_url. The consumer recomputes the key from canonical compact JSON with sorted keys over artifact_url, ios_version, manifest_commit, and sha256.

Logical releases use deterministic branches:

  • feature/SW-812-android-sdk-<mlange_version>-<runtime_version>
  • feature/SW-812-ios-sdk-<ios_version>

Identical delivery updates the matching draft or exits successfully if main already contains the desired values and matching provenance. Lookup covers all pull-request states so a closed or conflicting logical release cannot create a duplicate draft. New drafts use the repository review template and assign shinil-zetic. Downgrades, conflicting same-version metadata, malformed contracts, and unexpected file sentinels fail before a branch or draft is created. Android aggregate and runtime versions are compared independently, and neither may move backward.

For a new deterministic branch, automation first creates a content-free ref at the selected main commit. GitHub's GraphQL createCommitOnBranch mutation then creates a GitHub-signed App commit and advances the ref atomically with an expected-head check; ref updates are never forced. If publication fails after the bootstrap, an identical event may recover that ref only while it remains an ancestor of current main with no branch-side content. A published orphan is recoverable only when GitHub reports a valid signature by the configured App bot and the single commit still has the exact readiness message, expected sentinel values, and an approved-only complete diff. Unrelated later changes on main are not copied into that branch.

The workflow mints a short-lived token from the organization Actions variable SDK_RELEASE_APP_ID and secret SDK_RELEASE_APP_PRIVATE_KEY. The GitHub App must be installed on mlange_sdk with repository contents and pull-request write permissions. Git and gh both use that App token; no user PAT is needed. The consumer also requires the dispatch sender to match the bot login derived from that App token and requires the target repository to be zetic-ai/mlange_sdk.

Run the deterministic updater tests without GitHub credentials:

python3 -m unittest discover -s scripts/tests -p 'test_*.py'

The tests use temporary Git repositories and recording GitHub boundaries. Live dispatch and App-authenticated evidence belongs to SW-793.

Monorepo native development

Android, iOS, and Flutter sources live in this checkout, so source-first development uses native artifacts built from the same source revision. It does not consume an internal Android or iOS release.

Build the platform artifacts, then set the local source overrides before building the Flutter example or integration target:

cd ../android
./gradlew \
  :zetic_mlange:assembleRelease \
  :zetic_mlange_core:assembleRelease \
  :backend:qnn_runtime:assembleRelease \
  :backend:qnn_runtime:qualcomm_runtime_v69:assembleRelease \
  :backend:qnn_runtime:qualcomm_runtime_v73:assembleRelease \
  :backend:qnn_runtime:qualcomm_runtime_v75:assembleRelease \
  :backend:qnn_runtime:qualcomm_runtime_v79:assembleRelease \
  :backend:qnn_runtime:qualcomm_runtime_v81:assembleRelease \
  :backend:tflite_runtime:assembleRelease \
  :backend:ort_runtime:assembleRelease \
  :backend:executorch_runtime:assembleRelease \
  :backend:llama_cpp_runtime:assembleRelease \
  :backend:litert_runtime:assembleRelease \
  :backend:llm_runtime:assembleRelease

cd ../ios
./build_spm.sh

cd ../flutter
export ZETIC_MLANGE_ANDROID_REPO="$(cd ../android && pwd)"
export ZETIC_MLANGE_IOS_REPO="$(cd ../ios && pwd)"

ZETIC_MLANGE_ANDROID_REPO stages the Android AARs in a local Maven repository. ZETIC_MLANGE_IOS_REPO selects build/ZeticMLange.xcframework; use ZETIC_MLANGE_IOS_XCFRAMEWORK to select a different local XCFramework directly. Source-first CI supplies these overrides explicitly.

An in-tree Flutter checkout without the required local override fails closed. It never silently falls back to a public native artifact from another source revision. The published pub.dev package is different: it has no monorepo-local repository wiring, resolves com.zeticai.mlange:mlange from the public Android GitHub Release repository configured by the consuming app, and downloads the versioned, checksummed public ZeticMLangeiOS XCFramework. Public native version and archive metadata remain updated through the native-sdk-ready workflow above.

Publishing to pub.dev

Run the publish guard before publishing:

python3 scripts/check_publish_config.py

It asks pub which files it would upload and fails if any carry source-checkout configuration, a repository injection into the consumer's projects, or compiled Python bytecode. .pubignore keeps local repository wiring and scripts out of the package; the guard verifies that this remains true. It also runs in CI.

FFI API Development Guide

To add or update FFI APIs exposed by the Android and iOS SDKs:

  1. Modify the FFI Contract: Update lib/src/ffi/mlange_ffi.h with your changes to the C-interface signatures.

  2. Generate Dart Bindings: Run ffigen to update lib/src/ffi/mlange_bindings.dart:

    dart run ffigen

    Note: You must have LLVM (Clang) installed locally.

  3. Update Dart FFI Logic: Modify lib/src/ffi/mlange_abi.dart to match the new signatures or structs.

  4. Cross-Platform Sync: Android and iOS stage this header directly from this Flutter SDK in the monorepo. Run their respective dependency-fetch or build command from the sibling android or ios directory.

Libraries

zetic_mlange