mnnllm 0.0.2

A Flutter Plugin binding for MNN-LLM, enable LLM inference on mobile phones.

mnnllm (MNN-LLM Flutter Plugin)

A Flutter Plugin for Android and IOS based on MNN-LLM.

	Android	iOS
Support	SDK 26+	Not Supported

Getting Started

Add mnnllm as a dependency in your pubspec.yaml file.

dependencies:
  mnnllm: ^{{latest version}}

Replace {{latest version}} with the version number in badge above.

Runtime Screenshot

Usage

1. Import Package.

import 'package:mnnllm/mnnllm.dart';

2. Download Model.

await _mnnllmPlugin.downloadWithListen(modelID, listen: (state) {
      setState(() {
        downloadingState = state;
      });
    });

3. Initialize Model.

int? val = await _mnnllmPlugin.initModel(
        modelID,
        _mnnllmPlugin.getModelName(modelID),
        configPath,
        _mnnllmPlugin.getSessionId(null),
        keepHistory: false
    );

4. Generate Tokens.

_mnnllmPlugin.generateStreamAnswer(inputText,
            (data) { setState(() { generateText = data.toString(); }); },
            (err) { print(err.toString()); });

Warnings

• The development for the iOS platform is still proceeding in pace.

RoadMap

• Support for iOS platform
• Support for text-to-image generation models
• Support for multimodal large models

Contact Me

Liu [e-mail]