flutter_gpt_engine 0.0.1

Flutter_GPT_Engine - headless/offline GGUF chat engine for Flutter. UI is fully controlled by the host app.

Flutter_GPT_Engine_Engine

Flutter_GPT_Engine_Engine is a headless local GGUF chat engine for Flutter.

It does not provide any chat screen, bubble, text field, app bar, theme, or other UI.

The host app owns 100% of the UI.

The package handles:

• GGUF model selection using File Picker
• Flutter asset model loading
• direct filesystem-path model loading
• GGUF validation
• GPU detection
• GPU -> CPU fallback
• model lifecycle
• streaming generation
• conversation history
• stop generation
• clear chat
• unload/dispose
• generation configuration

Package name

name: flutter_gpt_engine

Import:

import 'package:flutter_gpt_engine/flutter_gpt_engine.dart';

1. Create the client

final gpt = LocalLlmClient(
  config: const LocalLlmConfig(
    systemPrompt: 'You are a helpful offline AI assistant.',
    threads: 4,
    contextSize: 4096,
    maxTokens: 384,
  ),
);

2. User selects a GGUF model using File Picker

Your UI can have any button you want. On tap:

final selected = await gpt.pickModel();

if (selected) {
  print(gpt.model?.name);
}

If you want the picked model copied into Application Support:

await gpt.pickModel(
  persist: true,
);

3. Load from Flutter assets

Host app pubspec.yaml:

flutter:
  assets:
    - assets/models/qwen.gguf

Then:

await gpt.loadAsset(
  'assets/models/qwen.gguf',
);

4. Load from direct file path

await gpt.loadModel(
  '/storage/emulated/0/Download/model.gguf',
);

5. Stream chat response

await for (final token in gpt.generate('Hello')) {
  // Show this token however you want in your own UI.
  print(token);
}

Or get the complete answer:

final answer = await gpt.generateText(
  'Explain Flutter in Bangla.',
);

6. Observe package state

LocalLlmClient extends ChangeNotifier.

Your UI can listen to it:

gpt.addListener(() {
  print(gpt.isLoading);
  print(gpt.isLoaded);
  print(gpt.isGenerating);
  print(gpt.status);
  print(gpt.model);
  print(gpt.messages);
});

Available state:

gpt.isLoading
gpt.isLoaded
gpt.isGenerating
gpt.status
gpt.model
gpt.messages

7. Conversation history

The package automatically keeps user/assistant messages:

final messages = gpt.messages;

for (final message in messages) {
  print('${message.role}: ${message.text}');
}

Your UI can render these messages however it wants.

8. Stop / clear / unload

await gpt.stop();

await gpt.clearChat();

await gpt.unloadModel();

9. Dispose

gpt.dispose();

If deterministic native cleanup is needed before destroying the object:

await gpt.unloadModel();
gpt.dispose();

Architecture

YOUR APP UI
   |
   |-- Pick Model button
   |-- TextField
   |-- Send button
   |-- Chat bubbles
   |
   v
Flutter_GPT_Engine
   |
   |-- File Picker
   |-- GGUF validation
   |-- Asset -> local file preparation
   |-- llama_flutter_android
   |-- GPU detection
   |-- CPU fallback
   |-- chat history
   |-- streaming
   |-- stop / clear / unload

The package intentionally contains no UI implementation.