edge_veda library

Edge Veda SDK - On-device LLM inference for Flutter

Example usage:

import 'package:edge_veda/edge_veda.dart';

final edgeVeda = EdgeVeda();
await edgeVeda.init(EdgeVedaConfig(modelPath: '/path/to/model.gguf'));
final response = await edgeVeda.generate('Hello, world!');
print(response.text);
await edgeVeda.dispose();

Features

On-device LLM inference with llama.cpp and Metal acceleration
Model download with progress tracking and caching
Memory-safe operations with configurable limits
Zero server costs and 100% offline operation

Model Management

final modelManager = ModelManager();

// Download a pre-configured model
final modelPath = await modelManager.downloadModel(
  ModelRegistry.llama32_1b,
);

// Monitor download progress
modelManager.downloadProgress.listen((progress) {
  print('Progress: ${progress.progressPercent}%');
});

// Check downloaded models
final models = await modelManager.getDownloadedModels();
print('Downloaded: $models');

Memory Monitoring

// Check memory usage
final stats = await edgeVeda.getMemoryStats();
print('Memory: ${(stats.usagePercent * 100).toStringAsFixed(1)}%');

// Quick pressure check
if (await edgeVeda.isMemoryPressure()) {
  print('High memory usage!');
}

Classes

CancelToken

Token for cancelling ongoing operations (downloads, generation)

DownloadProgress

Model download progress information

EdgeVeda

Main Edge Veda SDK class for on-device AI inference

EdgeVedaConfig

Configuration for initializing Edge Veda SDK

GenerateOptions

Options for text generation

GenerateResponse