llamacpp_tools 0.1.2 copy "llamacpp_tools: ^0.1.2" to clipboard
llamacpp_tools: ^0.1.2 copied to clipboard

Tools to manage llama.cpp local setup (detecting, downloading or building, running).

Dart package and CLI tool for managing llama.cpp local setup (detecting, downloading, building, running).

Features #

  • Directory detection: Detect existing llama.cpp installations and get version info
  • GitHub integration: Scrape recent versions and download/setup releases from GitHub
  • Docker building: Build llama.cpp with CUDA support using containerized environments
  • Process management: Manages llama-server processes with configuration, lifecycle control, and monitoring

Usage #

GitHub Operations #

# List recent versions from GitHub
dart run llamacpp_tools github scrape-recent-versions

# Download and setup a specific release
dart run llamacpp_tools github setup-release --version 3875 --target ./llama-cpp

Docker Building #

# List available builders
dart run llamacpp_tools docker list-builders

# Build a container
dart run llamacpp_tools docker build-builder --builder cuda-builder --container llama-cuda

# Run container to build llama.cpp
dart run llamacpp_tools docker run-builder --container llama-cuda --version 3875 --output ./output

Process Management #

import 'package:llamacpp_tools/llamacpp_tools.dart';

// Configure and start a server
final config = LlamaserverConfig(
  modelPath: '/path/to/model.gguf',
  port: 8080,
  threads: 4,
);

final process = LlamaserverProcess(
  dir: LlamacppDir('/path/to/llama-cpp'),
  config: config,
);

await process.start();
print('Server running on port ${process.port}');
0
likes
150
points
305
downloads

Publisher

verified publisheragilord.com

Weekly Downloads

Tools to manage llama.cpp local setup (detecting, downloading or building, running).

Repository (GitHub)

Topics

#llamacpp #llama-cpp #cli #llm #tool

Documentation

API reference

License

BSD-3-Clause (license)

Dependencies

args, html, http, json_annotation, path, process_visor

More

Packages that depend on llamacpp_tools