Llama class
A Dart wrapper for llama.cpp functionality.
Constructors
- Llama(String modelPath, {String? mmprojPath, ModelParams? modelParams, ContextParams? contextParams, SamplerParams? samplerParams, bool verbose = false})
- Creates a new Llama instance.
Properties
- batch ↔ llama_batch
-
getter/setter pair
-
context
→ Pointer<
llama_context> -
no setter
- hashCode → int
-
The hash code for this object.
no setterinherited
- isDisposed → bool
-
no setter
-
model
↔ Pointer<
llama_model> -
getter/setter pair
- runtimeType → Type
-
A representation of the runtime type of the object.
no setterinherited
- status → LlamaStatus
-
no setter
-
vocab
↔ Pointer<
llama_vocab> -
getter/setter pair
Methods
-
clear(
) → void -
createSlot(
String slotId) → void -
Creates a new slot (context) in VRAM for a specific user/session.
slotId- Unique identifier for the user. -
dispose(
) → void -
freeSlot(
String slotId) → void - Frees a specific slot from VRAM.
-
generateCompleteText(
{int? maxTokens}) → Future< String> -
generateText(
) → Stream< String> -
generateWithMedia(
String prompt, {required List< LlamaInput> inputs}) → Stream<String> -
getEmbeddings(
String prompt, {bool addBos = true, bool normalize = true}) → List< double> -
getLib(
) → llama_cpp -
getNext(
) → (String, bool) -
getNextWithStatus(
) → (String, bool, bool) -
getRemainingContextSpace(
) → int -
loadSession(
String path) → bool - Loads session from disk. Optimized to read directly from Disk to C-memory.
-
loadState(
Uint8List stateData) → void - Restores state from a RAM byte array.
-
noSuchMethod(
Invocation invocation) → dynamic -
Invoked when a nonexistent method or property is accessed.
inherited
-
saveSession(
String path) → void - Saves session to disk. Optimized to stream directly from C-memory to Disk to avoid RAM spikes.
-
saveState(
) → Uint8List - Saves the current state to RAM (Heap) as a byte array. Useful for fast context switching between users/chats without disk I/O.
-
setPrompt(
String prompt, {void onProgress(int current, int total)?}) → void -
setSlot(
String slotId) → void - Switches the active "brain" to the specified user.
-
tokenize(
String text, bool addBos) → List< int> -
toString(
) → String -
A string representation of this object.
inherited
-
wasContextLimitReached(
) → bool
Operators
-
operator ==(
Object other) → bool -
The equality operator.
inherited
Static Properties
- lib → llama_cpp
-
no setter
- libraryPath ↔ String?
-
getter/setter pair