Ollama class

Wrapper around Ollama Completions API.

Ollama allows you to run open-source large language models, such as Llama 3 or LLaVA, locally.

For a complete list of supported models and model variants, see the Ollama model library.

Example:

final llm = Ollama(
  defaultOption: const OllamaOptions(
    model: 'llama3.2',
    temperature: 1,
  ),
);
final prompt = PromptValue.string('Hello world!');
final result = await openai.invoke(prompt);

Ollama API docs

Ollama base URL

By default, Ollama uses 'http://localhost:11434/api' as base URL (default Ollama API URL). But if you are running Ollama on a different one, you can override it using the baseUrl parameter.

Call options

You can configure the parameters that will be used when calling the completions API in several ways:

Default options:

Use the defaultOptions parameter to set the default options. These options will be used unless you override them when generating completions.

final llm = Ollama(
  defaultOptions: const OllamaOptions(
    model: 'llama3.2',
    temperature: 0,
    format: 'json',
  ),
);
final prompt = PromptValue.string('Hello world!');
final result = await openai.invoke(prompt);

Call options:

You can override the default options when invoking the model:

final res = await llm.invoke(
  prompt,
  options: const OllamaOptions(seed: 9999),
);

Bind:

You can also change the options in a Runnable pipeline using the bind method.

In this example, we are using two totally different models for each question:

final llm = Ollama();
const outputParser = StringOutputParser();
final prompt1 = PromptTemplate.fromTemplate('How are you {name}?');
final prompt2 = PromptTemplate.fromTemplate('How old are you {name}?');
final chain = Runnable.fromMap({
  'q1': prompt1 | llm.bind(const OllamaOptions(model: 'llama3.2')) | outputParser,
  'q2': prompt2| llm.bind(const OllamaOptions(model: 'mistral')) | outputParser,
});
final res = await chain.invoke({'name': 'David'});

Setup

Download and install Ollama
Fetch a model via ollama pull <model family>

e.g., for Llama-7b: ollama pull llama3.2

Advance

Custom HTTP client

You can always provide your own implementation of http.Client for further customization:

final client = Ollama(
  client: MyHttpClient(),
);

Using a proxy

HTTP proxy

You can use your own HTTP proxy by overriding the baseUrl and providing your required headers:

final client = Ollama(
  baseUrl: 'https://my-proxy.com',
  headers: {'x-my-proxy-header': 'value'},
  queryParams: {'x-my-proxy-query-param': 'value'},
);

If you need further customization, you can always provide your own http.Client.

SOCKS5 proxy

To use a SOCKS5 proxy, you can use the socks5_proxy package and a custom http.Client.

Constructors

Ollama({String baseUrl = 'http://localhost:11434/api', Map<String, String>? headers, Map<String, dynamic>? queryParams, Client? client, OllamaOptions defaultOptions = const OllamaOptions(model: defaultModel), String encoding = 'cl100k_base'}): Create a new Ollama instance.

Properties

defaultOptions → OllamaOptions: The default options to use when invoking the Runnable.
finalinherited
encoding ↔ String: The encoding to use by tiktoken when tokenize is called.
getter/setter pair
hashCode → int: The hash code for this object.
no setterinherited
modelType → String: Return type of language model.
no setter
runtimeType → Type: A representation of the runtime type of the object.
no setterinherited

Methods

batch(List<PromptValue> inputs, {List<OllamaOptions>? options}) → Future<List<LLMResult>>: Batches the invocation of the Runnable on the given inputs.
inherited
bind(OllamaOptions options) → RunnableBinding<PromptValue, OllamaOptions, LLMResult>: Binds the Runnable to the given options.
inherited
call(String prompt, {OllamaOptions? options}) → Future<String>: Runs the LLM on the given String prompt and returns a String with the generated text.
inherited
close() → void: Cleans up any resources associated with it the Runnable.
countTokens(PromptValue promptValue, {OllamaOptions? options}) → Future<int>: Returns the number of tokens resulting from tokenize the given prompt.
inherited
getCompatibleOptions(RunnableOptions? options) → OllamaOptions?: Returns the given options if they are compatible with the Runnable, otherwise returns null.
inherited
invoke(PromptValue input, {OllamaOptions? options}) → Future<LLMResult>: Invokes the Runnable on the given input.
noSuchMethod(Invocation invocation) → dynamic: Invoked when a nonexistent method or property is accessed.
inherited
pipe<NewRunOutput extends Object?, NewCallOptions extends RunnableOptions>(Runnable<LLMResult, NewCallOptions, NewRunOutput> next) → RunnableSequence<PromptValue, NewRunOutput>: Pipes the output of this Runnable into another Runnable using a RunnableSequence.
inherited
stream(PromptValue input, {OllamaOptions? options}) → Stream<LLMResult>: Streams the output of invoking the Runnable on the given input.
streamFromInputStream(Stream<PromptValue> inputStream, {OllamaOptions? options}) → Stream<LLMResult>: Streams the output of invoking the Runnable on the given inputStream.
inherited
tokenize(PromptValue promptValue, {OllamaOptions? options}) → Future<List<int>>: Tokenizes the given prompt using tiktoken.
toString() → String: A string representation of this object.
inherited
withFallbacks(List<Runnable<PromptValue, RunnableOptions, LLMResult>> fallbacks) → RunnableWithFallback<PromptValue, LLMResult>: Adds fallback runnables to be invoked if the primary runnable fails.
inherited
withRetry({int maxRetries = 3, FutureOr<bool> retryIf(Object e)?, List<Duration?>? delayDurations, bool addJitter = false}) → RunnableRetry<PromptValue, LLMResult>: Adds retry logic to an existing runnable.
inherited

Operators

operator ==(Object other) → bool: The equality operator.
inherited

Constants

defaultModel → const String: The default model to use unless another is specified.