loadModel method

Future<void> loadModel(ModelSpec spec, { String? localPath, String? localMmprojPath, String? localDraftPath, LlamaLoadProgress? onProgress, ModelMemoryEstimate? estimateOverride, int? sequenceSlots, })

Loads spec, choosing a context size that fits the current memory budget (clamped to spec.contextSize, the policy ceiling, and the model's trained context — all scaled by sequenceSlots).

On native, artifacts are downloaded into the cache directory unless localPath (and friends) point at existing files; the GGUF header is then read to build the memory estimate. estimateOverride skips header-based estimation (tests, pre-computed estimates). Replaces any previously loaded model.

sequenceSlots overrides the constructor default for this load — e.g. 1 to keep speculative decoding (a draft model) usable, since multi-sequence sessions drop the drafter.

Implementation

Future<void> loadModel(
  ModelSpec spec, {
  String? localPath,
  String? localMmprojPath,
  String? localDraftPath,
  LlamaLoadProgress? onProgress,
  ModelMemoryEstimate? estimateOverride,
  int? sequenceSlots,
}) => _tasks.run(() async {
  _checkNotDisposed();
  await _unloadLocked();
  _requestedSlots = math.max(sequenceSlots ?? _defaultSlots, 1);
  final staged = await stageModelArtifacts(
    spec,
    cacheDirectory: _cacheDirectory,
    localPath: localPath,
    localMmprojPath: localMmprojPath,
    localDraftPath: localDraftPath,
    downloader: _downloader,
    onProgress: onProgress,
  );
  _spec = spec;
  _staged = staged;
  _estimate = estimateOverride ?? staged.estimate;
  final initial = await _initialContextTokens(spec);
  final session = await _loadSession(spec, initial);
  _session = session;
  _contextTokens = initial;
  _resetResidency(session.capabilities.maxSequences);
  _logger.logInformation(
    'Model "${spec.id}" loaded: $initial context tokens across '
    '$_slots sequence slot${_slots == 1 ? '' : 's'}.',
  );
  _emit(ModelLoadedEvent(contextTokens: initial));
  _startPolling();
});