ReplaySession class

Wraps real on-device inference calls with deterministic record/replay.

A ReplaySession is runtime-agnostic: it doesn't care whether the model is flutter_gemma over a platform channel, llama_cpp_dart over FFI, or a custom engine. You hand it the request (for fingerprinting) and a thunk that performs the real inference; the session decides whether to call that thunk or serve a previously recorded result.

Typical usage in a test:

final session = ReplaySession.open(
  name: 'chat',
  store: CassetteStore('test/cassettes'),
  // CI runs with replay; record on a device with LLM_REPLAY_MODE=record.
  mode: replayModeFromString(
    const String.fromEnvironment('LLM_REPLAY_MODE'),
    fallback: ReplayMode.replay,
  ),
);

final answer = await session.run(
  request: {'prompt': prompt, 'model': 'gemma-2b', 'temperature': 0},
  live: () => gemma.getResponse(prompt),
);

session.flush(); // persist any newly recorded interactions

Constructors

ReplaySession({required Cassette cassette, required ReplayMode mode, CassetteStore? store, bool captureTiming = false})
ReplaySession.open({required String name, required ReplayMode mode, CassetteStore? store, bool captureTiming = false})
Opens a session, loading an existing cassette named name from store if one is present (otherwise starting empty).
factory

Properties

captureTiming → bool
Whether to record inter-chunk timing for streaming responses. Off by default so re-recording an unchanged interaction yields a byte-identical cassette file — keeping git diffs meaningful. Turn on for demos that want to replay the model's real typing cadence.
final
cassette → Cassette
final
hashCode → int
The hash code for this object.
no setterinherited
isDirty → bool
Whether there are unsaved recordings.
no setter
mode → ReplayMode
final
runtimeType → Type
A representation of the runtime type of the object.
no setterinherited
store → CassetteStore?
Where cassettes are persisted. null keeps everything in memory.
final

Methods

flush() → void
Persists the cassette to store if there are unsaved recordings. No-op when there is no store or nothing changed.
noSuchMethod(Invocation invocation) → dynamic
Invoked when a nonexistent method or property is accessed.
inherited
run({required Map<String, Object?> request, required Future<String> live()}) → Future<String>
Runs a one-shot inference with record/replay applied.
runStream({required Map<String, Object?> request, required Stream<String> live()}) → Stream<String>
Runs a streaming inference with record/replay applied.
toString() → String
A string representation of this object.
inherited

Operators

operator ==(Object other) → bool
The equality operator.
inherited