generationMutex property

Mutex generationMutex

Shared across all sessions of the owning model — serializes generation (concurrent contexts, serialized inference). Acquired for the whole duration of a getResponse(Async) call.

Implementation

final Mutex generationMutex;