generateChatResponse method

  1. @nonVirtual
Future<ModelResponse> generateChatResponse()

Runs one generation and returns its answer: text or the tool call(s). When the model writes text before a call, only the call is returned; generateChatResponseAsync streams the text as well.

In a plain chat, fails with StateError when the turn's text completes and the session's InferenceModelSession.startTurn reports TurnEnd.closed. Any partial text is committed to fullHistory first. If the text itself fails, the failed-turn take-back rule below applies. An engine whose EngineTurn.end is null cannot expose the close distinction to core; structured chats report it as IncompleteStructuredOutputException as described below.

A turn the engine fails commits nothing: the messages staged for it, and any tool call it had committed, are taken back out of fullHistory before the error reaches the caller (for an input too long for the context window the error is a ContextLimitException). A stop is not a failure and keeps what was generated; a turn stopped before any text is taken back as well. Turns an engine runs through ChatTurnHooks.responses keep their own history.

In a chat with a responseFormat the reply is text, and a reply that fails the format's check is committed to fullHistory, then thrown: a StructuredOutputException (IncompleteStructuredOutputException or NonConformingStructuredOutputException). Such a turn always runs core's path; ChatTurnHooks.responses is not called for it.

Core implements this once for every engine (@nonVirtual); an engine changes a chat's turns through ChatTurnHooks.

Implementation

@nonVirtual
Future<ModelResponse> generateChatResponse() async {
  if (responseFormat != null) {
    return _collectResponses(_streamStructuredResponse());
  }
  final hooked = _hooks.responses(_engine(session), tokenBuffer: tokenBuffer);
  if (hooked != null) return _collectResponses(hooked);
  return _generateCoreResponse();
}