Skip to main content

gemini_live 2026.11.19 copy "gemini_live: ^2026.11.19" to clipboard
gemini_live: ^2026.11.19 copied to clipboard

A Flutter package for the Gemini Live API, enabling real-time, multimodal conversations with Google's Gemini models.

2026.11.19 #

Added & Fixed (Upstream Parity with js-genai 3.6.1 & python-genai 3.6.1) #

  • Gemini 4 Neon Model Identifier (InteractionModels.gemini4Neon):
    • Added InteractionModels.gemini4Neon ('gemini-4-neon') for frontier-level intelligence with fast end-to-end task completion across complex software engineering, analytical, and science workflows.
  • Public File Processing Metadata on Part & FunctionResponsePart (DocumentMetadata, SpreadsheetMetadata, AudioMetadata, VideoMetadata.contentLayers):
    • Added DocumentMetadata (DocumentMetadataContentLayer: PAGE_IMAGE, PAGE_TEXT), SpreadsheetMetadata (SpreadsheetMetadataContentLayer: DATA, FORMULAS, VISUAL), AudioMetadata, and VideoMetadata.contentLayers (VideoMetadataContentLayer: FRAMES, AUDIO, TRANSCRIPT).
    • Wired documentMetadata, spreadsheetMetadata, and audioMetadata on Part, and mediaResolution, videoMetadata, documentMetadata, spreadsheetMetadata, and audioMetadata on FunctionResponsePart (including fromBytes and fromUri).
  • Live Interactions Auth Tokens, Wire Events & Deferred MCP Tool Loading:
    • Added AuthTokensService.createForInteraction() (POST /{api_version}/interactions/{id}/auth_tokens) and AuthTokensService.deleteForInteraction() (DELETE /{api_version}/interactions/{id}/auth_tokens/{token}), along with InteractionAuthToken and InteractionAuthTokenCreateParams.
    • Added Live Interactions & World Simulation event models (CreateInteractionEvent, ConnectInteractionEvent, CompleteInteractionEvent, CancelInteractionEvent, DisconnectEvent, LiveInteractionCreatedEvent, InteractionConnectedEvent, LiveInteractionCompletedEvent, InteractionCancelledEvent, WorldPromptContent, WorldReferenceImage, WorldSetupEvent, UserActionType, UserActionKey, UserActionInput).
    • Added McpServer.deferLoading (defer_loading / deferLoading) for deferred MCP tool loading.
  • SDK Client Telemetry Alignment (3.6.1):
    • Updated google-genai-sdk telemetry headers across ApiClient, LiveService, and LiveMusicService to 3.6.1.

2026.11.16 #

Added & Fixed (Upstream Parity with js-genai & python-genai) #

  • Interactions Static Media Processing Options (StaticMediaProcessing, AudioStaticMediaProcessing, DocumentStaticMediaProcessing):
    • Added StaticMediaProcessing (VideoContentLayer), AudioStaticMediaProcessing, DocumentStaticMediaProcessing, PageProcessingOptions (PageContentLayer: page_image, page_text), and SpreadsheetProcessingOptions (SpreadsheetContentLayer: visual, data, formulas) for configuring video, audio, paginated document, and spreadsheet extraction in the Interactions API.
  • Interactions Dynamic Tool Search & Deferred Function Loading (ToolSearch, InteractionFunction):
    • Added ToolSearch (ToolSearchExecution.server / client), InteractionFunction, ToolSearchCallArguments, ToolSearchCallStep, ToolSearchResultStep, ToolSearchCallDelta, and ToolSearchResultDelta, plus deferLoading and shortDescription support on FunctionDeclaration.
  • FunctionResponsePart.fromUri MIME Inference & Gemini Files API URIs:
    • Updated FunctionResponsePart.fromUri() (inferMimeType, isFilesApiUri) to automatically infer MIME types from file extensions when mimeType is omitted, leave mimeType unset for Gemini Files API URIs (https://generativelanguage.googleapis.com/.../files/...) so the server uses uploaded file metadata, validate unknown non-Files-API URIs, and forward displayName.
  • Header Case-Variant Collapsing & SSE / JSON Stream Parsing Fixes (ApiClient, RawJsonLineBuffer):
    • Added ApiClient.collapseHeaderCaseVariants() and ApiClient.appendLibraryVersionHeaders() and updated ApiClient.buildRequestHeaders() and LiveService.connect() to merge case variants of User-Agent and x-goog-api-client without emitting duplicate headers.
    • Added ApiClient.splitSseLines() (splitting strictly on CR, LF, and CRLF while preserving U+2028, U+2029, and U+0085 inside JSON strings and handling split CRLF chunks) and RawJsonLineBuffer (ignoring { / } inside quoted JSON strings).
  • Recursive Union JSON Schema $defs / $ref Resolution (JsonSchemaTransformer.processSchema) & v1beta Model Docs:
    • Added JsonSchemaTransformer.processSchema() to expand $defs / $ref per recursion path without clobbering shared anyOf union branches while terminating cyclic back-edges as {}.
    • Documented v1beta requirement for -latest aliases (gemini-flash-latest, gemini-flash-lite-latest, gemini-pro-latest) and -preview models.

2026.11.13 #

Added & Fixed (Upstream Parity with js-genai 3.3.0 & python-genai 3.3.0) #

  • Gemini 4 Hydrogen & Full Interactions Model Catalog (InteractionModels.gemini4Hydrogen):
    • Added InteractionModels.gemini4Hydrogen ('gemini-4-hydrogen') for fast, dependable frontier execution across coding, agentic workflows, and multimodal knowledge work, along with the full upstream Interactions model constants (gemma-4-26b-a4b-it, gemma-4-31b-it, gemini-flash-latest, gemini-flash-lite-latest, gemini-pro-latest, gemini-3.7-flash, gemini-3.6-flash, gemini-3.5-flash, gemini-3.5-flash-lite, gemini-3.8-flash-tts, gemini-3.8-flash-lite-tts, gemini-3.5-transcribe, gemini-nano-banana-2.1, gemini-robotics-er-2-preview, gemini-robotics-er-1.6-preview, lyria-3.5, lyria-3-clip-preview, and lyria-3-pro-preview).
  • Default UTF-8 HTTP & Streaming SSE Response Decoding (ApiClient):
    • Added ApiClient.resolveResponseEncoding(), ApiClient.decodeResponseBody(), and ApiClient.decodeSseJsonStream() so JSON, error-envelope, and text/event-stream responses without an explicit Content-Type charset parameter decode as UTF-8 by default (instead of package:http's ISO-8859-1 / latin1 fallback) while preserving explicit server charsets and reassembling multi-byte UTF-8 sequences split across stream chunks.
  • Environment Variable API Key & Debug Flag Resolution (GEMINI_API_KEY, GOOGLE_API_KEY, GOOGLE_GENAI_DEBUG):
    • Added ApiClient.resolveDefaultApiKey() (GEMINI_API_KEY with fallback to GOOGLE_API_KEY) and optional apiKey parameter support on GoogleGenAI and ApiClient, plus ApiClient.isDebugEnabled() recognizing 'true', '1', 'yes', and 'on' in GOOGLE_GENAI_DEBUG.
  • SDK Client Telemetry Alignment (3.3.0):
    • Updated google-genai-sdk telemetry headers across ApiClient, LiveService, and LiveMusicService to 3.3.0.

2026.11.12 #

Added & Fixed (Upstream Parity with js-genai 3.2.0 & python-genai 3.2.0) #

  • Live API Proactive Video (ProactivityConfig.proactiveVideo):
    • Added proactiveVideo (proactive_video / proactiveVideo) to ProactivityConfig for triggering model generation on incoming Live video frames when proactiveAudio is enabled, and ensured LiveClientSetup.toJson() serializes nested proactivity maps directly.
  • Extra-High Thinking Effort (ThinkingLevel.XHIGH):
    • Added ThinkingLevel.XHIGH ('XHIGH') and updated ThinkingConfig.fromJson() to normalize both snake_case/camelCase keys and lowercase 'xhigh' wire values across Gemini API and Interactions API payloads.
  • Decisions API (DecisionsService / genAI.decisions.create):
    • Added DecisionsService (POST /{api_version}/decisions) on GoogleGenAI.decisions, along with CreateDecisionRequest, CreateDecisionResponse, DecisionQuestion (NoulQuestion, ChoiceQuestion, ScoreQuestion), DecisionAnswer (NoulAnswer, ChoiceAnswer, ScoreAnswer), NoulCriteria, DecisionPromptFeedback, and DecisionBlockReason.
  • Interactions Model/Agent Identifiers, EnvironmentMode, & SDK Telemetry (3.2.0):
    • Added InteractionModels (gemini-4-carbon, gemini-omni-1.2-flash, etc.), InteractionAgents (antigravity-preview-09-2026, etc.), and EnvironmentMode (on_demand, self_hosted).
    • Updated google-genai-sdk telemetry headers to 3.2.0, refreshed Dartdoc descriptions for FinishReason, MediaProcessing, MediaResolution, and Modality, and updated Gemini API documentation links from ai.google.dev to aistudio.google.com.

2026.11.8 #

Added & Fixed (Upstream Parity with js-genai & python-genai) #

  • Expanded FinishReason & ReinforcementTuningThinkingLevel Enums:
    • Added MISSING_THOUGHT_SIGNATURE, ESCALATION, PUP_LIMITED_DISABLED, and MODEL_ARMOR to FinishReason.
    • Added ReinforcementTuningThinkingLevel (REINFORCEMENT_TUNING_THINKING_LEVEL_UNSPECIFIED, MINIMAL, LOW, MEDIUM, HIGH).
  • Typed 4XX/5XX HTTP API Error Responses (ApiError, GoogleGenAiError, ApiClientError, ApiServerError):
    • Added ApiError (code, message, status, details), GoogleGenAiError, ApiClientError (thrown on 4XX HTTP responses), and ApiServerError (thrown on 5XX HTTP responses) in ApiClient, surfacing structured backend error messages while preserving API Error: <statusCode> <body> string formatting.
  • Multimodal CodeExecutionResult & Office/OpenDocument MIME Types (DocumentMimeTypes):
    • Extended CodeExecutionResult with multimodal content (List<Content>?), result union getter/constructor parameter (String or List<Content>), isError, signature, and call_id JSON alias support.
    • Added DocumentMimeTypes constants and DocumentMimeTypes.isSupported() covering PDF, CSV, Microsoft Office (.pptx, .ppt, .docx, .doc, .dotx, .xlsx, .xls), and OpenDocument (.odp, .odt, .ods) MIME types.
    • Extended UsageMetadata.fromJson() to normalize snake_case traffic_type and service_tier fields.
  • Curated History AFC Exchange Trimming (GeminiLiveTranscriptExporter.extractCuratedHistory):
    • Added GeminiLiveTranscriptExporter.extractCuratedHistory() and validateContent(), dropping the entire automatic function calling (AFC) exchange from curated chat history when the final model turn is invalid while preserving unanswered spent-budget function calls.

2026.11.6 #

Added & Fixed (Upstream Parity with js-genai 3.0.0 & python-genai 3.0.0) #

  • Voice-Driven Editing & Transcription (AudioTranscriptionConfig.sourceText, SmartTranscriptionMode):
    • Added sourceText, wordTimestamp, and diarization to AudioTranscriptionConfig with snake_case (source_text, word_timestamp) and camelCase (sourceText, wordTimestamp) JSON serialization support.
    • Added SmartTranscriptionMode (type: 'smart', sourceText) for voice-driven text editing and replacement in smart transcription mode.
  • Finish Reason Enum & MALFORMED_RESPONSE (FinishReason):
    • Added FinishReason enum including CONTINUATION and MALFORMED_RESPONSE aligned with upstream js-genai 3.0.0 and python-genai 3.0.0.
  • Pre-Setup WebSocket Close & Error Rejection (LiveService.connect, LiveMusicService.connect):
    • Fixed LiveService.connect() and LiveMusicService.connect() to immediately reject with a StateError (Live connection closed before setup completed (code <code>): <reason>) when the WebSocket closes before setupComplete arrives instead of waiting for setup timeout, while still invoking user onError and onClose callbacks and closing the underlying socket on setup failure.
  • API Version Environment Variable & {api_version} Path Placeholder (ApiClient):
    • Added ApiClient.resolveDefaultApiVersion(), ApiClient.setDefaultGoogleAiApiVersion(), and GOOGLE_GENAI_API_VERSION environment variable support (beta -> v1beta, or verbatim version string when no explicit apiVersion is passed in code).
    • Added ApiClient.resolveApiVersionInPath() and {api_version} path placeholder substitution in ApiClient for versioned upload and resource paths.
    • Verified AuthTokensService fieldMask generation keeps repeated/array fields (tools, safetySettings, generationConfig.response_modalities) bare without array indices.
  • SDK Client Header Version & Documentation Links (3.0.0):
    • Updated default User-Agent and x-goog-api-client SDK version headers across LiveService, LiveMusicService, and ApiClient to google-genai-sdk/3.0.0.
    • Updated codegen_instructions.md reference links from ai.google.dev to aistudio.google.com/docs.

2026.11.1 #

Added (Upstream Parity with js-genai 2.32.0 & python-genai 2.34.0) #

  • Automatic Continuation Deadline, Timeout & Multi-Hop Usage Aggregation (UsageMetadata.merge, CallDeadline, ContinuationTimeoutError):
    • Added UsageMetadata.merge(UsageMetadata other) to sum cumulative token fields (promptTokenCount, responseTokenCount, cachedContentTokenCount, toolUsePromptTokenCount, thoughtsTokenCount, totalTokenCount) and per-modality ModalityTokenCount lists across continuation hops while preserving initialPromptTokenCount from the first hop.
    • Added CallDeadline (timeoutMs, bounded, remainingMs(), expired(), hopTimeoutMs(), check()) and ContinuationTimeoutError to bound wall-clock execution across multi-hop continuation turns.
  • Speech Transcription Model Identifier (LiveModels.gemini35Transcribe):
    • Added LiveModels.gemini35Transcribe ('gemini-3.5-transcribe') aligned with upstream speech transcription model support.
  • SDK Client Header Version Alignment (2.34.0):
    • Updated default User-Agent and x-goog-api-client SDK version headers across LiveService, LiveMusicService, and ApiClient to google-genai-sdk/2.34.0.

2026.10.29 #

Added (Upstream Parity with js-genai & python-genai) #

  • Video Understanding in Tool Responses (FunctionResponsePart.mediaProcessing):
    • Added mediaProcessing (MediaProcessing.STATIC / MediaProcessing.AGENTIC) to FunctionResponsePart and its .fromBytes() / .fromUri() factories.
    • Added snake_case (media_processing) and camelCase (mediaProcessing) deserialization support across FunctionResponsePart.fromJson() and Part.fromJson().
  • Long Decode Continuation & Interaction Status (REQUIRES_CONTINUATION, continuationToken, continuationIds):
    • Added InteractionStatus.REQUIRES_CONTINUATION to InteractionStatus with case-insensitive deserialization (requires_continuation / REQUIRES_CONTINUATION).
    • Added continuationToken, continuationIds, and requiresContinuation to LiveServerContent and GeminiLiveSessionController.
    • Added ContinuationInterruptedError to preserve partial progress, continuationToken, interactionId, and continuationIds across multi-hop continuation boundaries.
  • Interaction Usage Metadata Aliases:
    • Extended UsageMetadata.fromJson() and ModalityTokenCount.fromJson() to support initial_input_tokens (initialInputTokens), total_input_tokens, total_output_tokens, total_cached_tokens, total_tool_use_tokens, total_thought_tokens, total_tokens, and *_tokens_by_modality (tokens and lowercase modality names).
  • Per-Request Header Isolation, GenAICredentials, X-Server-Timeout, & SDK Headers:
    • Added GenAICredentials for dynamic bearer token (Authorization: Bearer ...) and quota project (x-goog-user-project) resolution across GoogleGenAI, LiveService, and ApiClient.
    • Added ApiClient.populateServerTimeoutHeader() (X-Server-Timeout in whole seconds rounded up from HttpOptions.timeout) and ensured per-request headers never mutate client default headers.
    • Aligned ApiClient.buildRequestHeaders() to include default User-Agent and x-goog-api-client SDK headers (google-genai-sdk/2.33.0 dart/<version>) while preserving caller-supplied overrides.
    • Added LiveService.supportsHeaders matching WebSocketFactory.supportsHeaders in @google/genai, and supported HttpOptions (baseUrl, apiVersion, headers, timeout, extraBody, retryOptions) on LiveConnectConfig and LiveConnectParameters.

Added (Pre-Built Widgets, Foldable/Adaptive UI & Developer Utilities) #

  • Foldable & Adaptive Responsive Layouts (GeminiLiveViewportInfo, GeminiLiveAdaptiveSplitView):
    • Added GeminiLiveViewportInfo (GeminiLiveSizeClass, GeminiLiveFoldPosture) detecting DisplayFeature foldable hinges (book, tabletop), ultra-narrow screens (< 360px), and tablet/desktop breakpoints.
    • Added GeminiLiveAdaptiveSplitView and adapted GeminiLiveTranslateSplitView, GeminiLiveChatView, GeminiLiveControlBar, GeminiLiveMusicControlDeck, GeminiLiveLogOverlay, and all 4 bottom sheets (GeminiLiveVoiceSelectorSheet, GeminiLiveVoiceStudioSheet, GeminiLiveSessionInfoSheet, GeminiLiveContextConfigSheet) for foldable hinges and compact viewports.
  • Universal In-App WebSocket & Diagnostic Log Overlay (GeminiLiveLogOverlay, GeminiLiveLogConsole):
    • Added GeminiLiveLogController, GeminiLiveLogOverlay, GeminiLiveLogConsole, and .withGeminiLiveLogOverlay() widget extension to attach a draggable floating log button and filterable diagnostic console to any widget.
  • Additional Widgets & Developer Utilities:
    • Added GeminiLiveMusicControlDeck, GeminiLiveReconnectBanner, GeminiLiveVoiceStudioSheet, and GeminiLiveSessionInfoSheet / GeminiLiveSessionInfoBadge.
    • Added GeminiLiveFunctionTool & GeminiLiveToolRegistry (inline tool declaration & auto-execution), GeminiLiveTranscriptExporter (HistoryConfig, Markdown, JSON, WebVTT, SRT), GeminiLiveFrameRateLimiter (camera frame throttling), and GeminiLiveAudioUtils (PCM16 resampling, stereo/mono conversion, silence detection, WAV header parsing).

Refactored & Example App #

  • Removed unused web_socket_client dependency and standardized WebSocket handshake via WebSocketChannel.ready.
  • Added 4-language (en, ko, ja, zh) runtime localization, global log overlay integration, and MG16XU hardware mixer strips across all 13 example screens.
  • Applied Effective Dart documentation guidelines across public API classes and methods.

2026.10.23 #

Added (Upstream Parity with js-genai 2.31.0 & python-genai 2.33.0) #

  • LiveConnectConfig & Session labels Support:
    • Added LiveConnectConfig combining GenerationConfig parameters and all Live session setup fields (systemInstruction, tools, realtimeInputConfig, sessionResumption, contextWindowCompression, inputAudioTranscription, outputAudioTranscription, proactivity, explicitVadSignal, avatarConfig, safetySettings, historyConfig, translationConfig, generationConfig, labels).
    • Supported passing config as either a LiveConnectConfig, GenerationConfig, or Map<String, dynamic> in both LiveConnectParameters and LiveConnectConstraints (AuthTokensService), mapping labels directly to setup.labels (bidiGenerateContentSetup.labels).
  • CallableTool & CallableToolConfig Support:
    • Added CallableTool and CallableToolConfig with automatic asynchronous tool declaration resolution via LiveService.buildSetupMessageAsync() and LiveService.connect().
  • Async Session Stream & Turn Iteration (LiveSession.receive() & sessionId):
    • Added LiveSession.sessionId (alias for setupComplete?.sessionId), LiveSession.messages, and turn-scoped LiveSession.receive() (yielding server messages until serverContent.turnComplete == true, matching AsyncSession.session_id and AsyncSession.receive() in python-genai).
    • Added LiveMusicSession.messages and LiveMusicSession.receive() on LiveMusicSession.
  • Realtime Input & Live Music Parameter Classes:
    • Added LiveSendRealtimeInputParameters.media, LiveMusicSetWeightedPromptsParameters, and LiveMusicSetConfigParameters, and widened LiveMusicSession.setWeightedPrompts() and LiveMusicSession.setMusicGenerationConfig() to accept typed parameter objects or maps.
    • Updated SlidingWindow.targetTokens and ContextWindowCompressionConfig.triggerTokens to accept both int and String token counts.
    • Added GeminiLiveVoice.id and GeminiLiveVoice.characterDescription convenience getters.
  • Default Coding System Prompt (codegen_instructions.md) & tContent Alignment:
    • Ported upstream codegen_instructions.md (# Gemini Live API Coding Guidelines) to the repository root and synchronized AGENTS.md, GEMINI.md, CLAUDE.md, and CODEX.md.
    • Aligned GeminiLiveChatView.defaultSystemInstruction with upstream tContent (role: 'user') and unified systemInstruction union parsing.
    • Bumped internal SDK client version header to 2.33.0 across LiveService and LiveMusicService.

2026.10.22 #

Added (Upstream Parity with js-genai 2.30.0+ & python-genai 2.32.0) #

  • In-Session Context Updates (sendContextUpdate):
    • Added LiveClientContextUpdate, LiveClientContextUpdateTools, and LiveClientMessage.contextUpdate to dynamically update systemInstruction and tools mid-session in order with session inputs without reconnecting.
    • Added LiveSession.sendContextUpdate() and GeminiLiveSessionController.sendContextUpdate(), supporting tools: const <Tool>[] to clear all previously registered tools.
  • Voice Design Base Voice (PromptedVoice.baseVoice):
    • Added baseVoice to PromptedVoice and CreateVoiceRequest.prompted() to adapt an existing voice persona with a natural-language style prompt.
  • Multimodal Tool Response & Binary Helpers:
    • Added FunctionResponsePart.fromBytes(), FunctionResponsePart.fromUri(), FunctionResponseBlob.fromBytes(), VoiceAudioData.fromBytes(), and Blob.fromBytes() (with .bytes getters).
    • Automatic recursive base64 encoding for raw Uint8List values inside LiveClientToolResponse.toJson().
    • Added snake_case (inline_data, file_data, mime_type, etc.) and raw byte array (List<int>) deserialization support to Part.fromJson() and Blob.fromJson().
  • Full Runtime & Union Parameter Parity (tModel, tContents, tLiveSpeechConfig):
    • Added LiveService.normalizeModelName() matching upstream tModel validation (rejecting empty model strings or ../?/&, and preserving models/ and tunedModels/ prefixes) across LiveService and LiveMusicService.
    • Added top-level generation parameters (responseModalities, temperature, topP, topK, maxOutputTokens, mediaResolution, seed, speechConfig accepting SpeechConfig or voice name String, thinkingConfig, and enableAffectiveDialog) directly on LiveConnectParameters.
    • Enforced upstream multiSpeakerVoiceConfig rejection (UnsupportedError) in LiveService.buildSetupMessage().
    • Widened LiveSession.sendClientContent() (turns union accepting String, Part, Content, Map, or List with automatic Part accumulation into a single user Content turn, mixed Content/Part rejection, and unwrapped functionCall/functionResponse part rejection).
    • Widened LiveSession.sendRealtimeInput() (media, audio, video accepting Blob or Map, and activityStart/activityEnd accepting true, {}, or ActivityStart/ActivityEnd) and LiveSession.sendToolResponse() (accepting single FunctionResponse, Map, or List).
    • Added enterprise tool validation (retrieval, enterpriseWebSearch, exaAiSearch, parallelAiSearch) and automatic mcp_used/unknown client header injection when Tool.mcpServers is present.
    • Bumped internal SDK client version header to 2.32.0 across LiveService and LiveMusicService.

Added (New Pre-Built Widgets & Controller Enhancements) #

  • GeminiLiveThinkingIndicator:
    • System-styled status pill and collapsible reasoning drawer for gemini-3.8-live-extended-thinking (Part(thought: true)), with .fromController() factory and built-in integration in GeminiLiveChatView (showThinkingIndicator, thinkingIndicatorBuilder).
  • GeminiLiveContextConfigSheet, GeminiLiveContextBadge, & GeminiLiveContextPreset:
    • Interactive Material 3 bottom sheet (.show()) and summary badge for switching persona presets, editing systemInstruction, and toggling or clearing session tools mid-call via sendContextUpdate.
  • GeminiLiveSessionController Enhancements:
    • Added Extended Thinking state tracking (isThinking, latestThought, thoughts).
    • Added live context state tracking (activeSystemInstruction, activeTools).

Documentation & Examples #

  • Updated live_api, math_tutor, and function_calling example demos to showcase sendContextUpdate and upstream union parameters.
  • Applied Effective Dart documentation guidelines across public Live API classes and widgets.
  • Updated and synchronized all AI Agent Skills (skills/, .agents/, .claude/, .codex/, .gemini/, .hermes/, .pi/).

2026.10.6 #

Added (Upstream Parity with js-genai 2.30.0 & python-genai 2.31.0) #

  • Word-Level Transcription & Timestamps:
    • Added WordInfo (word, startOffset, endOffset) and extended Transcription with words, startOffset, and endOffset fields.
    • Added interimTranscriptTimestampEnabled to RealtimeInputConfig.
  • Token Usage Metadata & Accounting:
    • Added initialPromptTokenCount to UsageMetadata and exposed initialPromptTokens and toolUsePromptTokens on GeminiTokenUsageTracker.
    • Added wire-format fallback deserialization for candidatesTokenCount / candidatesTokensDetails on UsageMetadata and type on VoiceActivity.
  • Live Grounding & Remote MCP Tool Models:
    • Added SearchTypes, WebSearch, and ImageSearch models and GoogleSearch.searchTypes.
    • Added AuthConfig (apiKey, authType) and GoogleMaps.authConfig / GoogleMaps.enableWidget.
    • Added typed McpServer and StreamableHttpTransport models along with Tool.fromMcpServers and Tool.typedMcpServers.
  • Full Live Setup Constraints on Ephemeral Tokens (LiveConnectConstraints):
    • Expanded LiveConnectConstraints and AuthTokensService.create() to support locking systemInstruction, tools, realtimeInputConfig, sessionResumption, contextWindowCompression, inputAudioTranscription, outputAudioTranscription, proactivity, avatarConfig, safetySettings, historyConfig, and translationConfig inside bidiGenerateContentSetup.
  • Bumped internal SDK client version header to 2.30.0 across LiveService and LiveMusicService.

Added (Widgets, Utilities & Controller Enhancements) #

  • New Pre-Built Widgets:
    • GeminiLiveToolActivityChip: Animated status pill that automatically appears while the model executes LiveServerToolCall functions, with custom labelBuilder and leadingIcon support.
    • GeminiLiveAudioRouteButton: Hardware audio route switcher (speaker, earpiece, bluetooth, wiredHeadset) with 2-route quick toggle and 3+ route bottom sheet selector.
    • GeminiLiveTranslateSplitView: Dual-pane real-time speech-to-speech interpreter view for gemini-3.5-live-translate-preview, including 180-degree faceToFaceMode rotation for across-the-table conversations.
  • New Audio & Reconnection Utilities:
    • GeminiLivePcmChunker: Streams irregular microphone buffers into fixed-duration (e.g., 100ms / 3,200-byte) 16-bit aligned PCM frames.
    • GeminiLiveAudioBuffer: Accumulates 16-bit PCM chunks with optional maxDuration rolling cap, duration calculation, and instant WAV export (toWavBytes()).
    • GeminiLiveReconnectPolicy: Exponential backoff with random jitter and retryable WebSocket close code evaluation (1006, 1011, 1012, 1013).
  • GeminiLiveSessionController Enhancements:
    • Automatic session resumption handle (resumptionHandle) tracking and injection on reconnect.
    • Optional automatic reconnection (reconnectPolicy) on GoAway notices and retryable disconnects.
    • Real-time tool execution state tracking (activeToolCalls, isExecutingTool) for LiveServerToolCall and LiveServerToolCallCancellation.
    • Optional model turn PCM recording (recordModelAudio) and WAV export (exportLastModelTurnWav()), plus sendAudioStreamEnd().

Fixed & Refactored #

  • Fixed GeminiLiveMicButton and CupertinoGeminiLiveMicButton ignoring custom activeIcon and inactiveIcon widget parameters.
  • Fixed GeminiLiveChatView.didUpdateWidget to properly rebind listeners when swapping controller, and guarded empty LiveTranscriptItem.text indexing.
  • Added auto-dismiss timer (displayDuration, onDismissed) and custom icon widget support to GeminiLiveBargeInBanner.
  • Fixed GeminiLiveUsageBadge custom backgroundColor and borderColor override precedence.
  • Fixed GeminiLiveSessionController.dispose() so externally injected GeminiTokenUsageTracker instances are not prematurely disposed.
  • Added GeminiLiveLogger.removeListener() and clearListeners() to prevent listener leaks.
  • Modularized lib/src/model/models.dart into domain-specific part files under lib/src/model/models/.
  • Refactored all example app demos into a layered MVVM architecture (data/services, ui/view_models, ui/widgets).

2026.10.5 #

Added #

  • LiveModels.gemini35LiveTranslatePreview (gemini-3.5-live-translate-preview) for speech-to-speech translation.
  • SpeechAnnotation model and serialization for voice synthesis metadata (js-genai 2.27.0 sync).
  • GoogleGenAI.music getter that exposes the Realtime Music service directly.
  • Type-Safe Voice Personas (GeminiLiveVoice):
    • Added enhanced enum GeminiLiveVoice covering Google's 30 official prebuilt voices (puck, charon, kore, fenrir, aoede, leda, orus, zephyr, etc.) with compile-time safety, tone styles, and pitch/gender metadata.
    • Added SpeechConfig.fromLiveVoice(GeminiLiveVoice voice, {String? languageCode}) and extended SpeechConfig.fromVoice(Object voice) to accept both enums and custom strings.
    • Added liveVoice getter across SpeechConfig, VoiceConfig, and PrebuiltVoiceConfig for typed inspection and JSON round-trip deserialization.
    • Integrated GeminiLiveVoice with GeminiLiveVoiceSelectorSheet (.showLiveVoice, onLiveVoiceSelected, .allLiveVoices 30-voice catalog) and GeminiVoiceOption.fromLiveVoice.
    • Synchronized GeminiLiveVoice and SpeechConfig.fromLiveVoice in package:gemini_live/compat/firebase_ai.dart.
  • Live/music service SDK version header bumped to js-genai 2.27.0.

Deprecated #

  • Model constants that were misfiled as Live models during the js-genai 2.27.0 sync. Upstream added them to the Interactions Model enum, not the Live API. They will be removed in a future release.
    • LiveModels.geminiOmni11Flash, LiveModels.geminiOmniFlashPreview (Gemini Omni is a video generation model).
    • LiveModels.gemini38FlashTts, LiveModels.gemini38FlashLiteTts, LiveModels.gemini31FlashTtsPreview (TTS models).
    • LiveMusicModels.lyria35, LiveMusicModels.lyria3ClipPreview, LiveMusicModels.lyria3ProPreview (not Realtime Music models; use LiveMusicModels.lyriaRealtimeExp).

Example App #

  • Live Music Studio demo for Lyria RealTime, with Prompt DJ crossfader, tag bank, and auto-reset context.
  • Pioneer-style Pro DJ Console and AI Studio Prompt DJ 4x4 rotary MIDI Box demos.
  • Web Audio API real-time PCM streamer for Flutter Web playback.
  • Removed non-realtime Lyria 3.x models from the Live Music Studio model picker.

Documentation #

  • README (EN/KR/JP/ZH): AI Agents guide, Supported Models tables, Voice Quick Start (mic + playback + platform permissions), API key security note, and fixed widget snippets that did not compile.
  • Agent Skills synchronized across Claude Code, Gemini CLI / Antigravity, Codex, and the open Agent Skills paths. Fixed snippets that did not compile, added mic/playback wiring, setup, and "Do NOT" guardrails.
  • doc/advanced_configuration.md: fixed the function calling example.

2026.10.4 #

Added (Upstream Parity with js-genai & python-genai) #

  • Live Music API (ai.live.music / LiveMusicService):
    • Full bidirectional WebSocket streaming client for Gemini Realtime Music generation (models/lyria-realtime-exp) via BidiGenerateMusic.
    • Accessible via genAI.live.music or liveService.music.
    • Added session controls: setWeightedPrompts(), setMusicGenerationConfig(), play(), pause(), stop(), resetContext().
    • Added full models and enums: Scale, MusicGenerationMode, LiveMusicPlaybackControl, WeightedPrompt, LiveMusicGenerationConfig, LiveMusicClientMessage, LiveMusicServerMessage, AudioChunk (with bytes helper getter), LiveMusicFilteredPrompt.
  • Ephemeral Authentication Tokens Service (genAI.authTokens / AuthTokensService):
    • Implemented AuthTokensService for issuing secure, short-lived client-side session tokens (POST /v1alpha/authTokens).
    • Supports liveConnectConstraints (locking model and GenerationConfig) and automated fieldMask generation with lockAdditionalFields.
    • Handles dual camelCase/snake_case JSON mapping for AuthToken responses.
  • Top-level translationConfig on LiveConnectParameters:
    • Directly pass TranslationConfig on LiveConnectParameters for seamless realtime bidirectional translation without nesting in GenerationConfig.
  • Audio Transcription languageCodes Support:
    • Removed outdated restriction prohibiting languageCodes in AudioTranscriptionConfig, aligning with upstream Gemini API support for transcription languages.

2026.10.3 #

Changed & Improved (Design & Aesthetics) #

  • Native Non-AI UI Overhaul across all Pre-built Widgets:
    • GeminiLiveCaptionBubble:
      • Replaced garish neon-colored badges with sleek, native frosted glass (enableBlur: true with BackdropFilter and ImageFilter.blur).
      • Added subtle neutral speaker chips (showSpeakerTag: true/false), elegant subtitle typography, and smooth breathing indicator dot (_PulsingDot 900ms ease-in-out).
    • GeminiLiveWaveform:
      • Replaced mechanical rectangular bars with organic capsule pills (borderRadius defaulting to barWidth / 2).
      • Added enableIdleBreathing: true with organic sinusoidal micro-motion when idle, mimicking professional native audio recording equipment.
    • GeminiLiveStatusBadge:
      • Swapped out raw color-outline boxes for refined neutral surface containers with 0.5px borders.
      • Upgraded pulse dot to a modern halo ripple ring effect (_PulseDot) and softened semantic palette.
    • GeminiLiveMicButton:
      • Replaced generic BoxShadow blur glow with crisp, expanding concentric acoustic ripple rings and neutral dark/light tactile surface layers.
    • GeminiLiveVoiceIndicator:
      • Upgraded from simple single sine to organic dual-harmonic motion with bell-curve center envelopes and pill capsules.
    • GeminiLiveUsageBadge & GeminiLiveUsageDetailsDialog:
      • Removed arcade-like neon cyan/amber accents in favor of clean, professional SaaS observability dark/light surface aesthetics.

2026.10.2 #

Added #

  • GeminiLiveSessionController (LiveSessionController):
    • All-in-one Flutter ChangeNotifier state manager for handling Live API WebSocket connections, lifecycle (disconnected, connecting, connected, error), message streams, and errors.
    • Integrated real-time transcript timeline tracking (LiveTranscriptItem) for user input, interim speech, and model output with speaker attribution and style metadata.
    • Seamless barge-in interruption handling via onInterrupted callback.
    • Built-in token usage tracking integration (TokenUsageTracker) and audio stream helpers (feedAudioPcm, sendText).
  • GeminiLiveWaveform Widget:
    • Real-time animated audio visualizer supporting continuous amplitude stream/normalized levels or raw 16-bit PCM byte streams.
    • Configurable bar count, wave style (bars vs sine), animated idle breathing, and gradient styling.
  • GeminiLiveCaptionBubble Widget:
    • Live subtitle/caption bubble widget for streaming speech-to-text transcripts with animated streaming dot indicator.
    • Displays speaker badge, emotional/style tone indicators, and auto-dismiss after inactivity.
  • GeminiLiveAudioUtils:
    • Production-grade PCM audio analysis helper: RMS amplitude, peak amplitude, decibels (dBFS), and logarithmic visual scaling for audio meters.

2026.10.1 #

Added #

  • LiveClientSetup & GenerationConfig Metadata Labels:
    • Added labels (Map<String, String>?) to LiveConnectParameters, LiveClientSetup, and GenerationConfig.
    • Enables user-defined metadata labels for tracking, attribution, and aggregator safety identifiers (e.g. safety_identifier) adhering to unified Cloud label formatting.
  • Speech Synthesis Metadata (SpeechMetadata):
    • Added SpeechMetadata data model with speaker and style fields.
    • Added Part.speechMetadata (speech_metadata) to control per-part multi-speaker allocation and vocal styles (e.g. "whispering", "excited, fast-paced") during speech synthesis and Live conversations.
  • Gemini 3.8 Flash TTS Canonical Model Identifiers:
    • Added canonical TTS model constants to LiveModels:
      • LiveModels.gemini38FlashTts ('gemini-3.8-flash-tts'): High-fidelity Gemini 3.8 Flash text-to-speech model.
      • LiveModels.gemini38FlashLiteTts ('gemini-3.8-flash-lite-tts'): Ultra-low-latency lightweight Gemini 3.8 Flash text-to-speech model.
  • Voice Resource Sample Audio:
    • Added sampleAudio (VoiceAudioData?) to VoiceResource and CreateVoiceRequest (sample_audio), providing audio payload reference for newly designed and replicated voice assets.
  • Upstream Sync with js-genai 2.25.0 & python-genai 2.26.0:
    • Bumped internal SDK client user-agent to google-genai-sdk/2.25.0.
    • Regenerated serialization code and updated unit test suites.

2026.9.20 #

Added #

  • Gemini Voices API Integration & Support:
    • Added GeminiVoicesService (accessible via genAI.voices or standalone) for managing custom and prebuilt system voice catalog resources (listVoices, createVoice, getVoice, deleteVoice).
    • Added VoiceResource model supporting custom Google-managed voices, client-managed replication keys, and prebuilt catalog voices.
    • Added CreateVoiceRequest supporting natural-language prompted voices (CreateVoiceRequest.prompted) and audio-cloned replicated voices (CreateVoiceRequest.replicated).
    • Added VoiceType (replicated, prompted, prebuilt), VoicePitch (low, medium, high), PromptedVoice, ReplicatedVoice, VoiceAudioData, ListVoicesResponse, and DeleteVoiceResponse data models.
  • VoiceConfig & SpeechConfig Enhancements:
    • Added VoiceConfig.voice string property to directly pass custom voice IDs (voice_...), replication keys (voicekey_...), or prebuilt voice names directly into speech synthesis.
    • Added VoiceConfig.promptedVoiceConfig support for natural-language designed voices.
    • Added ergonomic constructors: VoiceConfig.fromVoiceName(String name) and VoiceConfig.fromVoiceId(String id).
    • Added SpeechConfig.voice property and SpeechConfig.fromVoice(String voiceNameOrId, {String? languageCode}) convenience factory.
  • Upstream Sync & Documentation:
    • Synced with upstream js-genai and python-genai discovery doc and Voices API resource updates.
    • Updated AudioTranscriptionConfig.languageCodes documentation reflecting current upstream preference over deprecated languageHints/languageAuto.

2026.9.19 #

Added #

  • Real-time Token Usage & Cost Tracking:
    • Added GeminiTokenUsageTracker: Automatically tracks real-time prompt, candidate, thoughts, audio, video, image, and document token consumption per turn and session with model-specific cost estimation.
    • Added GeminiLiveUsageBadge: Sleek Material widget showing live token count and estimated cost, with interactive tap dialog (GeminiLiveUsageDetailsDialog) showing full multi-modal breakdowns.
  • Upstream Sync with js-genai 2.23.0 & python-genai 2.24.0:
    • Added TrafficType.ON_DEMAND_OFFPEAK ('ON_DEMAND_OFFPEAK') enum value.
    • Synced Live SDK request headers and user-agent string to google-genai-sdk/2.23.0.
    • Regenerated code generation artifacts (models.g.dart).
  • Example App Enhancements:
    • Integrated GeminiLiveUsageBadge across live demo pages.
    • Added Live Media Subtitle & Live Smart Notetaker workflows.
  • Removed legacy experimental designation across package metadata and documentation to reflect the official Stable release of the Gemini Live API.

2026.9.16 #

Added #

  • Gemini 3.8 Live & Extended Thinking Support:
    • Added LiveModels class providing canonical model identifiers:
      • LiveModels.gemini38Live ('gemini-3.8-live'): Default stable Live API model for low-latency voice and multimodal dialogue without reasoning-induced delays.
      • LiveModels.gemini38LiveExtendedThinking ('gemini-3.8-live-extended-thinking'): High-reasoning Live model for voice interactions requiring deep background reasoning.
      • LiveModels.gemini31FlashLivePreview ('gemini-3.1-flash-live-preview'): Previous preview model.
      • LiveModels.gemini25FlashNativeAudioPreview ('gemini-2.5-flash-native-audio-preview-12-2025').
  • Example App Updates:
    • Set default model to gemini-3.8-live in ApiKeyStore and LiveAPIDefaults.
    • Added gemini-3.8-live and gemini-3.8-live-extended-thinking to model selection lists across all demo pages.
    • Updated connection logging to display active Gemini 3.8 Live model code.
  • Documentation & Guides:
    • Updated README.md, README_KR.md, README_JP.md, and README_ZH.md with official Gemini 3.8 Live models.
    • Added Supported Live Models table to doc/api_reference.md.
    • Added Section 10: Gemini 3.8 Live Migration & Best Practices in doc/advanced_configuration.md covering model selection, omitting thinking_level on gemini-3.8-live, default non-blocking function execution, and completion tracking via LiveServerContent.isInteractionComplete.
    • Updated AI Agent Skill definition (SKILL.md) to guide assistant models with Gemini 3.8 Live patterns.

2026.9.11 #

Added #

  • Added LiveServerContent.isInteractionComplete getter aligned with upstream python-genai 2.23.0 interaction lifecycle logic:
    • If interactionStatus is specified, evaluates whether the status has settled to InteractionStatus.IDLE.
    • Otherwise, cleanly falls back to turnComplete == true.
    • Makes handling multi-turn conversational chunking, tool call completion, and live agent turn boundaries significantly more ergonomic for Flutter developers.

Changed #

  • Synced Gemini Live SDK request headers and user-agent string to google-genai-sdk/2.22.0 (aligned with upstream js-genai 2.22.0).
  • Bumped package version to 2026.9.11.

Tested #

  • Added unit tests for LiveServerContent.isInteractionComplete across all permutations (turnComplete, interactionStatus transitions).

2026.9.9 #

Added #

  • Synced DTOs and features with js-genai versions 2.17.0 through 2.21.0 and python-genai 2.17.0 through 2.22.0 (changes through September 2026):
    • InteractionStatus enum (UNSPECIFIED, IN_PROGRESS, @Deprecated('Use IDLE instead.') REQUIRES_ACTION, IDLE) representing the live session activity status.
    • LiveServerContent.interactionStatus property sent by the server alongside turnComplete.
    • MediaProcessing enum (MEDIA_PROCESSING_UNSPECIFIED, STATIC, AGENTIC) and Part.mediaProcessing field indicating how input media is processed.
    • AudioTranscriptionConfigMode enum (MODE_UNSPECIFIED, VERBATIM, SMART) and AudioTranscriptionConfig.mode field for configuring smart disfluency removal and formatting.
    • ToolParallelAiSearch model class and Tool.parallelAiSearch field (supported in enterprise agent platform grounding).
    • ToolType.MEDIA_PROCESSING enum value.
    • ServiceTier.DEFERRED ('deferred') enum value in ServiceTier.
    • AuthToken, LiveConnectConstraints, and CreateAuthTokenConfig DTO classes for ephemeral token constraints and configuration (expireTime, newSessionExpireTime, uses).
  • Reusable Flutter Material Widgets:
    • GeminiLiveStatusBadge & GeminiLiveStatusBadge.fromFlags(): Real-time visual badge tracking connection state and InteractionStatus with animated pulsing indicators.
    • GeminiLiveMicButton: Interactive microphone button with animated ripple effects during voice recording.
    • GeminiLiveVoiceIndicator: Lightweight waveform audio bar visualizer animating rhythmically when speech or streaming is active.
  • Example App Updates:
    • Integrated GeminiLiveStatusBadge, GeminiLiveMicButton, and GeminiLiveVoiceIndicator into LiveAPIDemoPage, ChatPage, and RealtimeMediaDemoPage.
    • Added live InteractionStatus monitoring (IN_PROGRESS / IDLE) and AudioTranscriptionConfigMode.SMART toggle to LiveAPIDemoPage.
  • Documentation & Specifications:
    • Added dedicated doc/widgets_guide.md specifying widget states, visual indicators, constructor APIs, and a complete end-to-end screen example.
    • Updated doc/api_reference.md, README.md, and README_KR.md with official widget documentation links.

Changed #

  • Upgraded package and example dependencies to their latest compatible versions (camera: ^0.12.1, build_runner: ^2.16.1, flutter_lints: ^6.0.0, etc.).
  • Synced Live SDK request headers and user-agent to google-genai-sdk/2.21.0 (synced with upstream js-genai 2.21.0).
  • LiveService.buildSetupMessage now explicitly validates and rejects Tool.parallelAiSearch in Gemini Developer API mode with clear descriptive error messages.
  • Bumped package version to 2026.9.9.

Fixed & Tested #

  • Expanded widget test suite (test/widgets_test.dart) to achieve 95%+ line coverage across all newly introduced Flutter widgets.
  • Verified 0 warnings and 0 errors across dart doc --dry-run and flutter analyze --fatal-infos --fatal-warnings.

Added #

  • Synced DTOs and features with js-genai versions 2.14.0, 2.15.0, and 2.16.0 (changes after July 24, 2026):
    • GenerationConfig.audioTranscriptionConfig (audio_transcription_config) and Part.audioTranscription (audio_transcription).
    • GoogleMaps model class and Tool.googleMaps grounding options (groundingTypes, e.g., places, routing).
    • TurnCompleteReason.TOO_MANY_TOOL_CALLS enum value for tool iteration safety terminations.

Changed #

  • Synced Live SDK request headers to google-genai-sdk/2.16.0 (synced with js-genai 2.16.0).
  • Bumped package version to 2026.8.12.

2026.7.24 #

Changed #

  • Synced Live SDK request headers to google-genai-sdk/2.13.0 (synced with js-genai 2.13.0).
  • Bumped package version to 2026.7.24.

2026.7.19 #

Added #

  • Synced Live-facing DTOs with js-genai 2.12.0:
    • HistoryConfig — new type with initialHistoryInClientContent flag; also added LiveClientSetup.historyConfig and LiveConnectParameters.historyConfig.
    • AudioTranscriptionConfig.customVocabulary — preferred replacement for adaptationPhrases (which is now @Deprecated).

Changed #

  • Updated Live SDK request headers to google-genai-sdk/2.12.0.
  • Updated all example and documentation model references from the deprecated gemini-live-2.5-flash-preview (shut down) to gemini-3.1-flash-live-preview (latest, March 2026).
    • Note: js-genai ref still uses the old model string in its test/sample files; this change is based on the official Gemini API models documentation.
  • Updated README and README_KR supported model descriptions to reflect the current lineup.
  • Bumped package version to 2026.7.19.

2026.7.11 #

Added #

  • Synced Live-facing DTOs with js-genai 2.11.0:
    • VoiceConsentSignature, plus ReplicatedVoiceConfig.consentAudio and ReplicatedVoiceConfig.voiceConsentSignature.
    • AudioTranscriptionConfig.languageAuto, AudioTranscriptionConfig.languageHints, and AudioTranscriptionConfig.adaptationPhrases, with the new LanguageAuto and LanguageHints types (languageCodes is now deprecated upstream).
    • ComputerUse.disabledSafetyPolicies and the SafetyPolicy enum.
    • Tool.exaAiSearch and the ToolExaAiSearch type (rejected for the Gemini API during setup).
    • Server fields: LiveServerSetupComplete.voiceConsentSignature, Transcription.languageCode, LiveServerContent.interimInputTranscription, VoiceActivity.audioOffset, and UsageMetadata.serviceTier with the new lowercase-valued ServiceTier enum.
    • LiveSession.setupComplete, populated once the initial setupComplete message arrives.

Changed #

  • Renamed StreamTranslationConfig to TranslationConfig and GenerationConfig.streamTranslationConfig to GenerationConfig.translationConfig (wire key translation_config). A deprecated StreamTranslationConfig typedef and GenerationConfig.streamTranslationConfig getter remain for backward compatibility.
  • connect() now resolves on the setupComplete message instead of the first frame: messages received before setup completes (including setupComplete itself) are queued and flushed to onMessage in arrival order once the session resolves.
  • Updated Live SDK request headers to google-genai-sdk/2.11.0.
  • Updated package metadata and README installation snippets to 2026.7.11.
  • Updated the example lockfile to reference the current local package version.

2026.6.6 #

Changed #

  • Updated package metadata and README installation snippets to 2026.6.6.
  • Updated the example lockfile to reference the current local package version.

2026.5.24 #

Added #

  • Synced Live-facing DTOs with js-genai 2.6.0:
    • GenerationConfig.streamTranslationConfig
    • ThinkingConfig.thinkingLevel
    • multi-speaker and replicated voice speech config types
    • FunctionCall.partialArgs / FunctionCall.willContinue
    • FunctionResponseBlob.displayName and FunctionResponseFileData.displayName
    • typed ComputerUse config including enablePromptInjectionDetection

Changed #

  • Updated Live SDK request headers to google-genai-sdk/2.6.0.
  • Updated package metadata and README installation snippets to 2026.5.24.

2026.4.19 #

Added #

  • Added configurable logging for Live API sessions through LiveService(logger: ...).
  • Example app now plays Gemini Live audio responses in-app by buffering PCM chunks, wrapping them as WAV, and playing them when a response turn completes.
  • Added timestamped chat demo debug logging for connection state, user sends, recorder events, and voice playback transitions.

Changed #

  • Live WebSocket traffic logging is now disabled by default.
  • Passing print as the logger restores the previous verbose logging behavior.
  • Updated publish metadata and README installation snippets to the current date-based version.
  • Split example app Live model defaults by demo:
    • Realtime Media now targets gemini-3.1-flash-live-preview.
    • chat, function-calling, and feature demos stay on the 2.5 compatibility path until the SDK exposes the additional 3.1 history/session controls.
  • Restored chat demo Text Mode / Voice Mode switching while keeping audio-response sessions compatible with current Live API behavior.
  • Refined chat voice playback control so autoplay is consumed once per response and old audio bubbles do not restart when the list rebuilds.
  • Reduced noisy chat logging by suppressing empty text chunk logs and summarizing audio chunk progress per response.

Fixed #

  • Fixed the example app disconnection issue reported in GitHub issue #8 by removing non-audio request flows from Live setup in the app demos.
  • Fixed example audio playback cleanup so buffered audio is cleared on interruption, connection close, and error paths.
  • Fixed example playback triggering to handle turns that finish with generationComplete as well as turnComplete.
  • Fixed chat voice playback so new user input or a newer response stops the currently playing voice response instead of overlapping playback.
  • Fixed macOS image picking in the example app by adding the required user-selected read-only sandbox entitlement.
  • Fixed chat screen teardown races so closing or leaving the page no longer triggers setState() on a defunct widget after the session closes.

2026.4.17 #

Added #

  • Synced additional Gemini Live setup fields from the current js-genai reference:
    • LiveClientSetup.avatarConfig
    • LiveClientSetup.safetySettings
    • SafetySetting, AvatarConfig, CustomizedAvatar
  • Expanded Live enum coverage for newer server responses:
    • Modality.VIDEO
    • TurnCoverage.TURN_INCLUDES_AUDIO_ACTIVITY_AND_ALL_VIDEO
    • additional TurnCompleteReason values
    • additional TrafficType values
    • latest safety enums for Gemini blocking config

Changed #

  • Updated Live SDK request headers to google-genai-sdk/1.50.1.
  • Updated README and README_KR to match the current API surface and date-based package versioning.

2026.3.21 #

Added #

  • Added the latest Gemini Live part-schema fields from the current js-genai reference:
    • Part.mediaResolution, Part.fileData, Part.videoMetadata
    • Part.thoughtSignature, Part.toolCall, Part.toolResponse, Part.partMetadata
    • PartMediaResolution, PartMediaResolutionLevel, ToolCall, ToolResponse, ToolType
  • Added AudioTranscriptionConfig.languageCodes and code execution IDs on ExecutableCode / CodeExecutionResult.

Changed #

  • Tightened Gemini Live setup validation to reject unsupported input/outputAudioTranscription.languageCodes before sending setup.
  • Added client-side MIME validation for realtime audio/video payloads so malformed blobs fail fast.

Fixed #

  • Fixed LiveServerMessage.data so multiple inline binary chunks are concatenated as bytes before base64 encoding.
  • Fixed session resumption parsing parity by treating lastConsumedClientMessageIndex as a string.

0.2.2 #

Added #

  • Example app now supports in-app Gemini API key configuration via Settings UI.
    • Users can save/update/clear API keys directly from the app screen.
    • API key is persisted locally with shared_preferences.
  • Added examples/ephemeral_token.dart:
    • demonstrates Live API connection using ephemeral tokens (auth_tokens/...)
    • uses GoogleGenAI(apiVersion: 'v1alpha')
  • Added platform-specific runtime info adapters so Dart version reporting works consistently across IO and web builds.
  • Added comprehensive automated coverage for Live service flows, transport connectors, and model serialization.

Changed #

  • Improved Gemini Live setup handling:
    • added LiveService.buildSetupMessage(...) to centralize setup message construction
    • added LiveService.validateFunctionResponses(...) to validate tool responses before sending
    • default setup now falls back to responseModalities: [Modality.AUDIO] when omitted
    • unsupported Gemini API options now fail fast during setup construction
  • Expanded Function Calling demo in the example app:
    • added richer tool declarations:
      • get_exchange_rate
      • convert_currency
      • search_places
      • create_reminder (non-blocking)
    • improved simulated tool responses with structured payloads
    • added quick prompt chips for multi-tool scenarios
  • Updated example/docs descriptions to reflect richer function calling capabilities.
  • Documented the public model API more clearly in the package docs.

Fixed #

  • Enforced Gemini Live function response IDs so tool replies match ToolCall.functionCalls contract.
  • Removed Gemini example usage that relied on unsupported setup parameters.

0.2.1 #

Added #

  • Expanded Live tool schema support:
    • Tool.functionDeclarations
    • Tool.googleSearch
    • Tool.googleSearchRetrieval
  • Added richer function response fields:
    • FunctionResponse.scheduling
    • FunctionResponse.willContinue
    • FunctionResponse.parts
  • Added Live generation config parity fields:
    • GenerationConfig.mediaResolution
    • GenerationConfig.speechConfig (voice config)
    • GenerationConfig.thinkingConfig
    • GenerationConfig.enableAffectiveDialog
    • GenerationConfig.seed
  • Added additional Live server metadata parsing:
    • LiveServerContent.interrupted
    • LiveServerContent.waitingForInput
    • LiveServerContent.turnCompleteReason
    • LiveServerSetupComplete.sessionId
    • detailed UsageMetadata modality/token breakdown fields

Changed #

  • Updated WebSocket connection behavior for Gemini Live ephemeral tokens:
    • auto-switches to BidiGenerateContentConstrained
    • uses access_token query parameter when API key starts with auth_tokens/
  • Added apiVersion option to GoogleGenAI(...) so Live API version can be configured from the top-level client.
  • Updated Live User-Agent version to google-genai-sdk/1.42.0.
  • Updated enums and compatibility mapping for current Live API:
    • ActivityHandling.NO_INTERRUPTION (with backward-compatible alias)
    • VAD/voice activity typed signals
    • traffic/modality token metadata enums

Fixed #

  • Fixed tool serialization so configured tools are actually sent in setup.
  • Fixed session resumption parsing type for resumable (bool).
  • Fixed GoAway parsing to align with timeLeft and added helper conversion to seconds (timeRemaining).
  • Improved LiveServerMessage.text getter to ignore thought-only parts.

0.2.0 #

New Features (Based on js-genai v1.39.0 Gemini Live API Updates) #

Live Server Message Types

  • Added LiveServerToolCall - Handle tool/function call requests from the model
  • Added LiveServerToolCallCancellation - Handle tool call cancellation
  • Added LiveServerGoAway - Receive server disconnect warnings
  • Added LiveServerSessionResumptionUpdate - Handle session resumption updates
  • Added VoiceActivityDetectionSignal - Voice activity detection start/end signals
  • Added VoiceActivity - Real-time voice activity status

Live Client Setup Configuration

  • Added RealtimeInputConfig - Configure automatic activity detection, activity handling, and turn coverage
  • Added SessionResumptionConfig - Enable session resumption with handle and transparent mode
  • Added ContextWindowCompressionConfig - Configure context window compression with trigger tokens and sliding window
  • Added AudioTranscriptionConfig - Enable input/output audio transcription
  • Added ProactivityConfig - Configure proactive audio features
  • Added explicitVadSignal option - Enable explicit VAD signaling

Live Client Realtime Input Enhancements

  • Added mediaChunks support - Send multiple media chunks at once
  • Added audioStreamEnd - Signal end of audio stream
  • Added text - Send real-time text input
  • Added ActivityStart / ActivityEnd - Manual activity detection signals

Function Calling Support

  • Added LiveClientToolResponse - Send tool/function responses to the model
  • Added FunctionCall - Model function call representation with id, name, and args
  • Added FunctionResponse - Function response with id, name, and response data

New Session Methods

  • Added sendClientContent() - Send multi-turn client content
  • Added sendRealtimeInput() - Send combined real-time input (audio, video, text, activity signals)
  • Added sendMediaChunks() - Send media chunks array
  • Added sendAudioStreamEnd() - Signal audio stream end
  • Added sendRealtimeText() - Send real-time text
  • Added sendActivityStart() / sendActivityEnd() - Manual activity detection
  • Added sendToolResponse() - Send tool responses
  • Added sendFunctionResponse() - Send single function response
  • Added close() - Close WebSocket connection
  • Added isClosed getter - Check connection status

New Enums

  • Added ActivityHandling - Activity handling strategies (START_OF_ACTIVITY_INTERRUPTS, etc.)
  • Added TurnCoverage - Turn coverage options (TURN_INCLUDES_ALL_INPUT, etc.)
  • Added StartSensitivity - Speech start sensitivity levels
  • Added EndSensitivity - Speech end sensitivity levels

Example App Updates

  • Added live_api_demo.dart - Comprehensive demo of all new features
  • Added function_calling_demo.dart - Function calling demo with weather/time functions
  • Added realtime_media_demo.dart - Real-time media input with manual/auto VAD
  • Updated main.dart - New home page with navigation to all demos

Improvements #

  • Added data getter to LiveServerMessage for accessing base64 encoded inline data
  • Updated User-Agent to google-genai-sdk/1.39.0
  • Added TimeoutException for connection timeout handling
  • Improved documentation and examples

0.1.1 #

  • Update dependencies and generated model serialization
  • Update web_socket_service_web.dart
  • Update web_socket_service_stub.dart

0.1.0 #

  • Update Readme
  • Documentation

0.0.5 #

  • Update Readme

0.0.4 #

  • Update Readme

0.0.3 #

  • Add Example
  • Improve Web Support
  • Remove Platform.version (dartVersion)

0.0.2 #

  • Update README.md

0.0.1 #

  • Initial version of the package.
  • Added Gemini Live Code
10
likes
150
points
2.84k
downloads

Documentation

API reference

Publisher

unverified uploader

Weekly Downloads

A Flutter package for the Gemini Live API, enabling real-time, multimodal conversations with Google's Gemini models.

Repository (GitHub)
View/report issues

License

BSD-3-Clause (license)

Dependencies

flutter, http, json_annotation, web_socket_channel

More

Packages that depend on gemini_live