gemini_live 2026.10.6

A Flutter package for the Gemini Live API, enabling real-time, multimodal conversations with Google's Gemini models.

Flutter Gemini Live

English | 한국어 | 日本語 | 简体中文

---

• A Flutter package for the Gemini Live API, enabling real-time, multimodal conversations with Google's Gemini models.
• Zero Firebase Dependency: Direct WebSocket connection without Firebase or Firebase AI Logic.
• Firebase AI Logic compatible: Optional drop-in layer that mirrors the firebase_ai Live API, so code moves between the two packages by swapping imports. See Firebase AI Logic Compatibility.
• Supports the latest Gemini Live, translation, and Lyria music models. See Supported Models.
• Supports TEXT, AUDIO, and VIDEO response modalities.

https://github.com/user-attachments/assets/7d826f37-196e-4ddd-8828-df66db252e8e

🤖 AI Agents & Coding Assistants Guide

If you are an AI coding assistant (Claude Code, Gemini CLI / Antigravity, OpenAI Codex, Cursor, Windsurf, or GitHub Copilot): 👉 Read the official package skill specification first:

Available Skills

Skill	Description	Example prompt
flutter-gemini-live	Build real-time, multimodal streaming applications in Flutter using Google Gemini Live API without Firebase. Covers low-latency voice chat, video camera streaming, function calling, audio transcription, live translation, and Lyria real-time music generation.	Build a low-latency Gemini Live voice chat screen with camera streaming and function calling
gemini-live-widgets	Pre-built Flutter Material 3 & Cupertino UI components: mic buttons, FFT spectrum waveforms, frosted glass captions, status pills, and token usage monitors with GeminiLiveSessionController wiring.	Add an animated microphone button and real-time audio waveform visualizer to my Gemini Live call view
gemini-live-firebase-migration	1:1 bidirectional migration between firebase_ai (Firebase AI Logic) and gemini_live. Drop Firebase dependencies without changing your Live session code using package:gemini_live/compat/firebase_ai.dart.	Migrate my existing firebase_ai LiveSession code to gemini_live to remove Firebase dependencies

Agent Compatibility & Discovery Paths

Assistant / Ecosystem	Native Skill Discovery Path	Project Instructions
Root Universal / AGENTS.md	SKILL.md	AGENTS.md
Common Skills Directory / Hermes	skills/flutter-gemini-live/SKILL.md	skills/README.md
Hermes Agent	.hermes/skills/flutter-gemini-live/SKILL.md	-
Pi Agent	.pi/skills/flutter-gemini-live/SKILL.md	-
Claude Code	.claude/skills/flutter-gemini-live/SKILL.md	CLAUDE.md
Gemini CLI / Antigravity	.gemini/skills/flutter-gemini-live/SKILL.md	GEMINI.md
OpenAI Codex / Cursor	.codex/skills/flutter-gemini-live/SKILL.md	CODEX.md
Agent Skills Standard	.agents/skills/flutter-gemini-live/SKILL.md	-

Install via npx skills (skills.sh)

# Install the core skill to your current project (or add -g for global)
npx skills add JAICHANGPARK/flutter_gemini_live

# Install all skills (core, widgets, migration) across all agents
npx skills add JAICHANGPARK/flutter_gemini_live --full-depth --all

Install via curl (Zero-Node One-Liner)

# Auto-detects existing agents (~/.claude, ~/.gemini, ~/.codex, etc.) and installs
curl -fsSL https://raw.githubusercontent.com/JAICHANGPARK/flutter_gemini_live/main/tool/install_skills.sh | bash

# Or install to specific agent(s) only:
# curl -fsSL https://raw.githubusercontent.com/JAICHANGPARK/flutter_gemini_live/main/tool/install_skills.sh | bash -s -- claude gemini

📌 Core Agent Rules & Invariants

1. Zero Firebase: Never import firebase_core or firebase_vertexai for live streaming. Connect directly via GoogleGenAI WebSocket endpoints.
2. State Management: For Flutter UI apps, prefer GeminiLiveSessionController (ChangeNotifier) to handle connection state, transcripts, audio streams, and interruptions.
3. Pre-built Material 3 Widgets: Use built-in widgets (GeminiLiveWaveform, GeminiLiveCaptionBubble, GeminiLiveMicButton, GeminiLiveStatusBadge, GeminiLiveVoiceIndicator, GeminiLiveUsageBadge).
4. Barge-in Interruption: Always clear/stop local audio playback buffers immediately when serverContent.interrupted == true or controller.isInterrupted is true.
5. Audio Formats:
   • Mic Input: Linear PCM 16-bit, 16,000 Hz mono.
   • Live Output: Linear PCM 16-bit, 24,000 Hz mono (Music: 48,000 Hz stereo).
6. Models: Default to gemini-3.8-live (low-latency) or gemini-3.8-live-extended-thinking (deep reasoning).
7. Audio I/O is yours: GeminiLiveSessionController does not record or play audio. Stream mic PCM into sendRealtimeAudio() and play incomingAudioStream (e.g. record + flutter_soloud).

Supported Models

Use the model ID string, or the matching constant from LiveModels / LiveMusicModels.

Live API (genAI.live.connect)

Model ID	Constant	Use for	Status
gemini-3.8-live	LiveModels.gemini38Live	Default. Low-latency voice & multimodal dialogue	Stable
gemini-3.8-live-extended-thinking	LiveModels.gemini38LiveExtendedThinking	Voice with deeper reasoning (thinkingConfig)	Stable
gemini-3.5-live-translate-preview	LiveModels.gemini35LiveTranslatePreview	Speech-to-speech translation (TranslationConfig)	Preview
gemini-3.1-flash-live-preview	LiveModels.gemini31FlashLivePreview	Previous generation	Preview
gemini-2.5-flash-native-audio-preview-12-2025	LiveModels.gemini25FlashNativeAudioPreview	Native audio output	Preview

Live Music (genAI.live.music.connect)

Model ID	Constant	Use for	Status
models/lyria-realtime-exp	LiveMusicModels.lyriaRealtimeExp	Default. Real-time music generation	Experimental

• Pass thinkingConfig only to gemini-3.8-live-extended-thinking. gemini-3.8-live rejects it.
• gemini-3.8-live runs tool calls as non-blocking by default (Behavior.NON_BLOCKING).
• Live output audio is 16-bit PCM 24 kHz mono. Lyria output is 16-bit PCM 48 kHz stereo.
• Model availability changes over time. Check the Gemini API models page for the latest status.

Installation

Add the package to your Flutter project:

flutter pub add gemini_live

Import the package in Dart:

import 'package:gemini_live/gemini_live.dart';

Quick Start

Get up and running in under 20 lines of code:

import 'package:gemini_live/gemini_live.dart';

void main() async {
  // 1. Initialize Gemini Live client
  final genAI = GoogleGenAI(apiKey: 'YOUR_GEMINI_API_KEY', logger: print);

  // 2. Connect to the Live API
  final session = await genAI.live.connect(
    LiveConnectParameters(
      model: 'gemini-3.8-live',
      config: GenerationConfig(responseModalities: [Modality.TEXT]),
      callbacks: LiveCallbacks(
        onOpen: () => print('Live Session Connected!'),
        onMessage: (message) {
          if (message.text != null) {
            print('Gemini: ${message.text}');
          }
        },
        onError: (error, st) => print('Error: $error'),
        onClose: (code, reason) => print('Closed: $code - $reason'),
      ),
    ),
  );

  // 3. Send a message
  session.sendText('Hello Gemini, tell me a quick joke!');
}

Voice Quick Start

Most Live apps are voice apps. GeminiLiveSessionController streams audio, but it does not record the microphone or play sound. Connect it to record (input) and flutter_soloud (output):

flutter pub add gemini_live record flutter_soloud

Add the microphone permission for each platform:

Platform	Setting
Android	AndroidManifest.xml: android.permission.RECORD_AUDIO, android.permission.INTERNET
iOS	Info.plist: NSMicrophoneUsageDescription
macOS	Entitlements: com.apple.security.device.audio-input, com.apple.security.network.client

import 'package:flutter/foundation.dart';
import 'package:flutter_soloud/flutter_soloud.dart';
import 'package:gemini_live/gemini_live.dart';
import 'package:record/record.dart';

final genAI = GoogleGenAI(apiKey: 'YOUR_GEMINI_API_KEY');
final controller = GeminiLiveSessionController(liveService: genAI.live);
final recorder = AudioRecorder();

Future<void> startVoiceChat() async {
  if (!await recorder.hasPermission()) return;

  await controller.connect(
    LiveConnectParameters(
      model: 'gemini-3.8-live',
      config: GenerationConfig(
        responseModalities: [Modality.AUDIO],
        speechConfig: SpeechConfig.fromLiveVoice(GeminiLiveVoice.puck),
      ),
      outputAudioTranscription: AudioTranscriptionConfig(),
      callbacks: LiveCallbacks(onError: (e, st) => debugPrint('Live error: $e')),
    ),
  );

  // Speaker: model audio is 16-bit PCM, 24 kHz, mono.
  await SoLoud.instance.init();
  final speaker = SoLoud.instance.setBufferStream(
    sampleRate: 24000,
    channels: Channels.mono,
    format: BufferType.s16le,
    bufferingType: BufferingType.released,
  );
  SoLoud.instance.play(speaker);
  controller.incomingAudioStream.listen(
    (pcm) => SoLoud.instance.addAudioDataStream(speaker, pcm),
  );

  // Barge-in: drop queued model audio when the user interrupts.
  controller.addListener(() {
    if (controller.isInterrupted) SoLoud.instance.resetBufferStream(speaker);
  });

  // Mic: send 16-bit PCM, 16 kHz, mono.
  final mic = await recorder.startStream(
    const RecordConfig(
      encoder: AudioEncoder.pcm16bits,
      sampleRate: 16000,
      numChannels: 1,
      echoCancel: true,
      noiseSuppress: true,
    ),
  );
  mic.listen(controller.sendRealtimeAudio);
}

For a complete screen with captions, waveform, and barge-in handling, see the Agent Skill quickstart and the example app.

🔐 API key security: Do not ship a raw Gemini API key in a production app. Mint short-lived ephemeral tokens on your backend with genAI.authTokens.create(...) and connect the client with that token. See Ephemeral Tokens and examples/ephemeral_token.dart.

Firebase AI Logic Compatibility

package:gemini_live/compat/firebase_ai.dart mirrors the Live API of firebase_ai 4.x: same class names, constructors, methods and defaults (FirebaseAI.googleAI(), liveGenerativeModel, LiveGenerationConfig, LiveSession.send*, receive(), tool calls, session resumption). It runs on the gemini_live engine with no Firebase project. The core gemini_live API is unchanged; this is a separate entry point.

import 'package:gemini_live/compat/firebase_ai.dart';

FirebaseAI.initialize(apiKey: 'YOUR_KEY_OR_EPHEMERAL_TOKEN');

final model = FirebaseAI.googleAI().liveGenerativeModel(
  model: 'gemini-2.5-flash-native-audio-preview-12-2025',
  liveGenerationConfig: LiveGenerationConfig(
    responseModalities: [ResponseModalities.audio],
    speechConfig: SpeechConfig.fromLiveVoice(GeminiLiveVoice.puck),
    outputAudioTranscription: AudioTranscriptionConfig(),
  ),
);
final session = await model.connect();

session.receive().listen((response) async {
  final message = response.message;
  if (message is LiveServerContent) {
    for (final part in message.modelTurn?.parts ?? const <Part>[]) {
      if (part is InlineDataPart) playPcm(part.bytes);
    }
  } else if (message is LiveServerToolCall) {
    for (final call in message.functionCalls ?? const <FunctionCall>[]) {
      await session.sendToolResponse([
        FunctionResponse(call.name, await runTool(call), id: call.id),
      ]);
    }
  }
});

await session.sendAudioRealtime(InlineDataPart('audio/pcm;rate=16000', chunk));

// gemini_live-only extra (firebase_ai has no Live usage metadata):
print(session.tokenTracker.formatCost());

Only the Live API is covered. vertexAI(), generativeModel and other non-Live APIs are not provided and fail at compile time.

Migration

firebase_ai → gemini_live (two edits; everything after FirebaseAI.googleAI() stays the same):

- import 'package:firebase_core/firebase_core.dart';
- import 'package:firebase_ai/firebase_ai.dart';
+ import 'package:gemini_live/compat/firebase_ai.dart';

- await Firebase.initializeApp(options: DefaultFirebaseOptions.currentPlatform);
+ FirebaseAI.initialize(apiKey: 'YOUR_KEY_OR_EPHEMERAL_TOKEN');

• Firebase App Check no longer protects the key. Use ephemeral tokens in production; auth_tokens/... tokens select v1alpha automatically.
• FirebaseAI.vertexAI() must become googleAI().

gemini_live → firebase_ai (reverse the two edits):

• Remove the gemini_live-only extras the compiler flags: FirebaseAI.initialize, googleAI(apiKey:), session.tokenTracker, session.rawSession, response.rawMessage.
• Subscribe to receive() before sending. gemini_live buffers early responses; firebase_ai drops them.
• Only code written against the compat layer is portable. Code using the core API (GoogleGenAI, GeminiLiveSessionController) must be rewritten.

The Firebase AI compatibility guide has the full API table, behavior differences, and how to use both packages in one app. Two-way compatibility is checked by tool/check_firebase_ai_compat.sh, which compiles the same sample against both packages.

Documentation & Guides

For deep dives and complete references, see the modular guides in the doc/ directory:

• AI Agent Skill Guide (skills/): Agent instructions for Claude Code, Gemini CLI / Antigravity, OpenAI Codex, Cursor, and Copilot.
• API Reference: Complete class & method documentation for GoogleGenAI, LiveSession, LiveServerMessage, etc.
• Widgets Guide & UI Specification: Detailed specification and interactive code examples for GeminiLiveSessionController and every pre-built widget.
• Advanced Configuration Guide: Guides for Function Calling, VAD, Session Resumption, Audio Transcription, Translation, Grounding, and Ephemeral Tokens.
• Firebase AI Compatibility & Migration: firebase_ai-compatible Live API, two-way migration checklists, and behavior differences.
• Error Codes & Specifications: Complete error codes, close codes, TurnCompleteReason enums, and troubleshooting strategies.
• Runnable Examples: Dedicated CLI scripts for basic usage, function calling, audio/video streaming, and Google Maps grounding.

Key Features Overview

• Real-time Communication: Low-latency WebSocket interaction.
• Multimodal Input & Streaming Output: Text, audio, and camera frame input with live streaming responses.
• Function Calling: Synchronous and asynchronous function execution.
• Session Resumption: Resume dropped connections via session handles.
• Google Maps & Search Grounding: Location and routing-aware responses.
• Voice Activity Detection: Automatic and manual VAD.
• Type-Safe Voice Personas (GeminiLiveVoice): Official 30-voice Google Gemini Live and TTS roster (puck, charon, kore, fenrir, aoede, leda, orus, zephyr, etc.) with compile-time safety and tone metadata.
• Live Speech Translation: Real-time speech-to-speech translation (TranslationConfig).
• Pre-built Flutter Widgets & Controller: GeminiLiveSessionController, GeminiLiveWaveform, GeminiLiveCaptionBubble, GeminiLiveMicButton, GeminiLiveStatusBadge, GeminiLiveUsageBadge, GeminiLiveVoiceIndicator.
• Live Music (Lyria Realtime): Steerable real-time music generation via genAI.live.music.
• Ephemeral Auth Tokens: Short-lived client tokens via genAI.authTokens to keep API keys off devices.
• Token Usage Tracking: Per-session token accounting with GeminiTokenUsageTracker.

Demo 1: Chihuahua vs muffin	Demo 2: Labradoodle vs fried chicken
Chihuahua vs muffin	Labradoodle vs fried chicken

---

Pre-built UI Widgets

The package ships with ready-to-use Flutter Material widgets to accelerate building Live conversational interfaces:

// 1. Reactive Live Session Controller (ChangeNotifier)
final controller = GeminiLiveSessionController(liveService: genAI.live);
await controller.connect(
  LiveConnectParameters(
    model: 'gemini-3.8-live',
    config: GenerationConfig(responseModalities: [Modality.AUDIO]),
    callbacks: LiveCallbacks(),
  ),
);

// 2. Real-time Audio Waveform Visualizer (Capsule bars & idle breathing)
GeminiLiveWaveform(
  audioStream: controller.incomingAudioStream, // or amplitudeStream
  barCount: 28,
  height: 64,
  color: Theme.of(context).colorScheme.primary,
  enableIdleBreathing: true,
)

// 3. Frosted-Glass Live Caption Bubble (BackdropFilter blur & speaker chips)
GeminiLiveCaptionBubble(
  text: controller.latestTranscript ?? '',
  speaker: controller.latestTranscriptRole == 'user' ? 'You' : 'Gemini',
  isStreaming: controller.isModelSpeaking,
  enableBlur: true,
)

// 4. Concentric Ripple Microphone Button
GeminiLiveMicButton(
  isRecording: controller.isConnected,
  onPressed: () => toggleLiveSession(),
)

// 5. Connection Status Badge with Halo Pulse
GeminiLiveStatusBadge.fromFlags(
  isConnected: controller.isConnected,
  isConnecting: isConnecting,
)

// 6. Real-time Token Usage & Observability Badge
GeminiLiveUsageBadge(
  tracker: controller.tokenTracker,
)

// 7. Dual-Harmonic Voice Indicator
GeminiLiveVoiceIndicator(
  isSpeaking: controller.isModelSpeaking,
  barCount: 5,
)

---

License

This project is licensed under the BSD 3-Clause License - see the LICENSE file for details.