pipecat 0.3.0

Pipecat client SDK for Flutter: build real-time voice and multimodal AI agents on iOS and Android with Daily or SmallWebRTC, over the native SDKs.

pipecat: Pipecat client SDK for Flutter

pipecat is a community-maintained Flutter plugin for Pipecat, the open-source framework for real-time voice and multimodal AI agents. It wraps the official Pipecat iOS and Android client SDKs behind a type-safe Dart API and supports Daily and SmallWebRTC transports. Package: pipecat on pub.dev; import package:pipecat/pipecat.dart.

Last updated: 2026-10-12 · Version 0.3.0 · iOS SDK 1.3.0 · Android SDK 1.2.0 (transports 1.2.2) · RTVI 2.0.0

Install

Run flutter pub add pipecat, or add the dependency by hand. The plugin needs Flutter 3.22 or newer (Dart 3.4+), iOS 15.0 or newer, and Android minSdk 24, compileSdk 36, Kotlin Gradle Plugin 2.2 or newer and JVM 17.

dependencies:
  pipecat: ^0.3.0

iOS. Add usage strings to ios/Runner/Info.plist:

<key>NSMicrophoneUsageDescription</key>
<string>This app uses the microphone to talk to the AI assistant.</string>
<key>NSCameraUsageDescription</key>
<string>This app uses the camera so the AI assistant can see you.</string>

SmallWebRTC on iOS needs Flutter's Swift Package Manager integration (flutter config --enable-swift-package-manager). With CocoaPods only, SmallWebRTCTransport fails with a transport-unavailable error and Daily still works.

Android. Add permissions to android/app/src/main/AndroidManifest.xml:

<uses-permission android:name="android.permission.INTERNET"/>
<uses-permission android:name="android.permission.RECORD_AUDIO"/>
<uses-permission android:name="android.permission.CAMERA"/>
<uses-permission android:name="android.permission.MODIFY_AUDIO_SETTINGS"/>
<uses-permission android:name="android.permission.BLUETOOTH"/>
<uses-permission android:name="android.permission.BLUETOOTH_CONNECT"/>

You request runtime permissions (microphone, camera) yourself before initDevices(). A package such as permission_handler works for this.

Quick start (60 seconds)

Create a client with a transport, listen to the streams you need, initialize it, then connect. The snippet below uses SmallWebRTCTransport and points at the offer URL of your own Pipecat server. Before connecting, it grabs futures for the first onBotReady and onBotTranscript events, so neither event can fire before anything is listening. After connecting it waits for the bot to be ready, sends one text message, waits for the first bot transcript, then disconnects and releases the client.

import 'package:pipecat/pipecat.dart';

Future<void> main() async {
  final client = PipecatClient(transport: const SmallWebRTCTransport());

  client.onConnected.listen((_) => print('connected'));
  client.onBotReady.listen((data) => print('bot ready: ${data.version}'));
  client.onUserTranscript.listen((t) {
    if (t.finalStatus ?? false) print('user: ${t.text}');
  });
  client.onBotTranscript.listen((text) => print('bot: $text'));

  // Subscribe before connecting so neither event is missed.
  final ready = client.onBotReady.first;
  final firstReply = client.onBotTranscript.first;

  await client.initialize(enableMic: true, enableCam: false);
  await client.initDevices();
  await client.connect(
    transportParams: const SmallWebRTCConnectParams(
      webrtcUrl: 'https://your-pipecat-server.example/offer',
    ),
  );
  await ready;
  await client.sendText('Hello!');
  await firstReply;

  await client.disconnect();
  await client.release();
  client.dispose();
}

With the Pipecat quickstart bot (uv run bot.py), use http://<your-machine-ip>:7860/api/offer (Android emulator: http://10.0.2.2:7860/api/offer). Plain http needs an iOS ATS exception and Android cleartext traffic allowed.

How do I connect with Daily instead of SmallWebRTC?

Construct the client with DailyTransport and connect with DailyConnectParams. The transport and the params must match. A mismatch throws PipecatTransportMismatchException before any platform-channel call. If your server returns transport params from an HTTP endpoint, call startBotAndConnect(endpoint:) instead of connect.

import 'package:pipecat/pipecat.dart';

Future<void> connectDaily() async {
  final client = PipecatClient(transport: const DailyTransport());
  await client.initialize();
  await client.connect(
    transportParams: const DailyConnectParams(
      roomUrl: 'https://your-org.daily.co/your-room',
      token: 'meeting-token-from-your-server',
    ),
  );
}

Future<void> connectViaEndpoint(PipecatClient client) async {
  await client.startBotAndConnect(
    endpoint: 'https://your-pipecat-server.example/start',
  );
}

How do I handle LLM function calls in Flutter?

Register a handler by function name with registerFunctionCallHandler. The handler receives the call, including its arguments as a Value, and returns a Value that goes back to the LLM. Remove it with unregisterFunctionCallHandler. The older registerFunctionHandler and unregisterFunctionHandler names still work but are deprecated.

import 'package:pipecat/pipecat.dart';

void registerWeather(PipecatClient client) {
  client.registerFunctionCallHandler('get_weather', (call) async {
    final args = call.args;
    final location = (args is ValueObject)
        ? (args.properties['location'] as ValueString?)?.value
        : null;
    print('weather requested for $location');
    return ValueObject({
      'weather': const ValueString('sunny'),
      'tempF': const ValueNumber(72),
    });
  });
}

Which native Pipecat SDK versions does this use?

Version 0.3.0 pins the official Pipecat client SDKs listed below. Both platforms report RTVI protocol 2.0.0. The plugin fires onProtocolMismatch when the bot and the client differ in major version. The pins come from ios/pipecat/Package.swift, ios/pipecat.podspec and android/build.gradle.

Component	Version	Package manager	Source
PipecatClientIOS	1.3.x (~> 1.3.0)	SwiftPM, CocoaPods	pipecat-client-ios
PipecatClientIOSDaily	1.3.x (~> 1.3.0)	SwiftPM, CocoaPods	pipecat-client-ios-daily
PipecatClientIOSSmallWebrtc	1.3.x	SwiftPM only	pipecat-client-ios-small-webrtc
ai.pipecat:client	1.2.0	Gradle	Pipecat Android SDK
ai.pipecat:daily-transport	1.2.2	Gradle	Pipecat Android SDK
ai.pipecat:small-webrtc-transport	1.2.2	Gradle	Pipecat Android SDK
RTVI protocol	2.0.0	iOS and Android	Pipecat docs

Feature support vs the native Pipecat SDKs

Most of the Pipecat client API is available on both platforms through one Dart interface. The table lists what the plugin exposes in 0.3.0 and where iOS and Android differ. A cross mark means the plugin cannot offer it because the native SDK does not.

Capability	pipecat (Flutter) on iOS	pipecat (Flutter) on Android	Notes
Daily transport	✅	✅	SwiftPM and CocoaPods on iOS
SmallWebRTC transport	⚠️	✅	iOS needs Swift Package Manager
RTVI 2.0 protocol	✅	✅	Both report 2.0.0
Bot output fields (willBeSpoken, spokenStatus, spokenProgress, segmentId)	✅	✅	Custom aggregated_by values are not decoded on iOS (upstream limit)
Transcripts (onUserTranscript, onBotTranscript)	✅	✅
Function calls	✅	✅	registerFunctionCallHandler
sendText, sendAction	✅	✅
Client messages and typed sendClientRequest	✅	✅	Request returns ClientMessageData
appendToContext	⚠️	⚠️	Deprecated upstream. iOS accepts user and assistant roles, Android also system
Screen share	❌	❌	Not exposed by the native SDKs
setLogLevel	✅	⚠️	No-op on Android: the SDK has no runtime log level
Mic and camera selection	✅	✅
Speaker selection	✅	❌	Android returns an empty list and unsupported-on-android
Audio levels (onLocalAudioLevel, onRemoteAudioLevel)	✅	✅
Metrics (onMetrics)	✅	✅
Device errors (onDeviceError)	✅	✅	Synthesized by the plugin
User mute events	✅	✅	Android uses a transport interceptor
Full error envelope	✅	✅	onServerError, onErrorResponse, PipecatBackendException
iceConfig	✅	✅	Only through connect(SmallWebRTCConnectParams(...))

pipecat vs pipecat_flutter: which Flutter package should I use?

Two Flutter packages wrap Pipecat clients: this one (pipecat) and pipecat_flutter by a different author. This package bundles Daily and SmallWebRTC in a single plugin. pipecat_flutter is a federated plugin that uses Daily. Check each package's README against your transport and RTVI needs before choosing.

	pipecat (this package)	pipecat_flutter
Latest version	0.3.0	5.0.0
Last published	0.3.0, on release	2026-08-31
Transports	Daily, SmallWebRTC	Daily (README: "The native implementation uses a Daily transport")
Platforms	Android, iOS	Android, iOS (web, desktop: not supported per README)
Function calling	Yes (registerFunctionCallHandler)	not stated
RTVI version	2.0.0	not stated
Architecture	Single plugin	Federated (_android, _ios, _platform_interface)
Licence	BSD 2-Clause	BSD-3-Clause
Maintainer	minhtri1401	kirkcharlesniv (publisher: unverified uploader)
Package page	pub.dev/packages/pipecat	pub.dev/packages/pipecat_flutter

Checked on 2026-10-12.

FAQ

Is this an official Pipecat SDK?

No. It is a community-maintained Flutter plugin and is not published by the Pipecat team. It wraps the official Pipecat iOS and Android client SDKs, so server behavior and RTVI messages follow the upstream projects. The documentation for those SDKs is at docs.pipecat.ai.

Does it support Flutter web or desktop?

No. The plugin declares Android and iOS only, because it wraps the native iOS and Android client SDKs. Web, Windows, macOS and Linux are not supported in 0.3.0. For a web client, use the official Pipecat JavaScript client SDK from the Pipecat docs.

Which Pipecat server do I need?

Any Pipecat server that speaks RTVI and offers the transport you choose. Use a Daily room for DailyTransport, or an endpoint that accepts a SmallWebRTC offer for SmallWebRTCTransport. The client reports RTVI protocol 2.0.0 and raises onProtocolMismatch if the bot's major version differs.

Why is the package called pipecat and not pipecat_flutter?

The pub.dev name is pipecat, and you import package:pipecat/pipecat.dart. The GitHub repository is named pipecat_flutter, which is why the two differ. A separate, unrelated package on pub.dev is called pipecat_flutter. See the comparison section above.

How do I request microphone permission?

The plugin does not prompt for you. Declare NSMicrophoneUsageDescription on iOS and RECORD_AUDIO on Android, then request the runtime permission in your app, for example with permission_handler, before calling initDevices(). Camera works the same way with NSCameraUsageDescription and CAMERA.

Can I reuse one client for several sessions?

Yes. release() frees the native resources and dispose() closes the Dart streams and the hardware notifier, so they are separate calls. Connect, disconnect and reconnect as often as you need on one instance. After release() you must call initialize() again before the next connect(). Call dispose() only when you are done with the Dart object for good.

Example app

The example/ app shows an endpoint field with connect and disconnect, a live transcript chat (user, bot and system messages), mic and camera pickers wired to hardwareState, and a registered get_weather function handler. It uses SmallWebRTCTransport. Switch to DailyTransport and DailyConnectParams if your server hosts a Daily room.

cd example
flutter pub get
flutter run

API reference

The full Dart API reference is at pub.dev/documentation/pipecat/latest. The main entry point is PipecatClient. Events are broadcast Streams, payloads use a sealed Value type instead of raw JSON, and hardwareState is a ValueNotifier<HardwareState> for device lists and mute flags. Errors are typed: PipecatException, PipecatConnectionException, PipecatFunctionCallException, PipecatBackendException and PipecatTransportMismatchException.

Lifecycle: initialize, initDevices, startBot, connect, startBotAndConnect, disconnect, disconnectBot, release, dispose, getState, getVersion.

Messaging:

• sendText, sendAction, setLogLevel
• sendClientMessage(msgType, data): fire-and-forget client message
• sendClientRequest(msgType, data): request that returns the bot's ClientMessageData reply
• PipecatClient.rtviProtocolVersion: static, the protocol version the plugin targets (2.0.0)
• nativeRtviProtocolVersion(): the version reported by the native SDK

Devices and audio:

• getAllMics, getAllCams, getAllSpeakers
• selectedMic, selectedCam, selectedSpeaker
• updateMic, updateCam, updateSpeaker
• enableMic, enableCam, isMicEnabled, isCamEnabled
• getTracks

Event streams:

• Connection: onConnected, onDisconnected, onTransportStateChanged
• Bot: onBotReady, onBotConnected, onBotDisconnected, onBotStarted
• Speech: onBotStartedSpeaking, onBotStoppedSpeaking, onUserStartedSpeaking, onUserStoppedSpeaking
• Text: onUserTranscript, onBotTranscript, onBotLlmText, onBotTtsText, onBotOutput
• LLM and TTS lifecycle: onBotLlmStarted/Stopped, onBotTtsStarted/Stopped
• Participants: onParticipantJoined/Left/Updated
• Tracks: onTracksUpdated, onTrackStarted/Stopped
• Screen share: onScreenTrackStarted/Stopped, onScreenShareError (never fire: the native SDKs do not expose screen share)
• Inputs: onInputsUpdated
• Levels: onLocalAudioLevel, onRemoteAudioLevel
• Messages: onServerMessage, onMessageError
• Device and mute: onDeviceError, onUserMuteStarted, onUserMuteStopped
• Protocol: onProtocolMismatch
• Metrics: onMetrics
• Search: onBotLLMSearchResponse
• Errors: onError, onBackendError, onServerError, onErrorResponse

Platform caveats for events:

• Never fires on Android: onParticipantUpdated, onTrackStarted/onTrackStopped (and the screen-track callbacks), onBotStarted, onMessageError. Selected mic/cam/speaker and available-speaker updates to hardwareState also never arrive on Android (the Android SDK has no callback for them). onInputsUpdated and onTracksUpdated do fire on Android.
• Never fires on iOS: onParticipantUpdated, onInputsUpdated.
• onError carries RTVI error envelopes (PipecatBackendException) and function-call failures (PipecatFunctionCallException). Native generic SDK errors are not forwarded on Android or iOS.
• Track enabled is always true on both platforms.
• onScreenTrackStarted/Stopped and onScreenShareError never fire on either platform.

Learn more

• How we built pluggable voice AI transports in Flutter with Pipecat (Medium, Part 1)
• Medium profile, for Part 2 and later articles
• Pipecat client docs
• API reference on pub.dev
• Changelog
• llms.txt: https://minhtri1401.github.io/pipecat_flutter/llms.txt

Known limitations

These are the gaps in 0.3.0 that still apply after the RTVI 2.0 upgrade. Platform gaps in the feature table above are not repeated here. Developed and tested with Flutter 3.47.5. Flutter 3.41.0 with Xcode 27 fails flutter build ios --simulator (lipo); use 3.47.5 or xcodebuild.

• Null in Value payloads on iOS. The iOS SDK Value has no null: a top-level null becomes {}, and null object values and array elements are dropped. Android keeps them.
• Custom aggregated_by values. Not decoded on iOS (upstream decoder limit). Android decodes them.
• SmallWebRTC startBot then connect. headers and requestData are dropped on that path, on both platforms. startBotAndConnect has no ICE option.
• onBotLLMSearchResponse. Mapped best-effort from the native payload, so some grounding data is lost.
• Screen share. Not available on either platform.
• Speaker selection and setLogLevel on Android. Unsupported and a no-op.

Contributing

PRs are welcome: bug reports, platform-parity fixes and docs improvements. Run fvm flutter analyze and fvm flutter test before opening one. If you change the Pigeon interface, regenerate with:

fvm dart run pigeon --input pigeons/pipecat_api.dart

Run the native tests too (see CLAUDE.md):

(cd example/android && ./gradlew :pipecat:testDebugUnitTest)
(cd example && xcodebuild test -workspace ios/Runner.xcworkspace -scheme Runner \
  -destination 'platform=iOS Simulator,name=iPhone 17' -derivedDataPath /tmp/pipecat-dd)

License

BSD 2-Clause. See LICENSE.