voltkit_rtc
Flutter SDK for VoltKit RTC: realtime audio and video rooms, chat, and live broadcast publishing.
| Rooms | StreamClient — join, publish camera and mic, render everyone else, moderate |
| Chat | on the room's own connection, or ChatClient for chat with no media |
| Broadcast | BroadcastPublisher — go live to an audience of any size over WHIP |
Your backend mints a short-lived token and hands the app { token, wsUrl }.
This package turns that into a live session. The API secret never reaches the
device.
flutter pub add voltkit_rtc
It builds on flutter_webrtc 1.6
and works with current Flutter on Android (minSdk 23) and iOS. There is nothing
else to install or patch.
Coming from the unpublished stream_client package?
Only the package name changed. Class names, methods and events are the same.
dependencies:
voltkit_rtc: ^0.2.0 # was: stream_client: { path: ... }
import 'package:voltkit_rtc/voltkit_rtc.dart'; // was: package:stream_client/stream_client.dart
Quick start
import 'package:flutter_webrtc/flutter_webrtc.dart';
import 'package:voltkit_rtc/voltkit_rtc.dart';
final client = StreamClient(wsUrl: wsUrl, token: token);
// Wire the events BEFORE connecting: connect() consumes everyone who is
// already publishing, and those tracks arrive during the call.
final renderer = RTCVideoRenderer();
await renderer.initialize();
client.onTrack.listen((t) {
if (t.kind == 'video') renderer.srcObject = t.stream;
});
client.onTrackEnded.listen((e) => renderer.srcObject = null);
client.onClosed.listen((reason) => Navigator.pop(context));
final join = await client.connect();
if (join.role.canPublish) {
await client.publish(); // camera + mic, simulcast
}
Always await client.close() when leaving the screen. A client whose reference
is dropped without closing keeps its reconnect loop running. close() never
waits for the server: the session ends at once and the server releases what it
held when the socket goes. However a session ends (your close(), kicked, the
room closed, replaced, reconnecting given up), the SDK has stopped the camera,
microphone and screen capture it opened, closed its producers and transports,
and stopped the screen foreground service by the time onClosed fires (at most
5 s later if a platform call hangs). Streams you passed in keep running.
Permissions
flutter_webrtc needs camera and microphone permission before publish().
Android — android/app/src/main/AndroidManifest.xml:
<uses-permission android:name="android.permission.CAMERA" />
<uses-permission android:name="android.permission.RECORD_AUDIO" />
<uses-permission android:name="android.permission.INTERNET" />
<uses-permission android:name="android.permission.MODIFY_AUDIO_SETTINGS" />
Minimum SDK 23, and android:usesCleartextTraffic is not required — signaling
is wss and media is DTLS-SRTP.
iOS — ios/Runner/Info.plist:
<key>NSCameraUsageDescription</key>
<string>Used to broadcast video when you go live.</string>
<key>NSMicrophoneUsageDescription</key>
<string>Used to broadcast audio when you go live.</string>
Add audio to UIBackgroundModes if a broadcast should survive the app going
to background.
Roles
The role is decided by your backend and signed into the token; the client cannot change it, and the media node enforces it.
| Role | canPublish |
canModerate |
|---|---|---|
host |
yes | yes |
cohost |
yes | no |
moderator |
no | yes |
viewer |
no | no |
if (client.role.canPublish) await client.publish();
Events
All broadcast streams, safe to listen to more than once.
| Stream | Fires when |
|---|---|
onScreenShareStarted |
A screen is published; carries its local MediaStream. |
onScreenShareEnded |
Screen publishing ended (explicitly or when the platform reports the track ended). |
onScreenSharePaused |
The iOS ReplayKit preview paused/resumed capture; carries true/false. |
onTrack |
someone's audio or video is ready to render |
onTrackEnded |
that track went away |
onParticipantJoined / onParticipantLeft |
roster changes |
onParticipantMuted |
moderation locks changed: {identity, kind, paused}; roster muted / videoBlocked seed locks on reconnect |
onRoleChanged |
someone was promoted or demoted |
onHandRaised |
a hand went up or down; reason is declined, withdrawn or granted |
onForceMuted |
a moderator switched your mic or camera off, or released it |
onStreamInterrupted |
media stopped but the connection is fine — the publisher is reconnecting |
onSubscribeError |
one remote track could not be subscribed to |
onTrackMuted |
someone muted their mic, turned their music off or covered their camera, or un-did it; source says which |
onClaimChanged |
an exclusive slot was taken, moved, freed, locked, or its holder went away |
onParticipantUpdated |
someone's display metadata changed mid-session |
onRoomMetadataChanged |
room-scoped metadata changed — a board's seat format, say |
onPublishWarning |
publishing worked, but something affecting quality did not apply |
onMessage |
a chat message arrived — yours included, so render from this stream |
onMessageDeleted |
a message was removed, by its author or a moderator |
onChatMuted |
someone was muted or unmuted in chat; check identity against your own |
onTyping |
someone else started or stopped typing |
onUsageLimit |
the account behind the session ran out of minutes or hit a usage limit; the session ends at endsAt unless topped up |
onWalletLow |
wallet billing: the funds cover less than the grace period (remainingSeconds), or a recharge lifted it (cleared) |
onRoomClosed |
the room was ended, with the reason: quota-exhausted when the funds ran out. onClosed follows |
onKicked |
a moderator removed you (by, reason). onClosed follows |
onMustReconnect |
the server is moving this client (a promotion, say); handled for you, this says why |
onConnectionStateChanged |
connecting, connected, reconnecting, closed (see Session state) |
onTokenExpiring |
the token you would reconnect with expires in a minute and nobody refreshed it: call updateToken |
onNetworkQuality |
about every 3 s while listened to: level, rttMs, lossPct, availableKbps |
onMediaWarning |
0.8.0: a problem with your camera or microphone starts or clears (MediaWarning: micSilent, cameraFpsLow, cameraInterrupted, micInterrupted) |
onAudioLevels |
0.8.0: every participant's sound about twice a second while listened to (AudioLevel) |
onActiveSpeakers |
0.8.0: who is speaking, most active first, when that changes (microphones only) |
onPublishFallback |
0.8.0: the opt-in setPublishFallback(true) paused your camera because your upload could not carry the sound (true), or brought it back (false) |
onLocalStreamChanged |
0.9.0: the stream you publish changed (a camera switched by id, or devices captured again after a reconnect): set your preview renderer's srcObject again |
onDevicesChanged |
0.8.0: a camera, microphone or audio output was plugged in or removed: devices, added, removed. Watched only while listened to |
onStreamFallback |
0.8.0: a remote video is held back for this viewer's connection (fallback: true) or coming back; reason bandwidth (the media server cut it) or audioProtection (the SDK paused remote cameras to save breaking-up audio). See Poor connections |
onMediaReconnecting |
0.8.0: {transportId, reconnecting}; ICE recovery starts (true) or finishes/is discarded (false). Use onConnectionStateChanged for full reconnects |
onError |
warnings and refusals as RpcErrors with registry codes: what onPublishWarning says, plus replaced and reconnect-gave-up |
onClosed |
terminal; nothing follows. The SDK has released the media it opened by then |
Moderation
Moderator-only, and enforced at the SFU rather than asked of the target's client — the packets stop at the server, so nothing the target does can override it. Changing a role and ending the room are the host's alone.
await client.forceMute(identity); // mic off
await client.forceMute(identity, kind: 'video'); // camera off
await client.forceMute(identity, kind: 'video', paused: false); // give it back
await client.promote(identity, Role.cohost); // host only; anyone may step down to viewer
await client.kick(identity);
await client.endRoom(); // host only
endRoom() resolves once the room has ended: on the server's reply, or when
the server closes the session with roomClosed while the call waits (it can
do that before its reply arrives). A refusal (forbidden), a dropped socket or
no answer within 15 seconds (timeout) is still an error.
forceMute replaces requestMute, which still works and means
forceMute(identity, kind: 'audio').
The target cannot switch a force-muted kind back on — the server refuses it with
force-muted, and a setMicEnabled(true) / setCamEnabled(true) that was
already waiting for the server when a moderator decided fails with
force-muted too and leaves it off —
so render onForceMuted as a locked button, not a disabled feature. Its
kind says which, and paused: false means control was handed back. It is
about the camera and the microphone only: a screen share and music, which have
their own switches, keep going (0.9.4; a video force-mute used to black out
the screen share, and an audio one silence the music, for good). After a
release, switching the camera or microphone back on is the user's: the SDK
tells the server it is still off (the server's release resumes everything of
that kind), so viewers are not shown a camera that sends nothing. Leaving
the button live means the user taps it, nothing happens, and the app looks
broken rather than moderated.
The rest of the room needs no extra work: the result arrives on onTrackMuted
like any other pause.
Exclusive slots (seats on a board)
A slot is a named thing in a room that at most one participant may hold. Seats on a 20-seat audio board or a 9-seat video board are the motivating case, but the service never learns that — a key is an opaque string.
await client.claim('seat-13'); // take it, or throw slot-taken
await client.claim('seat-13', identity: 'guest-1'); // moderator, on someone's behalf
await client.release('seat-13');
await client.blockSlot('seat-20'); // lock it; moderator only
await client.unblockSlot('seat-20');
await client.setMetadata({'displayName': 'Priya'});
// room-scoped, moderator-only, persisted — not just broadcast
await client.setRoomMetadata({'seatFormat': 10},
requireFree: ['seat-11', 'seat-12']); // refuse if those are taken
setRoomMetadata merges one key deep (a null value deletes a key; replace: true swaps the whole object), and persists — a client joining afterwards
reads the new value from JoinResult.metadata. A broadcast-only change would
be right for whoever was present and wrong for everyone who arrived later,
which is invisible to the person who made it.
requireFree is checked and applied together, so nothing can be claimed in
between. Use it when a change invalidates slots — shrinking a board from twenty
seats to ten — and the refusal names them:
try {
await client.setRoomMetadata({'seatFormat': 10},
requireFree: [for (var i = 11; i <= 20; i++) 'seat-$i']);
} on RpcError catch (e) {
if (e.code == 'slot-occupied') {
final taken = (e.data?['slots'] as List?)?.cast<String>() ?? [];
// "Clear seats 13 and 17 first."
}
}
The service cannot work those slots out for you: metadata is opaque to it, so twenty-seats-becomes-ten is only a different blob. Naming them keeps that knowledge in your app while the check stays atomic.
join() seeds the board: result.claims, result.blockedSlots, and
result.metadata (room-scoped, set by your backend at room creation — a seat
format, say). getRoomState returns the same after a reconnect.
onClaimChanged carries a state, and away is not free.
| State | Draw it as |
|---|---|
held |
taken |
away |
taken, greyed — the holder's socket dropped and the slot is still theirs |
free |
empty |
blocked |
locked |
Rendering away as free shows a viewer their own seat apparently empty and
then refuses to let them retake it. It is not a rare state: every guest granted
a seat while on a relay node passes through it, because that promotion moves
them to the origin and closes their socket on the way. A claim is held for
about a minute after a disconnect and only the same identity can reclaim it.
Rules worth knowing: one claim per identity — claiming a second releases the
first, which is what moving seats means. blockSlot refuses an occupied
slot (slot-occupied) rather than evicting; remove, then lock. Any
moderator can release any slot, so a seat stranded by a half-finished demote
is recoverable by anyone.
Claims and roles are deliberately orthogonal: demoting someone does not free their slot, and your app should release it.
Hands and layers:
await client.raiseHand(metadata: {'slot': 'seat-13'}); // which seat you want
await client.lowerHand(); // withdraw
await client.lowerHand(identity: 'priya'); // DECLINE — moderators only
// the host reads it back off the event:
client.onHandRaised.listen((h) {
if (h.raised) print('${h.identity} wants ${h.slot}');
else if (h.wasDeclined) print('declined by ${h.by}'); // clear the countdown
else if (h.wasGranted) print('seated'); // drop it from the queue
});
await client.lowerHand();
await client.setPreferredLayers(track.consumerId, 0); // lowest rung
setPreferredLayers overrides the SFU's own choice for one subscription. You
rarely want it — the SFU picks per viewer from their available bandwidth — but
it is the lever for "this tile is a thumbnail, don't send it 720p". Pass
RemoteTrack.consumerId, which arrives on onTrack.
PK battles: two rooms at once
One identity can hold two rooms simultaneously — session replacement is scoped by room, so the second connection does not displace the first. Each room keeps its own roster, producers and events; nothing merges. No server-side PK feature is involved.
See example/pk_battle.dart for the full pattern,
including dropping a simulcast layer because each tile is half width. That file
is compiled by flutter analyze, so unlike a snippet it cannot quietly rot.
onTrackMuted is not optional to render. A paused video producer stops sending
frames and the renderer keeps showing the last one it decoded, so a host who
covers their camera becomes a frozen still of themselves — indefinitely, and
indistinguishable from the stream having hung. There is no frame-level signal on
RTCVideoRenderer to infer this from (onResize fires on a size change, not a
frame), so this event is the only way to know. It also fires on join for anyone
already muted, so a late viewer is not left looking at a black tile with no
explanation.
onSubscribeError is per producer, and the join still succeeds around it. Every
producer is subscribed to independently, so one failure costs you that one
track, not the rest of the room. A video that fails gives its slot under the
project's video limit to the next one waiting, at once. A producer that closes
while it is being subscribed to is dropped quietly (no onTrack, no error). Listening is optional — show the affected
participant as unavailable, or retry — but ignoring it means a missing tile with
no explanation.
Publish quality
publishStream() defaults to a broadcast profile, which wants the opposite
trade-offs from a video call:
await client.publishStream(stream); // broadcast defaults
await client.publishStream(stream, video: const VideoPublishOptions(
degradationPreference: RTCDegradationPreference.MAINTAIN_FRAMERATE,
layers: 3, // call-like profile
));
| Default | Why |
|---|---|
preferHardwareCodec: true |
H264 when the device has it. Phones have hardware H264 encoders and essentially none has one for VP8 — this is the single biggest lever. |
degradationPreference: MAINTAIN_RESOLUTION |
Under CPU load, drop framerate rather than resolution. WebRTC defaults to the reverse, which is right for a call and wrong when viewers are reading text. |
layers: 2 |
Each layer is a full encode. Three software encodes on a mid-range phone force the capture itself down a rung before anything reaches the wire. |
maxFramerate: 30 |
Layers budget for this whether or not the camera reaches it; unset, each one plans for 60 and spends bitrate on frames instead of pixels. |
quality: null |
The highest quality the project allows (see below). |
topBitrate: null |
6 Mbps at 1080p, 3 Mbps at 720p, 1.2 Mbps at 480p (30 fps). |
Video quality: 1080p, 720p, 480p
await client.publish(video: const VideoPublishOptions(quality: VideoQuality.p1080));
client.videoPolicy.maxQuality; // what this project may publish: p720 or p1080
client.videoQuality; // what is being published, after that limit
- 1080p, 720p and 480p are included on every plan. Up to 60 fps.
videoPolicy.maxQualityandmaxFrameRatedescribe any administrator restriction for this project; defaults are 1080p and 60 fps. - Unset
qualitychooses the highest allowed. Captured pictures are scaled down to fit, never upscaled. Portrait camera frames keep their orientation. - The default is 30 fps. Opt in with
frameRate: 60(the previousmaxFramerateoption remains supported). Only the top simulcast layer runs at 60; lower layers stay at 30. A 60 fps top layer uses 1.5× its 30 fps bitrate: 9 Mbps at 1080p, 4.5 Mbps at 720p, 1.8 Mbps at 480p. - 60 fps roughly doubles encoding work and increases heat, battery use and
data. Many phone front cameras provide only 30 fps. Capture constraints and
sustained outbound statistics trigger a 30 fps fallback, reported as
frame-rate-fallbackthroughonPublishWarning. Native platforms without usable statistics still get WebRTC's own adaptive encoding. - 60 fps is delivered only when the capture source provides it.
Phone makers usually keep 60 fps for their own camera app, so browsers and
most apps get 30 fps. Check a camera's supported rates in
chrome://media-internals→ Video Capture. Billing follows the frame rate actually delivered. In our tests a phone whose own camera app records 1080p60 gave Chrome 25–30 fps (14 fps in a dim room).getVideoStats()shows the rate actually sent. - Every publisher normally has a 25 Mbps incoming safety cap, covering camera, screen and audio together. It is an anti-abuse limit, not a plan quality gate.
At 30 fps, the default two-layer phone profile sends:
| Quality | Copies | Bitrates | Host upload, excluding audio |
|---|---|---|---|
| 1080p | 960×540 · 1920×1080 | 2 Mbps · 6 Mbps | about 8 Mbps |
| 720p | 640×360 · 1280×720 | 800 kbps · 3 Mbps | about 3.8 Mbps |
| 480p | 427×240 · 854×480 | 400 kbps · 1.2 Mbps | about 1.6 Mbps |
With layers: 3, 720p and 1080p add a quarter-size layer: 320x180 at
200 kbps, or 480x270 at 400 kbps. A soft picture usually comes from the phone
(too busy to encode) or its upload (too slow for the top layer):
troubleshooting shows how to tell
which, and what to do about each.
The full guide, with data use and troubleshooting: voltkit.in/docs/video-quality.
publish() forwards these too — publish(video: const VideoPublishOptions(...))
— and the profile is remembered, so a reconnect republishes with the same
one. Without that a host failing over to a new origin silently reverts to the
defaults, which reads as "the setting did not work" rather than "the setting
stopped being applied".
Audio quality: standard or HD
The microphone goes out as standard audio unless you ask for HD:
AudioQuality |
Sent as | Billed as (wallet billing) |
|---|---|---|
std (default) |
mono Opus, maxaveragebitrate 64000, the sender held at 60 kbps, DTX and FEC |
standard audio |
hd |
stereo Opus, maxaveragebitrate 128000, DTX and FEC |
HD audio |
await client.publish(audioQuality: AudioQuality.hd);
await client.setAudioQuality(AudioQuality.std); // while live; resolves to the quality in use
client.audioQuality; // AudioQuality.std or .hd
The server bills what it can verify, never what a client claims: HD only when
the microphone declares more than 64 kbps or is measured above 72 kbps, so a
standard microphone really stays at or under 64 kbps (its sender is held at
60 kbps because Opus overshoots its target a little). Both qualities declare
themselves to the server. Switching re-creates the microphone's producer (its
codec settings are fixed per producer): listeners hear a gap of a fraction of a
second, and a muted microphone stays muted. Music (publishMusic) is a
separate 128 kbps stereo track and is not affected: it is HD and free whatever
the microphone's quality (see Music).
Changing quality while live
await client.setVideoQuality(VideoQuality.p720); // resolves to the quality used
await client.setVideoFrameRate(60); // 30 or 60; resolves to the rate used
client.videoQuality; // after the project's limits
client.videoFrameRate; // 30 or 60
final stats = await client.getVideoStats(); // per layer: size, fps, bytes, limitation
Each replaces the camera's producer, so viewers see a brief pause while the new
one starts; a camera that is off stays off, and the screen, microphone and music
are untouched. The picture is scaled down to a quality, never up, so capture at
least that size: publish() captures the quality it publishes. A camera
captured at 30 fps cannot be sent at 60 (frame-rate-fallback): publish with
frameRate: 60 to capture at 60 first. getVideoStats() is null while no
camera is published.
Stopping
unpublish() releases the camera and microphone as well as closing the
producers. It has to: producers are created with stopTracks: false so a
reconnect can republish the same tracks, which means nothing else would let the
device go. A co-host who steps down and then takes a seat again would otherwise
ask Android for a camera the process still holds, and get a track that never
produces a frame — signalling all succeeds and the tile stays black.
A stream you passed to publishStream is yours and is left exactly as it was:
unpublish(), close() and a failed publish close the SDK's producers and never
stop or dispose its tracks, so a preview or another consumer keeps working. One
publish() acquired is stopped and disposed, once, also after a reconnect
published it again — every one of them, if you published more than once. A
publish that fails before anything went on air keeps nothing it captured, and a
capture that failed is never asked for again by a reconnect.
unpublish() releases the devices at once, without waiting for the server. The
server is told to close the producers and given 5 seconds to confirm, so a
server that stops answering cannot hold it up, and the session stays usable for
the next publish. A quality change still queued behind it fails with
unpublished.
It also drops the send transport, so the next publish() builds a fresh
one. A transport whose producers are closed and whose tracks are stopped does
not reliably carry media again — and the failure is invisible from the outside:
produce succeeds over the websocket, the server creates the producer and
announces it, viewers consume and resume it, and no RTP ever arrives. Perfect
state, dead pipe. Rebuilding costs a transport setup on a second promotion and
removes a class of bug that reads as everything-but-the-video working.
setCamEnabled(false) is a different thing: it pauses the producer and keeps
the device, so unmuting is instant.
To bisect a codec-specific problem, force one outright:
await client.publishStream(stream,
video: const VideoPublishOptions(codecMimeType: 'video/VP8'));
preferHardwareCodec: false is not the way to do that — it hands the
choice back to the library's preference order rather than selecting VP8, so the
variable you meant to isolate is still being chosen for you. codecMimeType
forces one, and reports on onPublishWarning when the device does not
advertise it, rather than quietly publishing with something else and inverting
the conclusion of your test.
The symptom when these are wrong is specific and easy to misread as bandwidth: the
host log shows a full ladder at capture, then ReconfigureEncoder drops every
rung the moment a viewer joins and all the encoders actually run. Raising the
capture resolution makes it worse — it gives the encoder more to throw away.
Check is_hardware_accelerated in the host log before suspecting the network.
onStreamInterrupted is not a disconnect. Show a spinner and wait — tearing the
session down here turns a two-second blip into a lost broadcast.
onClosed carries why:
| Reason | Meaning |
|---|---|
replaced |
the same identity opened a newer session elsewhere |
kicked |
removed by a moderator |
roomClosed |
the host or the service ended it (onRoomClosed says why: quota-exhausted when the funds ran out); also a rejoin refused because the room ended (0.9.4) |
gaveUp |
reconnection was attempted and abandoned |
left |
you called close() |
Project media settings
What a project may publish comes with the join: client.mediaPermissions
(audio and video on or off, the largest video quality, whether 60 fps is
allowed and up to which size, whether HD audio is allowed, how many videos one
participant may receive) and client.billingMode (BillingMode.legacy minutes
or the VoltKit wallet). Offer only these in your UI. The server enforces them
whatever a client sends; the SDK follows them so it never sends what would be
refused, and reports what it changed with the server's own code:
| Request | SDK behaviour | Code |
|---|---|---|
a kind the project switched off (publish/publishStream with such a track or constraint, publishScreen without video, publishMusic without audio) |
refused before any device opens or anything is produced | media-disabled |
setAudioQuality(AudioQuality.hd) without HD |
refused | audio-quality-not-allowed |
HD audio in publish without HD |
sent as standard, warning | audio-quality-not-allowed |
| 60 fps where no 60 fps mode is allowed | sent at 30, warning | frame-rate-not-allowed |
| a 60 fps size that is not enabled | the largest enabled 60 fps size, warning | video-quality-not-allowed |
| a 30 fps quality above the maximum | capped silently (the server does the same) | – |
| more videos than the project lets one participant receive (wallet billing) | the next waits until one ends, warning | video-subscription-limit |
| the camera or device cannot keep 60 fps | sent at 30, warning | frame-rate-fallback |
Warnings arrive on onError as RpcErrors and on onPublishWarning as
'<code>: message'. A project's media modes narrow wallet sessions only;
sessions billed in minutes keep every mode their settings allow. Older servers
send no media settings, and the project's video policy (videoPolicy) applies.
Publishing
await client.publish(); // sensible defaults
await client.publish(constraints: {...}); // or your own
await client.publishStream(existingStream); // or a stream you already have
await client.setMicEnabled(false);
await client.setCamEnabled(false);
await client.unpublish();
Calling publish() again adds to what is published (the microphone first, the
camera later, say); unpublish(), close() and a reconnect deal with every
capture, and localStream is the newest. A second capture of a source already
on air is refused and released; the first stays. Mic and camera switches run
one at a time, in turn with publishing and quality changes; one still waiting
when unpublish() or close() runs completes without doing more.
Video is published as simulcast layers (by default 360p and 720p; see Video quality). The server picks one per viewer from their available bandwidth — you do not manage that. At 720p and 480p viewers start on the top layer and move down only if their connection cannot carry it.
Moderation
await client.raiseHand();
await client.requestMute(identity); // moderating roles
await client.promote(identity, Role.cohost);
await client.kick(identity, 'spam');
await client.endRoom(); // host only
Promotion makes the promoted client reconnect. Publishers live on the origin node so their producers stay local, and a viewer may be on a relay. The SDK handles this: it receives the new role's token, reconnects, and rebuilds media. Your UI should expect a brief interruption for that participant, not an error.
Chat
Every room has a chat channel of the same name. StreamClient carries it on the
connection it already holds, so there is nothing more to connect:
client.onMessage.listen((m) => setState(() => messages.add(m)));
client.onMessageDeleted.listen((e) => setState(() => messages.removeWhere((m) => m.id == e.id)));
await client.sendMessage('hello everyone');
await client.sendMessage('psst', to: 'user-42'); // direct: only the two of you see it
await client.sendMessage('', type: 'reaction', // delivered, not stored
metadata: {'emoji': 'heart'}, ephemeral: true);
final older = await client.getMessages(limit: 50, before: messages.first.id); // page backwards
await client.setTyping(true); // safe on every keystroke
final state = await client.getChatState(); // members, and whether you are muted
Your own message comes back on onMessage like everyone else's. Pass a
clientMsgId to match it to one you rendered optimistically.
text is delivered exactly as it was sent. Render it with Text, never as
HTML or markup.
Moderators (the host and moderators; not co-hosts) can also do this:
await client.deleteMessage(messageId); // anyone's; everyone else, only their own
await client.muteChat('user-42'); // survives that user reconnecting
await client.muteChat('user-42', muted: false);
A muted sender gets RpcError with code chat-muted; a client sending too fast
gets rate-limited. Both are worth showing to the user rather than retrying.
Chat without media
An audience watching a broadcast has no reason to join a room. ChatClient
opens a connection that carries chat and nothing else, into the same
conversation:
// { token, wsUrl } from your backend: POST /v1/chat/{channel}/tokens
final chat = ChatClient(wsUrl: wsUrl, token: token);
chat.onMessage.listen(addMessage);
chat.onClosed.listen((reason) => showEnded(reason));
final joined = await chat.connect();
messages = joined.messages; // recent history, oldest first
await chat.sendMessage('first!');
// ...
await chat.close();
It has the same methods and events as the chat half of StreamClient, plus
onReconnected — the moment to call getMessages() and fill any gap. By
then role and joinResult already hold the rejoin's answer (a role or mute
that changed while the socket was away); its history is not replayed.
Music
A host can play music to the room without it going through the microphone. It
is published as a track of its own, labelled music, so the audience hears it
clean and everybody keeps separate control of it.
Hearing it (every platform). A music track arrives on onTrack like any
other, with source == 'music':
client.onTrack.listen((t) {
if (t.isMusic) {
musicTrack = t.track; // plays by itself, like every remote sound
} else if (t.kind == 'video') {
videoRenderer.srcObject = t.stream;
}
});
// Its own volume for this listener (flutter_webrtc), leaving the host's voice:
await Helper.setVolume(0.3, musicTrack);
// Or "mute the music" for this listener only:
musicTrack.enabled = false;
client.onTrackMuted.listen((m) {
if (m.source == 'music') setState(() => musicPlaying = !m.paused);
});
Playing it. Hand the SDK the audio as a MediaStream:
await client.publishMusic(musicStream);
await client.setMusicEnabled(false); // music off for the audience, mic untouched
await client.setMicEnabled(false); // mic off, music keeps playing
await client.stopMusic();
setMicEnabled and setMusicEnabled are independent, in both directions.
stopMusic() also cancels a publishMusic() still being set up: that call
fails with unpublished and nothing goes on air afterwards. A second
publishMusic() replaces the first, whether it is on air or still starting
(a start it replaces fails with unpublished). Your music stream is never
stopped.
Music is HD and free: it always goes out at 128 kbps stereo, also in a project limited to standard audio, and the service never bills it. Sending or hearing music never changes anyone's rate; the microphone and video alone decide it.
Where musicStream comes from depends on the platform, and this is the part
to plan for:
| Platform | In-app music as a stream |
|---|---|
| Android, iOS | Not with the WebRTC plugin this package builds on (flutter_webrtc). It can capture the microphone and the screen, but it does not expose audio your app is playing as a track. |
| Web | This package does not build for the web (it uses dart:io). A web app uses the browser SDK, @voltkit-rtc/client, which publishes music from an audio element. |
On Android and iOS you therefore need a native audio source to get a clean music track — a platform channel that feeds decoded audio into WebRTC, or a plugin that does. Playing the song through the phone speaker and letting the microphone pick it up is the workaround to avoid: the echo canceller exists to remove exactly that sound. Until your app has such a source, mobile hosts can still receive music published from a web host, with the separate controls above.
Broadcast
A broadcast is one publisher and an audience of any size, delivered over LL-HLS and WebRTC by a CDN-friendly media server rather than by a room. Use it when the audience watches and does not talk back.
// stream.publish.whipUrl from your backend: POST /v1/streams
final publisher = BroadcastPublisher(whipUrl: whipUrl);
publisher.onState.listen((s) => setState(() => state = s)); // connecting, live, interrupted, failed
publisher.onWarning.listen(debugPrint);
await publisher.start(); // camera + mic; or start(stream: yours)
preview.srcObject = publisher.localStream;
publisher.setMicEnabled(false);
publisher.setCamEnabled(false);
await publisher.stop(); // off air, camera released
The whipUrl carries the publish token. Treat it like a password: fetch it
when the user taps "go live", and do not log it.
start() throws WhipError if the media server refuses. isRefused means the
URL itself was not accepted — expired, for another stream, or the account is
out of minutes — and a retry with the same URL will be refused again. Any other
failure of the exchange (the server closing the connection without answering,
an answer that is not text) is a WhipError with status 0 too, and no error
names the URL (0.9.4).
If the connection fails once live (failed), the publish is over and
released: the connection, a camera and microphone it captured, and the media
server's session. Call start() again (0.9.4; it used to throw StateError
until stop()).
stop() wins over a start() still in flight (during the camera prompt, or
while the media server answers). That start() throws RpcError with code
unpublished and nothing goes on air: the camera, connection and media server
session it acquired are released, even ones that arrive after stop()
returned. One publish at a time: start() while another is in flight or live
throws StateError. A stopped publisher can start again; after dispose(),
create a new one.
Video is offered as H.264 only. The media server repackages H.264 into HLS
untouched; any other codec makes it transcode every frame. A device with no
H.264 encoder still publishes, and onWarning says so.
Viewers need nothing from this package: play the stream's llhls URL with any
HLS player (video_player handles it on iOS and Android).
BroadcastPublisher's HTTP exchange is tested here against a real server. The media half runs only on a device — check it on your own hardware against your deployment before you ship.
Session state, tokens and network quality
client.connectionState; // SessionState.idle ... closed
client.onConnectionStateChanged.listen((c) => print('${c.previous.name} -> ${c.state.name}'));
client.onTokenExpiring.listen((_) async {
final expiresAt = await client.updateToken(await myBackend.mintRoomToken());
});
client.onNetworkQuality.listen((q) => showBars(q.level)); // excellent, good, poor, bad
client.onWalletLow.listen((w) => w.cleared ? hideRecharge() : showRecharge(w.remainingSeconds));
client.onError.listen((e) => log('${e.code}: ${e.message}'));
-
State.
SessionStatehas the five states every VoltKit SDK shares:idle(created),connecting,connected,reconnecting(the socket dropped and is being restored),closed(onClosedsays why). NamedSessionStateso it never clashes with Flutter's ownConnectionState. -
Tokens. The server refreshes the token it handed out while the account may be used.
updateToken(token)hands over one your backend minted for the same room and identity: on a live connection the server checks it like a join token (token-mismatchfor another room or identity,bad-tokenfor a bad one; the session keeps its token either way), otherwise it is kept for the next join. Calls run one at a time in call order, so the last token you hand over is the one kept. A token the server installs while a call waits (its own refresh, or a promotion's, which carries the new role) is newer: that call installs nothing and returns the expiry of the server's token.onTokenExpiringfires a minute before the held token expires when nobody refreshed it. -
Network quality comes from the media transports' statistics about every 3 s, and only while something listens: the worse round trip, packet loss over the interval in the worse direction, and the platform's bandwidth estimate.
Level Round trip below and loss below excellent150 ms 1 % good300 ms 3 % poor500 ms 8 % badotherwise An unknown value does not lower the level; bandwidth is reported, not graded.
-
Wallet. With wallet billing,
onWalletLowfires while the funds the project may use cover less than the grace period at the current burn, and again withcleared: trueonce a recharge or a lower burn lifts it (remainingSecondsis -1 when nothing limits the funds). At zero the room closes:onRoomClosedwithquota-exhausted.onUsageLimitstill covers minutes and usage caps. -
Errors. Every code an SDK can report is in one registry shared by every VoltKit SDK:
errorCodesmaps each code toErrorInfo(origin,terminal,retryable), andRpcError.infoandRpcError.isTerminalread it. Switch oncode, never on the message. A code this release does not know is passed through unchanged and is never terminal.
Reconnection
Handled for you, and worth knowing what it does:
- exponential backoff with jitter, one chain at a time
- each attempt is bounded: the socket must open, and
joinbe answered, within 15 s (StreamClient(joinTimeout: ...)); a stalled attempt is closed, its late reply ignored, and the next attempt follows, somaxReconnectAttemptsbounds the time spent reconnecting. The firstconnect()fails withtimeoutthe same way - reconnects with the refreshed token the server pushes over the live socket, not the one you started with — the original expires in ~10 minutes, and a broadcast outlasts it
- republishes automatically, so a host survives its origin node dying
- from 0.8.0, a media transport that fails or stays disconnected for 3 s first
restarts ICE on the same transport; recovery has a 10 s deadline including
the request. A timeout or a server without
restartIcerejoins instead.onMediaReconnectingreports each transport's recovery - from 0.8.0, sends
pingafter 15 s without a socket frame; no frame for 8 s after that triggers reconnect at the next 5 s check. Any frame proves liveness, including an older server'sunknown-method. Timers may run late when the OS suspends the app. Recovery never waits for the dead socket's close handshake to finish - gives up on terminal errors rather than looping: a revoked key or an ended room will refuse every attempt
- restores what you were publishing with the same video options and audio quality, and the receive quality you chose for each video
- after a reconnect,
role,participantsand the project's media settings come from the rejoin's own reply (a role changed while the socket was away applies even if the follow-upgetRoomStatefails; a demoted publisher is unpublished instead of republished); if you hold the token yourself, hand newer ones over withupdateTokenand the next reconnect uses them - stops for good on
close(), whatever is in flight: a connect, a join or a reconnect wait is abandoned and its socket closed, so a session closed while leaving a page cannot come back. A closed client stays closed:connect()fails withclosed, so create a newStreamClient - a session that ends on its own (refused for good, out of attempts, replaced,
room closed) closes its socket too; later calls fail with
closed
disconnected is not the end of a session. onClosed is.
Two rooms at once (PK battles)
Session replacement is scoped per room, so one identity can hold two rooms simultaneously — the shape of a PK battle, where two hosts each broadcast in their own room and viewers watch both.
final red = StreamClient(wsUrl: wsUrl, token: redToken);
final blue = StreamClient(wsUrl: wsUrl, token: blueToken);
await Future.wait([red.connect(), blue.connect()]);
// ... and close BOTH when the battle ends
Two clients means two sockets, two transport pairs and two ICE negotiations. On a mobile network that doubles the reconnect surface at the moment the audience is largest, so keep them independent: a reconnect on one must not disturb the other, and each must be closed before it is replaced.
Diagnosing "it just hangs"
Pass onWire and every frame is logged in both directions (a logger that
throws is ignored: it never changes a call's outcome):
final client = StreamClient(wsUrl: wsUrl, token: token, onWire: debugPrint);
-> #1 join
<- #1 join 6ms
-> #2 createTransport
<- #2 createTransport 8ms
-> #3 connectTransport
<- #3 connectTransport 12ms
-> #4 produce
<- #4 produce 9ms
This exists because two very different failures look identical from the app: the server never answered, or the client never asked. They point at opposite halves of the system, and guessing wastes days.
If publish() times out with produce-failed and there is no -> produce
line, the RPC never left the device. The request is assembled inside
mediasoup's _produce, which first awaits _handler.send(...) — the SDP and
peer-connection work — and only emits produce afterwards. A stall in that
handler means nothing is ever sent, and the server has no idea you wanted to
publish.
That handler runs inside a task queue which catches and prints exceptions
rather than surfacing them, so the underlying throw appears in the platform
log and nowhere else. Search it for a throw from unified_plan.run() or
send() before looking at the server.
If you do see -> produce with no <- line, that is genuinely server-side —
say so, because the orchestrator answers every produce, including invalid ones,
in about a millisecond.
Errors
Calls fail with an RpcError, with three exceptions: a camera or microphone
that cannot open (flutter_webrtc throws a String then), WhipError from
BroadcastPublisher (see Broadcast) and RecordingException from recording. Switch on code, never on the message;
e.info says whether trying again can help.
Future<void> enterRoom() async {
try {
final joined = await client.connect();
if (joined.role.canPublish) await client.publish();
} on RpcError catch (e) {
switch (e.code) {
case 'bad-token':
case 'no-token':
// Expired or wrong. A client that never joined can be given a new
// token and asked again.
await client.updateToken(await myBackend.mintRoomToken());
return enterRoom();
case 'no-room':
case 'room-ended':
showMessage('This room has ended.');
case 'quota-exhausted':
case 'daily-limit-reached':
case 'monthly-limit-reached':
showMessage('Calls are paused for this app right now.');
case 'forbidden':
case 'media-disabled':
showMessage('You can watch, but not go on air.');
default:
// timeout, connect-failed, no-capacity, ...: worth a Retry button,
// which calls connect() again.
if (e.info?.retryable ?? false) {
showRetry();
} else {
showMessage(e.message);
}
}
} catch (e) {
// Not an RpcError: flutter_webrtc could not open the camera or
// microphone, and says so in a String. A refused permission contains
// NotAllowedError: ask for it (see Permissions), then offer Retry.
showMessage('$e'.contains('NotAllowedError')
? 'Allow the camera and microphone to go on air.'
: 'The camera or microphone could not start.');
}
}
A real app would cap the token retry. Once a session has ended, it stays ended:
every call fails with closed, so create a new StreamClient.
While connected, nothing is thrown at you. Listen instead:
| Stream | What it means | What to do |
|---|---|---|
onClosed |
the session is over: left, kicked, roomClosed, replaced or gaveUp |
leave the screen, or offer to join again with a new client |
onRoomClosed |
the room was ended, with the reason (quota-exhausted when the funds ran out) |
say why; onClosed follows |
onKicked |
a moderator removed you | say so; old tokens cannot rejoin |
onError |
warnings with registry codes (frame-rate-fallback, codec-fallback, ...), plus replaced and reconnect-gave-up |
log them; none stops the session by itself |
onSubscribeError |
one remote track could not be received | show that participant as unavailable |
onUsageLimit, onWalletLow |
the account is running out of minutes or funds | show a top-up or recharge prompt |
onTokenExpiring |
the token for the next reconnect expires in a minute | updateToken with a new one |
Reconnecting is handled for you (see Reconnection): reconnecting in
onConnectionStateChanged is not an error, and onClosed is the end. Every
code, what it means and what to do: https://voltkit.in/docs/errors.
Help
Documentation: https://voltkit.in/docs/flutter-sdk. Questions and bug reports: contact@voltkit.in.
Testing
flutter test
The signaling layer has no Flutter or WebRTC imports, so it is tested against a
real HttpServer WebSocket — reconnects, close code 4001, token refresh and
terminal errors are all exercised without a device. Audio quality, the
project's media settings, the live quality switches, connection state, token
updates, wallet notices and the roster are tested the same way, with a fake
send transport; network quality and statistics through their pure parsers.
Chat and the WHIP exchange are tested the same way. The lifecycle races (a
close() landing during a connect, a join or a reconnect wait) and malformed
server frames are tested with fake sockets, so each race is hit on purpose.
Protocol parsing is tested against a captured live response rather than a hand-written fixture, because the failure mode of drift is a renamed field parsing as null and showing up as a black screen:
ORCH_URL=http://127.0.0.1:8080 API_KEY=sk_... ./tool/capture_fixtures.sh
On a real Android runtime, against a running stack — a browser hosts a room and an app built on this package joins it, publishes, subscribes and chats:
./tool/device_e2e/run.sh
How it is built
Rooms use mediasoup's client protocol. That client is
included in this package (lib/src/mediasoup), adapted from
mediasoup_client_flutter
0.8.5 under its MIT licence. The published 0.8.5 does not build with current
flutter_webrtc, and a package on pub.dev cannot depend on a fork, so the
fixed copy ships here. Every change is marked VoltKit patch in the source:
- builds against
flutter_webrtc1.6 /webrtc_interface1.5 (encoding priority and RTCP CNAME types); - the transport waits for its peer connection before the first publish or subscribe (it could race it and fail with a null-check error);
- simulcast layers named with RIDs such as
r0parse (they were read as ints); - an encoding without a scalability mode is one layer, not a crash;
- the answer uses the header extension IDs the offer actually assigned (current libwebrtc rejects renumbering).
None of these could be seen without a device; tool/device_e2e/run.sh is the
test that found them, and test/vendored_mediasoup_test.dart pins them.
Viewer quality menu (0.5.0)
await client.setReceiveQuality(track.producerId, '480p');
await client.setReceiveQuality(track.producerId, 'auto'); // recommended
Offer Auto (recommended) / 1080p / 720p / 480p. The helper selects the
highest actual layer fitting the tier, using RemoteProducer.videoLayers
(VideoLayer: spatialLayer, width, height). It works with two or three
layers, portrait cameras and screens. A 720p publisher cannot provide 1080p;
480p may select a 360p copy when that is its next available rung. An old
publisher without layout metadata reports quality-unavailable; automatic
reception remains available. From 0.9.2 the chosen layer is what the viewer
gets while the video is shown, whatever size setVideoView reports (up to 0.9.1
it was only a ceiling). A call that fails leaves the previous choice in place
(0.9.3). Both this helper and setPreferredLayers remember
preferences by participant identity/source across reconnects, even when
producer or consumer IDs change. setPreferredLayers still takes a
consumer ID; the new helper takes a producer ID.
Smooth on busy devices (0.8.0)
Three things keep a room with several cameras smooth on phones and modest
laptops (the same ideas as LiveKit's adaptive stream and dynacast, Tencent
TRTC's small stream with receiveWhenViewVisible, and Agora's automatic low
stream):
Adaptive stream: each video at the size it is shown. Tell the SDK how you show a remote video, and it receives the smallest layer that fills that view without stretching it, and pauses the video at the server while it cannot be seen (no bandwidth, no decoding). A grid of small tiles then decodes small videos, not three full-size ones.
LayoutBuilder(builder: (context, box) {
client.setVideoView(track.producerId,
width: box.maxWidth, height: box.maxHeight,
pixelRatio: MediaQuery.devicePixelRatioOf(context),
visible: isOnScreen); // false off screen or with the app in the background
return RTCVideoView(renderer, objectFit: RTCVideoViewObjectFit.RTCVideoViewObjectFitCover);
});
// When the tile goes away:
client.clearVideoView(track.producerId);
Calling it on every layout is fine (the same view sends nothing). More is
applied at once; less after about a second, and a hidden video pauses after
1.5 s, so scrolling past a tile does not stop it. From 0.9.2 a quality picked
with setReceiveQuality replaces the view's size (a hidden view still pauses
the video).
Dynacast: publishers stop encoding layers nobody receives. Nothing to call: the server tells the publishing client which of its camera layers any viewer needs, and the SDK switches the others off (the smallest always stays on, so a new viewer starts at once). A host nobody watches in full size no longer spends its CPU on its largest layer.
When the device itself is overloaded:
client.onCpuConstrained.listen((busy) async {
if (busy) {
await client.prioritizePerformance(); // the camera's largest layer stops
// and your own savings: effects off, smaller remote views
}
});
// later, when the user wants full quality back:
await client.prioritizePerformance(false);
onCpuConstrained says true when WebRTC reports the camera limited by the
CPU for about 4 s, and false when that clears; performancePrioritized
tells whether the lighter mode is on.
Camera, speaker and devices (0.8.0)
The native controls Agora, LiveKit and Stream Video offer, on the published camera:
final front = await client.switchCamera(); // front <-> back, same track: no pause for viewers
await client.setCameraZoom(2.0); // 1 = none
if (await client.cameraHasTorch()) {
await client.setCameraTorch(true); // light for a dark room
}
await client.setCameraFocus(const Point(0.5, 0.4)); // tap to focus and expose; null = automatic
await client.setSpeakerphoneOn(true); // loudspeaker; a Bluetooth or wired headset still wins
A reconnect captures the camera switchCamera turned to. In a dark room a phone
camera lowers its frame rate to expose each frame longer; the light keeps it at
full rate.
final mics = await client.listDevices(DeviceKind.audioinput);
await client.switchDevice(DeviceKind.audioinput, mics.last.deviceId); // routed natively, same track
await client.switchDevice(DeviceKind.audiooutput, speakerId);
client.onDevicesChanged.listen((e) => refreshPicker(e.devices)); // e.added, e.removed
On a desktop (or any device with several cameras), switch to a specific camera
by id (0.9.0). It is captured with the settings you published with and swapped
in place, so viewers see no pause; the old camera is released, and a reconnect
captures the chosen one. A switch that fails, or that unpublish/close
interrupts, releases the new camera and keeps the old one on air (0.9.3). Only a
camera the SDK captured (publish) can be switched this way: a stream you handed
in with publishStream is yours, so switchDevice throws StateError for it;
capture the new camera and publish that stream. An id no camera has throws
RpcError('bad-params') and keeps the old camera on air (0.9.6). Bind your preview again when it
fires:
await client.switchDevice(DeviceKind.videoinput, usbCameraId);
client.onLocalStreamChanged.listen((stream) => previewRenderer.srcObject = stream);
Audio mode for a session (Stream Video's broadcaster/viewer policies, Agora's audio scenario). Chosen when the client is made and applied before it connects; without it the platform's own set-up is used, as before 0.8.0:
StreamClient(wsUrl: url, token: token, audioMode: AudioMode.media); // an audience that only listens
StreamClient(wsUrl: url, token: token, audioMode: AudioMode.communication); // hosts and co-hosts who speak
AudioMode.media plays like a video app: no voice processing, the media
volume, Bluetooth headphones in their high-quality mode. A viewer brought on
stage to speak should join again with AudioMode.communication (Android cannot
change the mode within a session), or others may hear an echo.
Background noise. Phones remove it while someone speaks, with nothing to
set up: on Android, WebRTC's noise suppression plus the phone's own hardware
noise suppressor where it has one; on iOS, Apple's voice processing. Both run
in the default audio set-up and in AudioMode.communication; AudioMode.media
turns them off, as it is meant for listening. The browser SDK's stronger AI
noise suppression (@voltkit-rtc/client/noise-suppression, RNNoise) runs in
browsers only.
Diagnostics for support (0.8.0)
final report = await client.getDiagnostics();
sendToSupport(jsonEncode(report.toJson()));
The SDK version (sdkVersion) and platform, the room, identity, role and
connection state, how each media transport is connected, what is published and
received, and the last 200 notable events with their times. No token, no media.
In the background (0.8.0)
When the app goes to the background, videos you report with setVideoView are
paused at the server (the sound keeps playing) and resume when it comes back,
as a background tab does in the browser SDK. That saves a listening viewer's
data and battery.
Phones stop giving a backgrounded app its camera. Viewers would see a frozen picture, so a host's app can switch the camera (and the microphone) off in the background and back on when it returns (Stream Video's options):
StreamClient(wsUrl: url, token: token, pauseCameraInBackground: true, pauseMicInBackground: false);
Only what the SDK switched off is switched back on: a camera the user turned off stays off, and so does one a moderator locked. From 0.9.3 that includes a switch made while the app is in the background: a microphone muted from a notification stays muted when the app comes back. From 0.9.4 a camera or microphone that comes on while the app is in the background, through something started before it went there (a switch-on answered late, a publish that finished, the publisher fallback letting go), is switched off for the background too and comes back with the app; one the user switches on in the background stays on.
To keep publishing sound in the background, the app itself needs:
iOS: the audio background mode (UIBackgroundModes in Info.plist).
Android: a foreground service of type microphone (and camera to keep the
camera), declared in the manifest and started while live. Without one, Android
cuts the microphone and camera shortly after the app leaves the screen.
A faster first picture (0.9.0)
Nothing to change in your app: the join brings the media transports with it (controllers from 2026-10-20; with older ones they are asked for as before), and neither the DTLS handshake nor a video's first layer waits for an acknowledgement any more. Each is a round trip saved, 50-150 ms on a mobile network, for the first publish and the first video alike.
Check before going live (0.8.0)
// Your backend mints a host token for a test room nobody else is in.
final report = await StreamClient.preflight(wsUrl: url, token: token); // duration:, audio:, video:
if (!report.ok) showProblem(report.steps.firstWhere((s) => !s.ok).error);
else if (report.relayed || report.protocol == 'tcp') showHint('A firewall slows your connection');
else if (report.warnings.contains(MediaWarningCode.micSilent)) showHint('Your microphone sends no sound');
Like Twilio's runPreflight and LiveKit's ConnectionCheck: it joins, publishes
the camera and microphone for a few seconds (so it checks them too), and leaves.
The report has each step (join, media) with its time or error, how media
travelled (protocol, relayed), rttMs, lossPct, uploadKbps, level and
any camera or microphone warnings. It never throws. The session is real and
billed like any other for its few seconds.
Camera and microphone warnings (0.8.0)
client.onMediaWarning.listen((w) {
if (w.code == MediaWarningCode.cameraFpsLow) showBanner(w.active, 'Too dark: add light, or setCameraTorch(true)');
if (w.code == MediaWarningCode.micSilent) showBanner(w.active, 'Your microphone sends no sound');
});
Each warning comes once when it starts and once when it clears (Agora's
exception events): micSilent (on, but exact silence for about 6 s),
cameraFpsLow (under 15 fps for about 6 s when 24+ were asked, usually a dark
room; 20 fps is not a warning), cameraInterrupted / micInterrupted (the
system took the device for more than 2 s: a call, another app). Checked every
2 s while listened to.
Who is speaking (0.8.0)
client.onAudioLevels.listen((levels) {
for (final l in levels) tileFor(l.identity).ring(l.speaking);
});
client.onActiveSpeakers.listen((speakers) => showLarge(speakers.first)); // most active first
onAudioLevels comes about twice a second while listened to, for every remote
microphone, music and shared audio this client receives and its own microphone
while on (AudioLevel: identity, producerId, source, level, speaking,
local). speaking is above -35 dBov (speakingLevel). onActiveSpeakers
ranks microphones the way Amazon Chime does; music never counts. Nothing is read
while nobody listens.
Poor connections: audio keeps going (0.8.0)
When a viewer's connection cannot carry video, the audio comes first, as in Agora's audio-only stream fallback:
- The media server cuts a video this viewer's bandwidth cannot carry even at its smallest size, while the sound keeps flowing, and says so.
- The SDK pauses remote cameras when the received sound itself breaks up badly (a quarter of it lost over about 15 seconds), so what bandwidth there is goes to the audio. Screen shares stay. The cameras come back once the connection has recovered for 20 seconds, longer if it falls back again soon.
Either way the app is told, per video, so it can show the person's name instead of a frozen picture:
client.onStreamFallback.listen((e) {
tileFor(e.producerId).showPoorConnection(e.fallback); // e.reason: bandwidth or audioProtection
});
The pause is on by default. Turn it off with StreamClient(wsUrl: ..., token: ..., audioOnlyFallback: false) or client.setAudioOnlyFallback(false), which
also brings back any camera held at that moment.
The host's side (opt-in). client.setPublishFallback(true) pauses this
client's camera while its own sound breaks up on the way to the server, and
resumes it once it recovers (onPublishFallback). A camera the user or a
moderator switches off meanwhile stays off, and one the fallback lets go of
while the app is in the background (with pauseCameraInBackground) comes back
with the app. unpublish() ends a hold (onPublishFallback says false), and
the next camera is watched afresh (0.9.4). audioOnlyFallbackActive says
whether it is holding cameras now. The server's bandwidth reports need the
controller and media node from 2026-10-20; older ones never send them.
Screen sharing (0.5.0)
Camera, screen, mic, music and optional screen audio have independent producers.
Only hosts/co-hosts can publish. setCamEnabled and setMicEnabled leave a
screen share alone; unpublishScreen() leaves camera/mic/music alone.
unpublish() and close() stop all publishing.
client.onScreenShareStarted.listen((stream) => screenPreview.srcObject = stream);
client.onScreenShareEnded.listen((_) => screenPreview.srcObject = null);
await client.publishScreen(
options: const ScreenPublishOptions(mode: 'detail', audio: true),
stream: displayStream, // optional: a screen capture the app already owns
);
await client.unpublishScreen();
screenSharing reports whether a capture is held; screenStream exposes it.
detail sends up to 1920×1080 at 15 fps, 2.5 Mbps plus a 600 kbps half-size
layer (~3.1 Mbps total). motion sends 30 fps at 6 Mbps plus 600 kbps (~6.6
Mbps total). Captures keep their native aspect ratio and are never upscaled.
The sender uses maintain-resolution for detail and maintain-framerate for
motion. A screen received on onTrack has source == 'screen'; granted
screen audio has source == 'screen-audio' and gets its own volume
(Helper.setVolume on its track), like music. Requesting audio does not guarantee it: when no
track is granted screen-audio-unavailable is reported on onPublishWarning
and video continues. Android's current flutter_webrtc capture provides video
only. Do not substitute the microphone for system audio.
A still-live screen is republished after reconnect, with the same profile.
Platform track onEnded automatically calls unpublishScreen; the application's
existing callback is preserved. flutter_webrtc 1.6.2 does not forward Android
MediaProjection's OS stop callback to Dart. Keep a visible Stop sharing
button wired to unpublishScreen; an app with a native projection-stop bridge
must call that method from its stop callback too. Do not infer that a static
slide has ended from a lack of new frames.
0.6.0 iOS preview: an extension template and native lifecycle plugin are
included, disabled by default. Native compilation and Apple-device acceptance
are unverified; the owner has no Apple hardware. Install/configure the extension
and explicitly pass ScreenPublishOptions(iosBroadcastEnabled: true) only when
integrating the preview. iosStartTimeout defaults to 60 seconds (positive,
at most four minutes). System stop, pause and resume use session-scoped lifecycle
events; onScreenSharePaused reports pause state. The native reader provides
video only, so audio: true warns and continues without shared audio. Camera/mic
rooms are unaffected, and caller-supplied screen streams remain caller-owned.
Read the complete iOS integration and validation guide.
Android foreground service
The application owns a mediaProjection foreground service and its visible
notification. For example, using flutter_background, add these entries:
<!-- AndroidManifest.xml, outside <application> -->
<uses-permission android:name="android.permission.FOREGROUND_SERVICE" />
<uses-permission android:name="android.permission.FOREGROUND_SERVICE_MEDIA_PROJECTION" />
<uses-permission android:name="android.permission.WAKE_LOCK" />
<!-- Inside <application>; the class is supplied by flutter_background -->
<service
android:name="de.julianassmann.flutter_background.IsolateHolderService"
android:enabled="true"
android:exported="false"
android:foregroundServiceType="mediaProjection" />
Initialize the app's foreground-service plugin with an appropriate notification. Then pass its start/stop functions (or equivalent functions for your own native service). VoltKit requests user projection consent before invoking start, as modern Android requires:
// Import package:flutter_background/flutter_background.dart in your app.
await FlutterBackground.initialize(
androidConfig: const FlutterBackgroundAndroidConfig(
notificationTitle: 'Screen sharing',
notificationText: 'Your screen is being shared in this room',
),
);
await client.publishScreen(
options: const ScreenPublishOptions(mode: 'detail'),
startForegroundService: () async {
if (!await FlutterBackground.enableBackgroundExecution()) {
throw StateError('Foreground screen-capture service could not start');
}
},
stopForegroundService: () async {
await FlutterBackground.disableBackgroundExecution();
},
);
Missing callbacks are rejected with screen-share-service-required. If the
app passes stream, it owns permissions and its service, and stops both after
unpublishScreen; the SDK does not dispose caller-owned capture tracks.
Make the application's stop callback idempotent: it is also invoked if service
startup partially succeeds and then throws. A stop/close during projection
consent prevents a late foreground service or capture start. Cleanup failures
report screen-stop-failed on onPublishWarning; remaining capture tracks,
the stream and foreground-service cleanup are still attempted, and leaving the
room can finish. Keep the notification's stop action wired to unpublishScreen.
Windows, macOS and Linux
Use desktopCapturer.getSources(types: [SourceType.Screen, SourceType.Window])
from flutter_webrtc, show a picker, and pass the source the user selects:
await client.publishScreen(
options: ScreenPublishOptions(mode: 'motion', sourceId: selectedSource.id),
);
Missing selection reports screen-share-source-required. macOS needs Screen
Recording permission in System Settings. Linux support depends on the display
server/desktop capture support in flutter_webrtc; validate your target desktop.
A capture that cannot produce measurable frames reports
screen-share-size-unavailable instead of advertising invented dimensions.
Other capture failures report screen-share-failed, permission denial can
report screen-share-denied, and starting twice reports screen-share-active.
See screen sharing for platform setup
and troubleshooting for connection,
quality and device problems.
Paid device recording (0.6.0)
Recording is a separate prepaid add-on. Enable it in the project's Recording console after purchasing recording storage. Live quality remains independent: recording resolution, bitrate and frame rate come from the funded admin policy (default 360p, 15 fps, 400 kbps video / 48 kbps audio). Developers cannot request an arbitrary quality. Camera-on hosts/cohosts may record; viewers may not.
startRecording is currently an Android video-only preview, explicitly off
by default. It uses the existing local camera, creates bounded short MP4 files
and uploads them directly from the device to private R2. Native microphone
recording is not included in this preview. Android compilation passed; physical
capture is unverified. iOS native recording is unavailable; the independent iOS
screen-sharing preview does not add recording support.
await client.startRecording(
apiUrl: Uri.parse('https://api.voltkit.in/api/v1'),
androidCapturePreviewEnabled: true,
chunkMiB: 10, // optional: admin-allowed 5 or 10 MiB
onRecordingReady: (file) {
// Save file.id, file.playbackUrl and file.retainUntil in your app database.
// This URL opens the authenticated owner console; it is not a public file.
},
);
final files = await client.stopRecording(); // flush before unpublish/close
For an application-provided camera+microphone encoder, RecordingUpload.create
returns the authoritative RecordingPolicy. Encode a finalized MP4/WebM file
with those dimensions, bitrates, frame rate and duration limit, then call
uploadFile. Keep the file immutable until completion; use deleteAfterUpload
to remove temporary local files. The file is deleted only after the server has
accepted the recording: a failed upload keeps it, since it may be the only
copy. If that delete fails, the upload still succeeded and the returned
RecordingFile.localDeleteError says why; do not upload it again. The
Android preview keeps a segment whose upload failed too, stops recording, and
stopRecording() throws a RecordingException whose keptFiles lists those
paths: upload them again or delete them. When a later segment cannot start,
stopRecording() still waits for the segments uploading and names any file
they keep (0.9.4). A finalisation the server accepted is
never aborted, even if its reply cannot be read (a RecordingException; the
file is kept). If the answer to finalising is lost or unclear (no reply, an
unreadable one, a 5xx), uploadFile asks again twice; still unclear, it keeps
the file, does not abort (the recording may be saved; the server finishes or
expires the session) and throws a RecordingException saying so. Use abort on cancellation. This native file
uploader streams parts from disk with bounded concurrency rather than reading
whole recordings into RAM. Uploaded files are validated inside Cloudflare and
remain private until accepted. Room/session tokens authorize JSON API calls;
R2 PUTs carry only short-lived presigned credentials. Never put the project's
API secret or an R2 key in a device app.
final upload = await RecordingUpload.create(
apiUrl: Uri.parse('https://api.voltkit.in/api/v1'),
roomToken: roomToken,
);
// Encode a short file using upload.policy. Never accumulate unlimited files.
final file = await upload.uploadFile(encodedFile, deleteAfterUpload: true);
Completed recordings expire 90 days after upload completion. Save the
recording ID and expiry, and let your authenticated application backend mint a
short-lived media URL through /recording/upload/server/playback using its RTC
secret. That server must authorize its own end user before issuing playback.
Media streams from R2 through a Cloudflare Worker, never through VoltKit's VPS.
Playback uses a separate delivery quota; buying packages repeatedly adds to
balances. Recording upload quota includes retention and is consumed once; it
is not replenished by deleting files. See recording integration.
Libraries
- voltkit_rtc
- Flutter SDK for VoltKit RTC: realtime audio and video rooms, chat, and live broadcast publishing.