voltkit_rtc 0.4.0

Flutter SDK for VoltKit RTC: realtime audio and video rooms, in-room and standalone chat, and live broadcast publishing over WHIP.

voltkit_rtc

Flutter SDK for VoltKit RTC: realtime audio and video rooms, chat, and live broadcast publishing.

Rooms	StreamClient — join, publish camera and mic, render everyone else, moderate
Chat	on the room's own connection, or ChatClient for chat with no media
Broadcast	BroadcastPublisher — go live to an audience of any size over WHIP

Your backend mints a short-lived token and hands the app { token, wsUrl }. This package turns that into a live session. The API secret never reaches the device.

flutter pub add voltkit_rtc

It builds on flutter_webrtc 1.6 and works with current Flutter on Android (minSdk 24) and iOS. There is nothing else to install or patch.

Coming from the unpublished stream_client package?

Only the package name changed. Class names, methods and events are the same.

dependencies:
  voltkit_rtc: ^0.2.0        # was: stream_client: { path: ... }

import 'package:voltkit_rtc/voltkit_rtc.dart';   // was: package:stream_client/stream_client.dart

Quick start

import 'package:flutter_webrtc/flutter_webrtc.dart';
import 'package:voltkit_rtc/voltkit_rtc.dart';

final client = StreamClient(wsUrl: wsUrl, token: token);

// Wire the events BEFORE connecting: connect() consumes everyone who is
// already publishing, and those tracks arrive during the call.
final renderer = RTCVideoRenderer();
await renderer.initialize();

client.onTrack.listen((t) {
  if (t.kind == 'video') renderer.srcObject = t.stream;
});
client.onTrackEnded.listen((e) => renderer.srcObject = null);
client.onClosed.listen((reason) => Navigator.pop(context));

final join = await client.connect();
if (join.role.canPublish) {
  await client.publish();          // camera + mic, simulcast
}

Always await client.close() when leaving the screen. A client whose reference is dropped without closing keeps its reconnect loop running.

Permissions

flutter_webrtc needs camera and microphone permission before publish().

Android — android/app/src/main/AndroidManifest.xml:

<uses-permission android:name="android.permission.CAMERA" />
<uses-permission android:name="android.permission.RECORD_AUDIO" />
<uses-permission android:name="android.permission.INTERNET" />
<uses-permission android:name="android.permission.MODIFY_AUDIO_SETTINGS" />

Minimum SDK 21, and android:usesCleartextTraffic is not required — signaling is wss and media is DTLS-SRTP.

iOS — ios/Runner/Info.plist:

<key>NSCameraUsageDescription</key>
<string>Used to broadcast video when you go live.</string>
<key>NSMicrophoneUsageDescription</key>
<string>Used to broadcast audio when you go live.</string>

Add audio to UIBackgroundModes if a broadcast should survive the app going to background.

Roles

The role is decided by your backend and signed into the token; the client cannot change it, and the media node enforces it.

Role	canPublish	canModerate
host	yes	yes
cohost	yes	no
moderator	no	yes
viewer	no	no

if (client.role.canPublish) await client.publish();

Events

All broadcast streams, safe to listen to more than once.

Stream	Fires when
onTrack	someone's audio or video is ready to render
onTrackEnded	that track went away
onParticipantJoined / onParticipantLeft	roster changes
onRoleChanged	someone was promoted or demoted
onHandRaised	a hand went up or down; reason is declined, withdrawn or granted
onForceMuted	a moderator switched your mic or camera off, or released it
onStreamInterrupted	media stopped but the connection is fine — the publisher is reconnecting
onSubscribeError	one remote track could not be subscribed to
onTrackMuted	someone muted their mic, turned their music off or covered their camera, or un-did it; source says which
onClaimChanged	an exclusive slot was taken, moved, freed, locked, or its holder went away
onParticipantUpdated	someone's display metadata changed mid-session
onRoomMetadataChanged	room-scoped metadata changed — a board's seat format, say
onPublishWarning	publishing worked, but something affecting quality did not apply
onMessage	a chat message arrived — yours included, so render from this stream
onMessageDeleted	a message was removed, by its author or a moderator
onChatMuted	someone was muted or unmuted in chat; check identity against your own
onTyping	someone else started or stopped typing
onUsageLimit	the account behind the session ran out of minutes or hit a usage limit; the session ends at endsAt unless topped up
onClosed	terminal; nothing follows

Moderation

Moderator-only, and enforced at the SFU rather than asked of the target's client — the packets stop at the server, so nothing the target does can override it.

await client.forceMute(identity);                       // mic off
await client.forceMute(identity, kind: 'video');        // camera off
await client.forceMute(identity, kind: 'video', paused: false);  // give it back
await client.promote(identity, Role.cohost);
await client.kick(identity);
await client.endRoom();

forceMute replaces requestMute, which still works and means forceMute(identity, kind: 'audio').

The target cannot switch a force-muted kind back on — the server refuses it — so render onForceMuted as a locked button, not a disabled feature. Its kind says which, and paused: false means control was handed back. Leaving the button live means the user taps it, nothing happens, and the app looks broken rather than moderated.

The rest of the room needs no extra work: the result arrives on onTrackMuted like any other pause.

Exclusive slots (seats on a board)

A slot is a named thing in a room that at most one participant may hold. Seats on a 20-seat audio board or a 9-seat video board are the motivating case, but the service never learns that — a key is an opaque string.

await client.claim('seat-13');                       // take it, or throw slot-taken
await client.claim('seat-13', identity: 'guest-1');  // moderator, on someone's behalf
await client.release('seat-13');
await client.blockSlot('seat-20');                   // lock it; moderator only
await client.unblockSlot('seat-20');
await client.setMetadata({'displayName': 'Priya'});

// room-scoped, moderator-only, persisted — not just broadcast
await client.setRoomMetadata({'seatFormat': 10},
    requireFree: ['seat-11', 'seat-12']);            // refuse if those are taken

setRoomMetadata merges one key deep (a null value deletes a key; replace: true swaps the whole object), and persists — a client joining afterwards reads the new value from JoinResult.metadata. A broadcast-only change would be right for whoever was present and wrong for everyone who arrived later, which is invisible to the person who made it.

requireFree is checked and applied together, so nothing can be claimed in between. Use it when a change invalidates slots — shrinking a board from twenty seats to ten — and the refusal names them:

try {
  await client.setRoomMetadata({'seatFormat': 10},
      requireFree: [for (var i = 11; i <= 20; i++) 'seat-$i']);
} on RpcError catch (e) {
  if (e.code == 'slot-occupied') {
    final taken = (e.data?['slots'] as List?)?.cast<String>() ?? [];
    // "Clear seats 13 and 17 first."
  }
}

The service cannot work those slots out for you: metadata is opaque to it, so twenty-seats-becomes-ten is only a different blob. Naming them keeps that knowledge in your app while the check stays atomic.

join() seeds the board: result.claims, result.blockedSlots, and result.metadata (room-scoped, set by your backend at room creation — a seat format, say). getRoomState returns the same after a reconnect.

onClaimChanged carries a state, and away is not free.

State	Draw it as
held	taken
away	taken, greyed — the holder's socket dropped and the slot is still theirs
free	empty
blocked	locked

Rendering away as free shows a viewer their own seat apparently empty and then refuses to let them retake it. It is not a rare state: every guest granted a seat while on a relay node passes through it, because that promotion moves them to the origin and closes their socket on the way. A claim is held for about a minute after a disconnect and only the same identity can reclaim it.

Rules worth knowing: one claim per identity — claiming a second releases the first, which is what moving seats means. blockSlot refuses an occupied slot (slot-occupied) rather than evicting; remove, then lock. Any moderator can release any slot, so a seat stranded by a half-finished demote is recoverable by anyone.

Claims and roles are deliberately orthogonal: demoting someone does not free their slot, and your app should release it.

Hands and layers:

await client.raiseHand(metadata: {'slot': 'seat-13'});   // which seat you want
await client.lowerHand();                                // withdraw
await client.lowerHand(identity: 'priya');               // DECLINE — moderators only
// the host reads it back off the event:
client.onHandRaised.listen((h) {
  if (h.raised) print('${h.identity} wants ${h.slot}');
  else if (h.wasDeclined) print('declined by ${h.by}');   // clear the countdown
  else if (h.wasGranted) print('seated');                  // drop it from the queue
});
await client.lowerHand();
await client.setPreferredLayers(track.consumerId, 0);  // lowest rung

setPreferredLayers overrides the SFU's own choice for one subscription. You rarely want it — the SFU picks per viewer from their available bandwidth — but it is the lever for "this tile is a thumbnail, don't send it 720p". Pass RemoteTrack.consumerId, which arrives on onTrack.

PK battles: two rooms at once

One identity can hold two rooms simultaneously — session replacement is scoped by room, so the second connection does not displace the first. Each room keeps its own roster, producers and events; nothing merges. No server-side PK feature is involved.

See example/pk_battle.dart for the full pattern, including dropping a simulcast layer because each tile is half width. That file is compiled by flutter analyze, so unlike a snippet it cannot quietly rot.

onTrackMuted is not optional to render. A paused video producer stops sending frames and the renderer keeps showing the last one it decoded, so a host who covers their camera becomes a frozen still of themselves — indefinitely, and indistinguishable from the stream having hung. There is no frame-level signal on RTCVideoRenderer to infer this from (onResize fires on a size change, not a frame), so this event is the only way to know. It also fires on join for anyone already muted, so a late viewer is not left looking at a black tile with no explanation.

onSubscribeError is per producer, and the join still succeeds around it. Every producer is subscribed to independently, so one failure costs you that one track, not the rest of the room. Listening is optional — show the affected participant as unavailable, or retry — but ignoring it means a missing tile with no explanation.

Publish quality

publishStream() defaults to a broadcast profile, which wants the opposite trade-offs from a video call:

await client.publishStream(stream);                       // broadcast defaults
await client.publishStream(stream, video: const VideoPublishOptions(
  degradationPreference: RTCDegradationPreference.MAINTAIN_FRAMERATE,
  layers: 3,                                              // call-like profile
));

Default	Why
preferHardwareCodec: true	H264 when the device has it. Phones have hardware H264 encoders and essentially none has one for VP8 — this is the single biggest lever.
degradationPreference: MAINTAIN_RESOLUTION	Under CPU load, drop framerate rather than resolution. WebRTC defaults to the reverse, which is right for a call and wrong when viewers are reading text.
layers: 2	Each layer is a full encode. Three software encodes on a mid-range phone force the capture itself down a rung before anything reaches the wire.
maxFramerate: 30	Layers budget for this whether or not the camera reaches it; unset, each one plans for 60 and spends bitrate on frames instead of pixels.
quality: null	The highest quality the project allows (see below).
topBitrate: null	Picked from the picture's size: 4 Mbps at 1080p, 2.5 Mbps at 720p, 1 Mbps at 480p.

Video quality: 1080p, 720p, 480p

await client.publish(video: const VideoPublishOptions(quality: VideoQuality.p1080));
client.videoPolicy.maxQuality;   // what this project may publish: p720 or p1080
client.videoQuality;             // what is being published, after that limit

• 720p and 480p are included in every plan. 1080p uses more bandwidth and is priced separately: request it from the project's Settings in the VoltKit dashboard. Your first project can also use it during the free trial, to test it.
• Asking for more than the project allows publishes the highest it does allow. publish() captures at the quality being published (1280x720 by default); with publishStream(), capture at least that size yourself.
• The layers follow the picture's real size: two at 480p (427x240 and 854x480), and never more than the picture can carry, so the full picture is always the one sent. A portrait picture gets the same layers on their side (360x640 and 720x1280 at 720p).
• The service caps each publisher's incoming bitrate for the project's tier (4 Mbps up to 720p, 7 Mbps with 1080p, audio included), whatever is set here.
• To change quality mid-stream, unpublish() and publish() again with the new options.

Quality	Layers sent (the default two)	Bitrates	Upload to keep the top layer
1080p	960x540 · 1920x1080	1.5 Mbps · 4 Mbps	about 5.5 Mbps; 8 Mbps or more to be safe
720p	640x360 · 1280x720	700 kbps · 2.5 Mbps	about 3.2 Mbps; 4 Mbps or more to be safe
480p	427x240 · 854x480	400 kbps · 1 Mbps	about 1.4 Mbps; 2 Mbps or more to be safe

With layers: 3, 720p and 1080p add a quarter-size layer: 320x180 at 200 kbps, or 480x270 at 400 kbps. A soft picture usually comes from the phone (too busy to encode) or its upload (too slow for the top layer): troubleshooting shows how to tell which, and what to do about each.

The full guide, with data use and troubleshooting: voltkit.in/docs/video-quality.

publish() forwards these too — publish(video: const VideoPublishOptions(...)) — and the profile is remembered, so a reconnect republishes with the same one. Without that a host failing over to a new origin silently reverts to the defaults, which reads as "the setting did not work" rather than "the setting stopped being applied".

Stopping

unpublish() releases the camera and microphone as well as closing the producers. It has to: producers are created with stopTracks: false so a reconnect can republish the same tracks, which means nothing else would let the device go. A co-host who steps down and then takes a seat again would otherwise ask Android for a camera the process still holds, and get a track that never produces a frame — signalling all succeeds and the tile stays black.

A stream you passed to publishStream is yours and is left alone; one publish() acquired is disposed.

It also drops the send transport, so the next publish() builds a fresh one. A transport whose producers are closed and whose tracks are stopped does not reliably carry media again — and the failure is invisible from the outside: produce succeeds over the websocket, the server creates the producer and announces it, viewers consume and resume it, and no RTP ever arrives. Perfect state, dead pipe. Rebuilding costs a transport setup on a second promotion and removes a class of bug that reads as everything-but-the-video working.

setCamEnabled(false) is a different thing: it pauses the producer and keeps the device, so unmuting is instant.

To bisect a codec-specific problem, force one outright:

await client.publishStream(stream,
    video: const VideoPublishOptions(codecMimeType: 'video/VP8'));

preferHardwareCodec: false is not the way to do that — it hands the choice back to the library's preference order rather than selecting VP8, so the variable you meant to isolate is still being chosen for you. codecMimeType forces one, and reports on onPublishWarning when the device does not advertise it, rather than quietly publishing with something else and inverting the conclusion of your test.

The symptom when these are wrong is specific and easy to misread as bandwidth: the host log shows a full ladder at capture, then ReconfigureEncoder drops every rung the moment a viewer joins and all the encoders actually run. Raising the capture resolution makes it worse — it gives the encoder more to throw away. Check is_hardware_accelerated in the host log before suspecting the network.

onStreamInterrupted is not a disconnect. Show a spinner and wait — tearing the session down here turns a two-second blip into a lost broadcast.

onClosed carries why:

Reason	Meaning
replaced	the same identity opened a newer session elsewhere
kicked	removed by a moderator
roomClosed	the host ended it
gaveUp	reconnection was attempted and abandoned
left	you called close()

Publishing

await client.publish();                       // sensible defaults
await client.publish(constraints: {...});     // or your own
await client.publishStream(existingStream);   // or a stream you already have

await client.setMicEnabled(false);
await client.setCamEnabled(false);
await client.unpublish();

Video is published as simulcast layers (by default 360p and 720p; see Video quality). The server picks one per viewer from their available bandwidth — you do not manage that. At 720p and 480p viewers start on the top layer and move down only if their connection cannot carry it.

Moderation

await client.raiseHand();
await client.requestMute(identity);              // moderating roles
await client.promote(identity, Role.cohost);
await client.kick(identity, 'spam');
await client.endRoom();                          // host only

Promotion makes the promoted client reconnect. Publishers live on the origin node so their producers stay local, and a viewer may be on a relay. The SDK handles this: it receives the new role's token, reconnects, and rebuilds media. Your UI should expect a brief interruption for that participant, not an error.

Chat

Every room has a chat channel of the same name. StreamClient carries it on the connection it already holds, so there is nothing more to connect:

client.onMessage.listen((m) => setState(() => messages.add(m)));
client.onMessageDeleted.listen((e) => setState(() => messages.removeWhere((m) => m.id == e.id)));

await client.sendMessage('hello everyone');
await client.sendMessage('psst', to: 'user-42');                  // direct: only the two of you see it
await client.sendMessage('', type: 'reaction',                    // delivered, not stored
    metadata: {'emoji': 'heart'}, ephemeral: true);

final older = await client.getMessages(limit: 50, before: messages.first.id);   // page backwards
await client.setTyping(true);                                     // safe on every keystroke
final state = await client.getChatState();                        // members, and whether you are muted

Your own message comes back on onMessage like everyone else's. Pass a clientMsgId to match it to one you rendered optimistically.

text is delivered exactly as it was sent. Render it with Text, never as HTML or markup.

Moderators — hosts, co-hosts and moderators — can also do this:

await client.deleteMessage(messageId);           // anyone's; everyone else, only their own
await client.muteChat('user-42');                // survives that user reconnecting
await client.muteChat('user-42', muted: false);

A muted sender gets RpcError with code chat-muted; a client sending too fast gets rate-limited. Both are worth showing to the user rather than retrying.

Chat without media

An audience watching a broadcast has no reason to join a room. ChatClient opens a connection that carries chat and nothing else, into the same conversation:

// { token, wsUrl } from your backend: POST /v1/chat/{channel}/tokens
final chat = ChatClient(wsUrl: wsUrl, token: token);
chat.onMessage.listen(addMessage);
chat.onClosed.listen((reason) => showEnded(reason));

final joined = await chat.connect();
messages = joined.messages;                      // recent history, oldest first
await chat.sendMessage('first!');
// ...
await chat.close();

It has the same methods and events as the chat half of StreamClient, plus onReconnected — the moment to call getMessages() and fill any gap.

Music

A host can play music to the room without it going through the microphone. It is published as a track of its own, labelled music, so the audience hears it clean and everybody keeps separate control of it.

Hearing it (every platform). A music track arrives on onTrack like any other, with source == 'music':

client.onTrack.listen((t) {
  if (t.isMusic) {
    musicRenderer.srcObject = t.stream;      // its own renderer = its own volume
  } else if (t.kind == 'video') {
    videoRenderer.srcObject = t.stream;
  }
});

// "Mute the music" for this viewer only, leaving the host's voice:
for (final track in musicStream.getAudioTracks()) {
  track.enabled = false;
}

client.onTrackMuted.listen((m) {
  if (m.source == 'music') setState(() => musicPlaying = !m.paused);
});

Playing it. Hand the SDK the audio as a MediaStream:

await client.publishMusic(musicStream);
await client.setMusicEnabled(false);     // music off for the audience, mic untouched
await client.setMicEnabled(false);       // mic off, music keeps playing
await client.stopMusic();

setMicEnabled and setMusicEnabled are independent, in both directions.

Where musicStream comes from depends on the platform, and this is the part to plan for:

Platform	In-app music as a stream
Flutter Web	Yes. Play the file in an audio element and take its stream with the Web Audio API, as the browser SDK does.
Android, iOS	Not with the WebRTC plugin this package builds on (flutter_webrtc 0.9.x). It can capture the microphone and the screen, but it does not expose audio your app is playing as a track.

On Android and iOS you therefore need a native audio source to get a clean music track — a platform channel that feeds decoded audio into WebRTC, or a plugin that does. Playing the song through the phone speaker and letting the microphone pick it up is the workaround to avoid: the echo canceller exists to remove exactly that sound. Until your app has such a source, mobile hosts can still receive music published from a web host, with the separate controls above.

Broadcast

A broadcast is one publisher and an audience of any size, delivered over LL-HLS and WebRTC by a CDN-friendly media server rather than by a room. Use it when the audience watches and does not talk back.

// stream.publish.whipUrl from your backend: POST /v1/streams
final publisher = BroadcastPublisher(whipUrl: whipUrl);
publisher.onState.listen((s) => setState(() => state = s));      // connecting, live, interrupted, failed
publisher.onWarning.listen(debugPrint);

await publisher.start();                         // camera + mic; or start(stream: yours)
preview.srcObject = publisher.localStream;

publisher.setMicEnabled(false);
publisher.setCamEnabled(false);
await publisher.stop();                          // off air, camera released

The whipUrl carries the publish token. Treat it like a password: fetch it when the user taps "go live", and do not log it.

start() throws WhipError if the media server refuses. isRefused means the URL itself was not accepted — expired, for another stream, or the account is out of minutes — and a retry with the same URL will be refused again.

Video is offered as H.264 only. The media server repackages H.264 into HLS untouched; any other codec makes it transcode every frame. A device with no H.264 encoder still publishes, and onWarning says so.

Viewers need nothing from this package: play the stream's llhls URL with any HLS player (video_player handles it on iOS and Android).

BroadcastPublisher's HTTP exchange is tested here against a real server. The media half runs only on a device — check it on your own hardware against your deployment before you ship.

Reconnection

Handled for you, and worth knowing what it does:

• exponential backoff with jitter, one chain at a time
• reconnects with the refreshed token the server pushes over the live socket, not the one you started with — the original expires in ~10 minutes, and a broadcast outlasts it
• republishes automatically, so a host survives its origin node dying
• gives up on terminal errors rather than looping: a revoked key or an ended room will refuse every attempt

disconnected is not the end of a session. onClosed is.

Two rooms at once (PK battles)

Session replacement is scoped per room, so one identity can hold two rooms simultaneously — the shape of a PK battle, where two hosts each broadcast in their own room and viewers watch both.

final red  = StreamClient(wsUrl: wsUrl, token: redToken);
final blue = StreamClient(wsUrl: wsUrl, token: blueToken);
await Future.wait([red.connect(), blue.connect()]);
// ... and close BOTH when the battle ends

Two clients means two sockets, two transport pairs and two ICE negotiations. On a mobile network that doubles the reconnect surface at the moment the audience is largest, so keep them independent: a reconnect on one must not disturb the other, and each must be closed before it is replaced.

Diagnosing "it just hangs"

Pass onWire and every frame is logged in both directions:

final client = StreamClient(wsUrl: wsUrl, token: token, onWire: debugPrint);

-> #1 join
<- #1 join  6ms
-> #2 createTransport
<- #2 createTransport  8ms
-> #3 connectTransport
<- #3 connectTransport  12ms
-> #4 produce
<- #4 produce  9ms

This exists because two very different failures look identical from the app: the server never answered, or the client never asked. They point at opposite halves of the system, and guessing wastes days.

If publish() times out with produce-failed and there is no -> produce line, the RPC never left the device. The request is assembled inside mediasoup's _produce, which first awaits _handler.send(...) — the SDP and peer-connection work — and only emits produce afterwards. A stall in that handler means nothing is ever sent, and the server has no idea you wanted to publish.

That handler runs inside a task queue which catches and prints exceptions rather than surfacing them, so the underlying throw appears in the platform log and nowhere else. Search it for a throw from unified_plan.run() or send() before looking at the server.

If you do see -> produce with no <- line, that is genuinely server-side — say so, because the orchestrator answers every produce, including invalid ones, in about a millisecond.

Help

Documentation: https://voltkit.in/docs/flutter-sdk. Questions and bug reports: [e-mail].

Testing

flutter test

The signaling layer has no Flutter or WebRTC imports, so it is tested against a real HttpServer WebSocket — reconnects, close code 4001, token refresh and terminal errors are all exercised without a device. Chat and the WHIP exchange are tested the same way.

Protocol parsing is tested against a captured live response rather than a hand-written fixture, because the failure mode of drift is a renamed field parsing as null and showing up as a black screen:

ORCH_URL=http://127.0.0.1:8080 API_KEY=sk_... ./tool/capture_fixtures.sh

On a real Android runtime, against a running stack — a browser hosts a room and an app built on this package joins it, publishes, subscribes and chats:

./tool/device_e2e/run.sh

How it is built

Rooms use mediasoup's client protocol. That client is included in this package (lib/src/mediasoup), adapted from mediasoup_client_flutter 0.8.5 under its MIT licence. The published 0.8.5 does not build with current flutter_webrtc, and a package on pub.dev cannot depend on a fork, so the fixed copy ships here. Every change is marked VoltKit patch in the source:

• builds against flutter_webrtc 1.6 / webrtc_interface 1.5 (encoding priority and RTCP CNAME types);
• the transport waits for its peer connection before the first publish or subscribe (it could race it and fail with a null-check error);
• simulcast layers named with RIDs such as r0 parse (they were read as ints);
• an encoding without a scalability mode is one layer, not a crash;
• the answer uses the header extension IDs the offer actually assigned (current libwebrtc rejects renumbering).

None of these could be seen without a device; tool/device_e2e/run.sh is the test that found them, and test/vendored_mediasoup_test.dart pins them.