utd_media_client 2.9.9

UTD Media Client — real-time audio and video for Flutter (iOS, Android, Web).

Changelog

2.9.9

• DataReceivedEvent.senderIdentity (new, optional): the identity the packet itself names, even when the room has no participant for it. DataReceivedEvent.participant is looked up by the packet's identity, so it is null whenever the room has no entry for the sender at that moment (his first packets before he was registered, a packet that arrives just after he left, a participant list being rebuilt), and that null looked exactly like data sent from the Server API. A receiver that has to decide something about the sender (who to mute, whose chat to drop) could not tell "a participant I cannot resolve" from "the server", and a payload can name anyone. senderIdentity is the identity the server stamped on the packet; it is null only for data sent from the Server API, which carries none.

  Nothing else changes: participant is resolved exactly as before, the constructor parameter is optional (existing code that builds a DataReceivedEvent compiles unchanged), and no event, field or default moves.

2.9.8

• The beauty filter survives a recovery re-publish. 2.9.7 added a fallback that re-publishes the source when un-mute fails, but it built the new track bare — so the processor restartTrack carefully preserves on the normal path was dropped on exactly the un-mute that had already gone wrong. The camera came back plain and nothing said why. The processor is now carried across, and a processor that refuses to re-attach is logged rather than allowed to sink the recovery: a camera back without its filter still beats a camera that never comes back.

2.9.7

Un-muting a track could fail and leave it muted for the rest of the session, silently.

• With stopOnMute (the default) un-mute runs restartTrack(), which throws outright when sender == null — after a reconnect takes the sender with it, for instance — and can also throw from getUserMedia when the OS has not finished releasing the capture device. Inside LocalTrack.unmute that throw skipped both enable() and updateMuted(false): the track stayed stopped, the server was never told, and the UI showed a live microphone publishing nothing.

  Measured on a production host: after a DUPLICATE_IDENTITY reconnect, the mic was un-muted and the participant then sat in their own live stream for 365 seconds having published zero seconds of audio — inaudible to their entire room, not only to a PK opponent. The camera showed the same shape in the same session (6 seconds published out of 365).

  setSourceEnabled now catches a failed un-mute, drops the dead publication and re-publishes the source from a fresh track, which does not depend on the old sender. The failure is logged at SEVERE instead of vanishing.

CHANGELOG

2.9.6

• A resume that does not hold is no longer treated as a recovery.

  A subscriber peer connection can fail while the resume that answers it succeeds at the signalling layer. The engine then cleared the pending reconnect, which zeroes reconnectAttempts — and defaultRetryDelaysInMs[0] is 0 ms. So the next failure retried instantly, from a zeroed counter, and the backoff could never grow to reach the full-reconnect escalation.

  Measured on a live room (2026-09-22, Samsung SM-A175F): 263 successful resumes in four minutes, one every 0.88 s, every one carrying RR_SUBSCRIBER_FAILED, until the user closed the room by hand. Audio dropped on every cycle. The session recovered the moment a new connection replaced it — which is now automatic: three resumes that fail again inside ten seconds force a full reconnect.

• The video renderer has one writer for its track instead of five.

  _attach() awaits in the middle, and srcObject was written from the attach itself, the stream-updated listener, didUpdateWidget, the iOS platform-view callback and disposeRenderer. Overlapping attaches resumed out of order, so the loser of the race wrote last. From the same session: the track was attached and then cleared again 16 ms later, leaving the camera black for the remote side until the whole room was rebuilt. Every write now carries the attach generation it was issued under and stale writers are dropped.

• Focus and exposure taps no longer throw into the app's error handler.

  On Android these reach CameraUtils.setFocusPoint, which dereferences the device orientation enum — null before the orientation manager attaches, so it throws NullPointerException: 'int java.lang.Enum.ordinal()'. Unawaited, that surfaced as an unhandled async error, four times in one second on a single preview tap. Focus is a convenience; a handset that refuses it is now logged, not raised.

2.9.5

• preferPlatformVoiceProcessing — hand echo cancellation to the device on handsets where ours measurably does not run.

  Measured on a Xiaomi M2004J7AC (Android 12), with our webrtc fork active and the media profile confirmed applied on every sample: ERL frozen at -29.1 dB and ERLE frozen at 0.18 across 24 consecutive reports — one distinct value, which is an instantiated canceller that never processed anything. On the same build, in the same room, a Samsung SM-A175F reported nine distinct ERLE values and removed up to 12 dB.

  Nothing is left to configure on such a device: hardware cancellation is off, the call capture path is kept, software cancellation is requested and confirmed delivered. It does not run. So that handset gets the platform's canceller — which is what cleaned its audio in the field — and the caller decides per model.

  Media ROUTING is unchanged either way: a listener is still never put in the telephony profile.

2.9.4

• Asks for the one audio configuration that works on every handset, through an option our fork of the webrtc plugin adds: webrtc-utd.

  Upstream exposes a single switch that reaches the Android audio device module, and it bundles five decisions: hardware AEC off, hardware NS off, stereo in, stereo out, and a raw MIC capture source instead of VOICE_COMMUNICATION. We need the first two and none of the last three — the capture source is what breaks software cancellation, because AEC3 aligns the microphone against what was played inside a bounded delay window and VOICE_COMMUNICATION is the low-latency path Android tunes for exactly that.

  Measured on 2026-09-21 across one live room, three handsets, one build: a Samsung SM-A175F cancelling 8-12 dB, a Redmi Note 9 at 0.18-1.3 dB — nothing — and a third reporting no stats at all. The two failing devices also never went quiet on the wire while their user was silent, sending 15-58 kbps of room noise continuously, because the noise suppressor lives in the same module and was not running either.

  Handing the job to the platform is not the safe answer: hardware cancellation is broken on a long tail of devices that advertise it (Signal ships a device blacklist for this), and ZEGO's own documentation says the system canceller is weaker on some Android handsets — which is why their default mode uses their own.

  A build without the fork ignores the unknown option and falls back to the platform canceller, so this is safe to send unconditionally. To pick it up:

  dependency_overrides:
    flutter_webrtc:
      git:
        url: https://github.com/UTD-Stream-Engine/webrtc-utd.git
  

2.9.3

• The platform's own echo canceller does the work again — on the path where it actually runs.

  2.9.1 broke a deadlock from the software side: the Android module announces a built-in canceller on Android 10+, libwebrtc's AEC3 stands down for it, and the built-in one never engaged because the media profile is MODE_NORMAL. Sending bypassVoiceProcessing stopped the module claiming it, so AEC3 could run.

  It ran on some handsets. Measured across one live room on 2026-09-21, three phones on the same build: a Samsung SM-A175F cancelling 8-12 dB, a Redmi M2004J7AC frozen at 0.77 dB — AEC3 present and never converging — and a third reporting no audio-source stats at all. Software cancellation has to align the microphone against what was played inside a bounded delay window, and that window is a property of the device's audio path. Some devices fit it. Some never will, and no amount of tuning on our side changes that.

  So the deadlock is broken from the other side instead. Since 2.9.2 a publisher runs in MODE_IN_COMMUNICATION — the built-in canceller's home — so the module keeps its hardware AEC and its hardware noise suppressor, and the platform cancels on the path every phone vendor tests for its own call app. Where a device reports no built-in canceller, the module does not enable one and AEC3 runs as before: every device ends up with exactly one canceller, and the device's own declaration picks which.

  The cost, stated plainly: where the hardware canceller takes over, the software one stands down and stops reporting, so echoReturnLoss arrives null on those devices. Null now means "the platform is handling it". The device model that ships with every quality report is what identifies a handset that claims a canceller and cancels nothing.

2.9.2

• The Android audio profile now follows the MICROPHONE, not the session. 2.9.1 let the software echo canceller run again, and on one handset it did: ERL ~20 dB, ERLE 6-9 dB, cancelling. On a second handset in the same live battle, on the same build, it sat at ERL -28 dB and ERLE 0.18 dB for the whole session — running, seeing the microphone far louder than the reference it was given, subtracting nothing. When that user muted, the other side was clean: the loop closed at the microphone of the device whose canceller never converged.

  The reason is the media profile itself. MODE_NORMAL plays out over the music path — buffered, high latency, and latency that varies by device. The canceller aligns the microphone against what it played inside a bounded delay window; past that window it cannot subtract anything. One device fits, another does not, and nothing in the app can tell which.

  So a publisher now gets MODE_IN_COMMUNICATION — Android's low-latency voice route, with a delay the canceller can align to — and a listener keeps the media profile, which is what the profile was for: no telephony state, no earpiece routing, other apps keep the microphone. The switch happens as the local audio track appears and reverses when the last one goes, so a listener who takes the microphone mid-session gets it too. That path used to run on iOS only: an Android session kept whatever profile it was given at connect for its entire life.

  The loudspeaker is re-asserted on every switch — the communication profile routes to the earpiece by default, and a broadcaster must not be moved there.

2.9.1

• The media audio profile left Android 10+ with NO echo canceller at all. mediaAudioProfile kept the software APM on at the track, as 2.9.0 intended — but the native audio module is built with setUseHardwareAcousticEchoCanceler(SDK_INT >= Q), so on Android 10+ it announced a built-in AEC and libwebrtc's AEC3 stood down in its favour, while the built-in one never engaged under the profile's MODE_NORMAL + music stream. Each canceller stepped aside for the other.

  Measured on a live PK broadcast on 2026-09-20: 112 quality samples, the media profile applied on every one, the microphone live (audioLevel up to 0.85), and echoReturnLoss / echoReturnLossEnhancement reported on zero of them. Both sides heard themselves come back.

  The module builder exposes no "hardware AEC off, software APM on" switch, so on Android only the media profile now passes the plugin's bypassVoiceProcessing option to the ENGINE — the module stops claiming a built-in AEC — while Native.bypassVoiceProcessing stays false so capture constraints keep requesting echoCancellation / noiseSuppression / autoGainControl. AEC3 then runs. The rest of that option's bundle comes with it: raw MIC capture source, stereo input/output, no hardware NS.

  iOS is untouched: there the same option would bypass the voice-processing unit, which IS the canceller and does engage under the playAndRecord + videoChat recipe.

  Verification is the telemetry, not a listening test: ERL/ERLE arriving at all means a canceller is running.

2.9.0

• Asking for media-style ROUTING was also switching every echo canceller off. mediaAudioProfile: true exists for one thing: keep the app out of the OS telephony profile so a LISTENER is never "in a call" — other apps keep the microphone (a WhatsApp voice note works with the room open), playout routes like music instead of to the earpiece, and the volume buttons move media volume. It was implemented by also passing bypassVoiceProcessing to the native engine, which on Android bundles five audio-device-module build settings (hardware AEC off, hardware NS off, stereo input, stereo output, and MediaRecorder.AudioSource.MIC in place of VOICE_COMMUNICATION), on iOS bypasses the voice-processing IO unit, and — through AudioCaptureOptions being overridden in options.dart — also forces WebRTC's own software APM (AEC/NS/AGC) off. A user on the mic and on the loudspeaker was therefore publishing a raw, unprocessed microphone, and the room heard itself come back.

  Routing and processing are two different decisions and are now spelled separately. mediaAudioProfile declares only the Android media audio attributes; bypassVoiceProcessing keeps its exact previous meaning for callers who ask for it explicitly. Nothing about the listener behaviour the profile exists for changes.

• Known limit, Android only. A device that advertises a built-in AEC makes libwebrtc's APM stand down in its favour, and a built-in AEC may not engage under MODE_NORMAL. Keeping the software APM while suppressing only the hardware advertisement needs an ADM switch flutter_webrtc does not expose, and the Android audio MODE cannot be changed once a room is live: setAndroidAudioConfiguration writes cached fields, and AudioManager.setMode runs only inside AudioSwitch's one-time activate(), which is not reachable again until every peer connection is disposed. On iOS there is no such limit — the voice-processing unit is back and cancelling.

2.8.5

• Uplink audio quality was invisible: every speaker looked perfect. packetsLost, jitter and roundTripTime are not fields of outbound-rtp — they are what the REMOTE end reports about what it received, and live in the linked remote-inbound-rtp entry. The audio sender read all three off outbound-rtp, where they are always null, and the caller turned that null into a zero. Measured on production: 802,417 quality reports with round-trip time null in ALL of them and loss reported as 0.0000 throughout — a speaker losing a fifth of their packets was indistinguishable from a flawless one. Now resolved through remoteId, the same way the video sender already did. With no linked report yet the values stay NULL, so "not known" stays distinguishable from "no loss".

2.8.4

• A replaced subscriber data channel no longer keeps delivering. When the server opened a data channel for a label it had already opened (a subscriber renegotiation), the field was overwritten but the previous channel stayed OPEN with its message handler attached — so from then on every message on that label was handled once per still-attached channel, silently duplicating the room's own state updates and anything an app sends, for the rest of the session. The replaced channel is now detached and closed.
• Fixed an unbounded listener leak on the same path. The subscriber channel's state-change handler registered ANOTHER listener on the publisher channel every time it fired, so both the listener count and the emitted SubscriberDataChannelStateUpdatedEvents grew without bound. It now subscribes once, reports the subscriber channel's own state (it previously reported the publisher's), and the subscription is cancelled when the channel is replaced or the connection is torn down.

2.8.3

• Loud failure when the audio profile can no longer be applied. The native engine is built once per process and flutter_webrtc builds it implicitly (call profile, no options) on the first native call of any kind — device enumeration, a video renderer, a speakerphone toggle, another kit. A UtdmClient.initialize(mediaAudioProfile: true) that arrives after that was silently a no-op: the room then ran in the call profile for the rest of the process, with no runtime way back. initialize now detects it and logs a SEVERE, unmistakable message instead; UtdmClient.engineHasMediaAudioProfile exposes the applied profile so a kit can assert it. Rule for integrators: the kit that owns the audio profile must be the first thing in the app to touch the media engine.

2.8.2

• Android: a listener's first join no longer plays through the earpiece. Under the media profile (mediaAudioProfile / bypassVoiceProcessing) the audio device module was still built with WebRTC's default USAGE_VOICE_COMMUNICATION playout attributes. Android routes that usage with the phone strategy, so under MODE_NORMAL — with nothing having forced the speaker — playout went to the earpiece on a large share of devices until something activated device routing (which only happens once a local audio track exists). Leaving and re-entering "fixed" it, which is exactly how it was reported. initialize now passes media attributes (USAGE_MEDIA / CONTENT_TYPE_SPEECH) so playout routes like music — speaker, A2DP, wired — by platform policy from the first frame. No API change; the call-profile initialize is untouched.

2.8.1

• Fix compile error on older Dart front-ends: publishOptions accesses inside the video negotiate() closure relied on flow-analysis promotion that older SDKs reject after an await ("could not be promoted due to an 'await' or 'yield'"). Read the options once into a non-nullable local instead.

2.8.0

• Add mediaAudioProfile to UtdmClient.initialize: keeps the OS audio session in the media profile (Android MODE_NORMAL + music stream on connect, no Apple voice-processing IO unit — same native layer as bypassVoiceProcessing) while the software audio processing (echoCancellation / noiseSuppression / autoGainControl) keeps following AudioCaptureOptions instead of being forced off. Fixes degraded speaker audio (no AEC/NS/AGC at all) in media-mode rooms. bypassVoiceProcessing semantics are unchanged: media profile AND raw, unprocessed microphone.
• Fix AudioCaptureOptions.toMediaConstraintsMap(): the voiceIsolation constraint was populated from noiseSuppression instead of the voiceIsolation field.

2.7.1

• Fix iOS/macOS build failure: ios/Classes and macos/Classes Swift sources were published as symlink path stubs (plain-text files containing ../../shared_swift/...) instead of real code, breaking pod install builds with "Expressions are not allowed at the top level". Sources are now real files.

2.7.0

• Realtime audio, video and data channels for Flutter across iOS, Android, Web, macOS, Windows and Linux.
• Adaptive stream and dynacast for efficient bandwidth usage.
• Screen sharing on all supported platforms.
• End-to-end encryption for audio/video.
• Declarative, event-driven room and participant APIs.