web library
WebCodecs (+ MSE fallback) codec backend for miniav_tools — browser only.
This is the web half of miniav_tools_codecs, kept as a separate entry so
web apps register the WebCodecs backend WITHOUT pulling in the minigpu
GPU-compute codecs (import package:miniav_tools_codecs/miniav_tools_codecs.dart
for those). Import this only on web (behind a dart.library.js_interop
conditional import) — it uses dart:js_interop / package:web.
Wraps VideoEncoder / VideoDecoder / AudioEncoder / AudioDecoder and
provides a MediaRecorderCapture fallback for browsers lacking WebCodecs.
Also registers the pure-Dart ContainerFramingBackend (WAV/Ogg/ADTS/MP4/M4A
demux+mux, MP3 demux) — it has NO dart:ffi/dart:io, so it is web-safe.
This is what lets openSource(MediaSource.bytes(mp4)) demux a container in
the browser and feed the WebCodecs decoders; without it, container playback
on web had no demuxer to fall through to (there is no FFmpeg on web). MP3 is
the clearest case: the browser can decode it (WebCodecs 'mp3') but nothing
could frame it, so .mp3 bytes had no path to playback on web at all.
Classes
- AudioDecoderConfig
- Configuration for an audio decoder (AAC, Opus, …).
- AudioEncoderConfig
- Configuration for an audio encoder (AAC, Opus, …).
- AudioRingSink
-
An
AudioWorkletplaying from shared memory. - AudioTrackInfo
- BackendContext
- Optional shared resources passed across backend boundaries.
- BackendPreference
- User preference for backend selection on a per-call basis.
- BytesDemuxerInput
- BytesMuxerOutput
- CallbackMuxerOutput
- CodecCapability
- One concrete thing a backend can do: a codec, in one direction, via one path, with a known I/O and cost profile. Immutable + cheap to enumerate.
- CodecExtraData
- Codec-private bitstream extras (SPS/PPS for H.264, codec-private for VP9, etc.) emitted once at stream start. Muxers need this to write track headers.
- CodecQuery
- What a consumer asks the negotiator for. Exactly one of videoCodec / audioCodec / container is set.
- ContainerFramingBackend
-
CpuExecutor<
I, O> - Runs CPU-bound CpuTasks off the main isolate where the platform allows it. Create one via createCpuExecutor and dispose it when finished.
- CpuFrameSource
- CvPixelBufferFrameSource
- D3D11TextureFrameSource
- DecodedAudio
- A chunk of decoded PCM audio.
- DecodedFrame
- A decoded frame. Backends may return CPU bytes or GPU handles depending on DecoderConfig.requestGpuOutput.
- DecoderConfig
- Configuration for a video decoder.
- DemuxerConfig
- Configuration for a demuxer (container file/stream → encoded packets).
- DemuxerInput
- Where a demuxer reads its input.
- DmabufFrameSource
- EncodedPacket
- A single encoded video or audio access unit.
- EncoderConfig
- Configuration for a video encoder.
- ExcludedBackendPreference
- FileDemuxerInput
- FileMuxerOutput
- FrameSource
- Sealed: every frame handed to an encoder is one of the variants below.
- GpuHandleLease
- GpuTextureFrameSource
- HwPreference
- MediaRecorderCapture
-
Lightweight wrapper around the browser
MediaRecorderAPI. - MiniAVAudioBuffer
- MiniAVBuffer
- MiniAVBufferSource
- MiniAVNativeFence
- GPU sync fence information for zero-copy buffer handoff.
- MiniAVToolsBackend
- MiniAVToolsPlatform
- Process-global registry of installed MiniAVToolsBackends.
- MiniAVVideoBuffer
- Video payload of a MiniAVBuffer.
- MuxerConfig
- Configuration for a muxer (encoded packets → container file/stream).
- MuxerOutput
- Where a muxer writes its output.
- PacketMessage
-
One demuxed packet. EOF is
nullrather than an empty packet, so the caller cannot mistake one for the other. - PinnedBackendPreference
- PlatformAudioDecoder
- Abstract audio decoder. Backends return a concrete subclass from MiniAVToolsBackend.createAudioDecoder.
- PlatformAudioEncoder
- Abstract audio encoder. Backends return a concrete subclass from MiniAVToolsBackend.createAudioEncoder.
- PlatformDecoder
- Lifecycle: create → decode* → flush → close
- PlatformDemuxer
- Abstract container demuxer.
- PlatformEncoder
- Abstract video/audio encoder. Backends return a concrete subclass from MiniAVToolsBackend.createEncoder.
- PlatformMuxer
- Abstract container muxer.
- RgbaYuvCoeffs
-
Canonical ×256 fixed-point RGB→YCbCr coefficients — the inverse direction,
same single-source-of-truth contract as YuvRgbCoeffs. Shared by the native
C converter (
frame_convert.cminiav_rgba_to_i420), the pure-Dart converter (rgba_yuv_dart.dart, the web / no-FFI path), and the GPU WGSL converter — all byte-identical. - A ring of interleaved f32 PCM in shared memory.
- StreamDemuxerInput
- A progressive/live container byte stream (fMP4 / MKV / MPEG-TS from the network, a recorder chunk stream, …).
- TrackInfo
- Description of a single track to be written by a muxer.
- TracksMessage
- The demuxer's opening answer: what it found in the container.
- VideoTrackInfo
- WarmupProgress
- A single progress snapshot emitted during backend warmup.
- WasmOpusBackend
- WebAudioFallbackBackend
- WebCapability
- Static helpers that probe the current browser environment.
- WebCodecsBackend
- WebVideoFrameSource
-
A browser WebCodecs
VideoFramewrapped as a FrameSource. - WorkerAudioTrack
- An audio track played entirely off the main thread.
- Yuv420pFrameSource
- Pre-converted planar YUV420P (I420) frame: three tightly-packed u8 planes.
- YuvRgbCoeffs
Enums
- AudioCodec
- Audio codec identifiers.
- CodecDirection
- The direction a capability runs in. Codec caps are encode/decode; container caps are mux (write) / demux (read).
- Container
- Container / file-format identifiers for muxing & demuxing.
- DecodedPixelLayout
- Planar layout + bit depth of a CPU-resident decoded frame's tightly-packed bytes (from DecodedFrame.readBytes). The presenter routes to the matching YUV→RGBA converter; combined with DecodedFrame.isFullRange it fully determines the colour conversion. GPU/browser frames ignore this (they route by DecodedFrame.outputKind / DecodedFrame.webVideoFrame).
- EncoderLevel
- H.264 / HEVC level. Backends interpret this codec-appropriately.
- EncoderProfile
- H.264 / HEVC profile identifiers (a sensible cross-codec subset).
- FrameSourceKind
- Discriminator for FrameSource subtypes — used by backends to declare support without runtime type checks.
- HwAccelPreference
- Hardware acceleration preference.
- HwPath
-
The specific acceleration path a capability runs on.
softwareis the CPU path;hardwareis a generic "some HW path" used only by the default boolean-derived probe — backends that override MiniAVToolsBackend.probe report the exact vendor path so the negotiator can order/exclude by it (e.g. excludeamfon AMD, prefernvdecoverd3d11va). - MiniAVAudioFormat
- MiniAVBufferContentType
- How a MiniAVBuffer's payload is stored.
- MiniAVBufferType
- MiniAVPixelFormat
- Platform-agnostic types for MiniAV platform interface. These are pure Dart types, not FFI structs.
- RateControl
- Rate-control mode for video encoders.
- VideoCodec
-
Video codec identifiers. Backends declare which they support via
MiniAVToolsBackend.supportsEncode /
supportsDecode. - YuvColorMatrix
- YCbCr→RGB matrix of a CPU-resident decoded frame. Combined with DecodedFrame.isFullRange it selects the converter's coefficient set.
Constants
- kControlSlots → const int
- Control-block size, in int32 slots. Sixteen rather than six so the sample data starts well clear of the cursors: a producer and a consumer hammering two adjacent int32s on different cores share a cache line and slow each other down for no reason (false sharing).
- kDataOffsetBytes → const int
- Byte offset of the sample data.
- kDefaultRingDepth → const Duration
- Default depth. The margin a stalled producer gets before anything is heard, and the latency added to a seek — 400 ms buys a very large Flutter hitch for 150 kB of stereo memory.
- kPacketTypeId → const int
- kTracksTypeId → const int
- Type ids. Local to this protocol; ids are per-application.
- kWorkletAssetUrl → const String
- Where the worklet module is served from in a Flutter app.
Functions
-
createCpuExecutor<
I, O> (CpuTask< I, O> task, {String? debugName}) → CpuExecutor<I, O> -
Creates a CpuExecutor for
task, appropriate to the current platform. -
dartI420ToRgba(
Uint8List y, Uint8List u, Uint8List v, int width, int height, {bool fullRange = false, YuvColorMatrix matrix = YuvColorMatrix.bt601, bool bgra = false, int strideY = 0, int strideU = 0, int strideV = 0, Uint8List? out}) → Uint8List -
Convert I420 planes -> packed RGBA8888 (alpha 255), or BGRA8888 with
bgra. Plane strides are in bytes (<=0 = tightly packed). Writes intooutwhen provided (must holdwidth*height*4); otherwise allocates. Byte-identical to the Cminiav_i420_to_rgbafor every matrix/range. -
dartI420ToRgbaAsync(
Uint8List y, Uint8List u, Uint8List v, int width, int height, {bool fullRange = false, YuvColorMatrix matrix = YuvColorMatrix.bt601, bool bgra = false, Uint8List? out, int chunkRows = 32}) → Future< Uint8List> -
dartI420ToRgba that yields to the event loop every
chunkRowsrows. -
dartI422ToRgba(
Uint8List y, Uint8List u, Uint8List v, int width, int height, {bool fullRange = false, YuvColorMatrix matrix = YuvColorMatrix.bt601, bool bgra = false, int strideY = 0, int strideU = 0, int strideV = 0, Uint8List? out}) → Uint8List -
Convert I422 planes (chroma half-width, FULL height — e.g. deinterleaved
YUY2) -> packed RGBA8888/BGRA8888. Same contract as dartI420ToRgba;
byte-identical to the C
miniav_i422_to_rgba. -
dartRgbaToI420(
Uint8List rgba, int width, int height, {bool fullRange = false, YuvColorMatrix matrix = YuvColorMatrix.bt601, bool bgra = false, int srcStrideBytes = 0, Uint8List? outY, Uint8List? outU, Uint8List? outV}) → I420Planes -
Convert packed RGBA8888 (or BGRA8888 with
bgra) -> tightly-packed I420 planes.srcStrideBytesis the source row stride (<=0 = tight, 4*width) — capture APIs (DXGI, V4L2) often pad rows. -
dartRgbaToI420Async(
Uint8List rgba, int width, int height, {bool fullRange = false, YuvColorMatrix matrix = YuvColorMatrix.bt601, bool bgra = false, int srcStrideBytes = 0, Uint8List? outY, Uint8List? outU, Uint8List? outV, int chunkRows = 32}) → Future< I420Planes> -
dartRgbaToI420 that yields to the event loop every
chunkRowsluma rows. -
ensureInitialized(
) → void - Forces the WebCodecs backend registration to run. Call once at startup if you need it registered before any other call touches the tools registry.
-
registerDemuxProtocol(
) → void -
Registers this protocol's decoders. Named by the worker entry so
spawnruns it on BOTH ends — a registry belongs to one isolate, so a registration the host makes does not exist in the worker.
Typedefs
-
CpuTask<
I, O> = O Function(I input) - A pure function executed by a CpuExecutor. MUST be top-level or static.
- I420Planes = ({Uint8List u, Uint8List v, Uint8List y})
-
Tightly-packed I420 planes (chroma dims
(w+1)~/2 x (h+1)~/2).
Exceptions / Errors
- CodecInitException
- A backend was selected but failed to initialise the codec (driver missing, hardware busy, invalid configuration, etc.).
- CodecRuntimeException
- Encoder/decoder is in an error state and must be closed/recreated.
- CpuExecutorException
- Error surfaced by CpuExecutor.run when the task throws (native) — the original error/stack are carried as strings because arbitrary error objects are not always sendable across isolates.
- MiniAVToolsException
- Base class for all miniav_tools exceptions.
- NoBackendForCodecException
- No registered backend can handle the requested codec / container.
- UnsupportedFrameSourceException
- The provided FrameSource kind is not accepted by this backend, and no fallback was possible.