utd_video_effects_kit 0.9.0-dev.1

Real-time video filters and beauty effects for LiveKit Flutter apps — LUT color grading, skin retouch, makeup, accessories, and background blur as a livekit_client TrackProcessor.

Changelog

0.9.0-dev.1 (prerelease — makeup mesh rework + quality/tracking enhancement pass)

Latency instrumentation + adaptive sync delegate (Android)

• DEBUG_LATENCY logcat probe suite (all lines gated on the existing debugLatency flag; the example app now arms it at startup): the async landmarker line gains detect-input dims, detection-interval EMA, a busy-drop count per 30-result window, and landmark age at draw; a new sync-mode line reports the detectForVideo wall time (no queue/callback share, so async − sync isolates MediaPipe dispatch overhead); a new pipeline line reports capture cadence vs frame wall vs per-stage EMAs every ~3 s. debugInfo adds detectIntervalMs.
• CPU-sync delegate pivot: when GPU-delegate sync inference blows the 14 ms budget at the minimum input rung (measured on-device: ~32 ms at every rung on Adreno 710-class GPUs, where the TFLite GPU delegate can fall back to its slow GL path and the blendshapes subgraph forces a GPU→CPU readback per frame), the ladder now retries frame-exact sync on the CPU/XNNPACK delegate under the same budget guard before surrendering to async+prediction. The delegate preference alternates across backoff retries (a transient stall can't permanently pin the session), GPU stays as the create-fallback, and debugInfo adds syncCpuPivot.

On top of the v0.9 canonical-UV mesh-makeup rework and frame-exact sync tracking already on this branch, a full enhancement pass over filters quality and tracking (both platforms; parity-spec'd in test/parity/ where math changed):

Tracking / degrade honesty

• Rescue-sync detect ladder actually works now: the 384→320→256 input ladder produced the identical bitmap at every rung on 720p/1080p/VGA (a fixed short-side≥256 clamp dominated the integer decimation). Deeper rungs now lower the floor in lockstep (256/224/192) with fractional sampling; rung 0 is bit-exact with previous behavior.
• Sync budget guard gains a sustained-EMA trip (bimodal-latency devices alternating just-under/just-over never accumulated 90 consecutive over-budget frames), a step-back-up after ~10 s of headroom (a thermal spike no longer pins a session at 256 px), and rung memory across the async latch (retries resume at the last surviving rung instead of re-burning the whole walk). debugInfo adds syncDetectShort.
• The degrade ladder never deletes the look, smoothing edition: iOS L2 no longer swaps the smoothing operator to CINoiseReduction (the documented "plastic" tell — and permanent on the Budget tier, whose floor is L2); the bilateral now ring-caps 3→2→1 on both platforms (skinLowpass gains a ringCap arg), and L3 coarsens instead of deleting smoothing.
• Head-pose + blendshape channels get a speed-adaptive publish-time EMA (raw per-frame noise wandered the gloss catch-light; partial blinks flickered the SDF liner).

Makeup mesh quality

• Fold-over culling: triangles whose projected winding inverts vs their canonical-UV winding (relative to the frame-global, mirror-safe parity sum) are skipped — folded far-cheek/eye triangles at 35–50° yaw were stacking Multiply/SoftLight pigment twice at the silhouette.
• Coarse-mode block lattice is frame-anchored: per-triangle phase caused double-blend lines and pigment holes along shared edges at softStep ≥ 2.
• Lips: hue-preserving headroom clip (whole-delta rescale instead of per-channel clamps that drifted saturated reds toward orange) + smooth teeth guard weighting the whole styled delta (no more lipstick/tooth pop).
• Blush: tanh soft-saturation replaces the min(1,·) clamp that flattened the inner ~75 % of the authored falloff into a "sticker" disc.
• Eyeshadow: authored 3-zone gradation restored (crease = midpoint of the two product tints); shimmer flakes regain the smoothstep population ramp (binary subpixel sparkles scintillated under motion).
• Brows can lighten: Soft-Light when the tint is lighter than the per-side P40 hair reference (Multiply is darken-only).
• Makeup tints are graded through the active LUT (× filterIntensity) — makeup composites after the grade, so stylized LUTs never touched product chroma (saturated red lips over a noir grade read pasted-on).

Filters / color

• Tetrahedral 3D-LUT sampling on the Android CPU path (4 fetches vs 8, no diagonal desaturation) — gated OFF behind debug key tetraLut pending the renderTestFrame parity A/B (iOS CIColorCube is fixed trilinear).

Segmentation / background

• Person mask now sources MediaPipe's confidence mask (true soft hair edges) with a latched polarity self-check against the category mask; edge-aware joint-bilateral mask upsampling; mask-normalized blur (kills the person-color halo ring); feather-width parity that scales with blur strength.
• No-person policy: background blur used to fail open (sharp background) when the user left frame or seg init failed — now NoPersonPolicy.blurAll by default (configurable on setBackgroundBlur).
• New setBackgroundDim(amount) — background desaturate+darken "spotlight" riding the same mask (cheaper than blur, composes with it).
• New setBackgroundImage(path) — virtual background, cover-fit and oriented through the same inverse mapping as the mask; decode failure degrades to blur/sharp per policy.

Skin retouch scope

• iOS bilateral low-pass crops to the face box (×1.4) like Android when only smoothing consumes it (was smoothing neck/hands frame-wide).
• Lips + brows are rasterized into a quarter-res exclusion mask and excluded from skin smoothing on both platforms (high smoothing ate lip edges and brow hairs through the chroma/luma gates).

Perf / telemetry / harness

• perfStats splits render-only vs sync-inference time (new renderMeanMs/renderP95Ms/syncMeanMs + render histogram) — the §9 GPU/SIMD go/no-go gate no longer reads contaminated wall time. Per-stage timing EMAs exposed as stageMs. Frame/budget/recover thresholds scale with a capture-interval EMA (30 fps was hard-coded; low-light 15–20 fps over-degraded).
• Decision-free CPU wins (byte-identical output): pooled I420 output planes (was a fresh ~1.4 MB native allocation per frame) and row-banded grade/low-pass across two threads on ≥4-core devices (rowBand debug key).
• renderTestFrame gains optional faceLandmarks — the harness can now exercise the real makeup path with a synthetic snapshot on both platforms.

0.8.2 (device-test round 2 — tracking latency at the root, makeup polish)

Round-2 pass on the two issues still visible after v0.8.1 on device (blur + preview lag confirmed fixed there). Both platforms; builds green.

Fixed / Changed

• Makeup still trailing during head motion — attacked at the root this time:
  • The MediaPipe face landmarker now runs on the GPU delegate by default (PREFER_GPU/preferGpu = true). Detector latency is the dominant term in the trail, and the GPU delegate is the only lever that shrinks it rather than hiding it behind extrapolation. Made safe to default: create-failure already fell back to CPU, and a NEW auto-fallback rebuilds on CPU after 3 consecutive in-inference GPU errors (previously a flaky GPU driver would just count misses forever). Runtime-flippable from Diagnostics (preferGpu); the Diagnostics readout shows the active delegate + latency.
  • Presence confidence 0.5 → 0.35 and tracking confidence 0.4 → 0.3: on device, motion blur dipped presence under 0.5, causing a miss → coast → snap-back cycle — exactly the "leaves the face, then comes back" read. A numFaces=1 selfie pipeline has little false-positive exposure; detection stays strict at 0.5.
  • Makeup-mask temporal EMA reaches full passthrough at moderate motion (MASK_MOVE_HI 0.018 → 0.012) — it was adding ~half a frame of trail on top of landmark staleness in the mid band.
• Makeup quality:
  • Contour densification CONTOUR_SEG 4 → 6 — lip/eyeshadow edges no longer read faceted at selfie distance.
  • Eyeshadow material upgraded to two-octave value noise with depth 0.15 → 0.22 — reads as pressed powder instead of smooth blotches (bit-identical math on both platforms).

0.8.1 (first-device-test fixes — blur polarity/rotation, tracking lag, accessory fit)

Fixes for the five issues found in the first real Android device test (2026-07-05). All are root-caused to code, fixed on BOTH platforms, and build green; they still want one confirming device pass (Diagnostics screen → seg-mask visualizer + perf stats).

Fixed

• Background blur blurred the person, not the background — two stacked bugs, both present since the feature shipped (never device-validated):
  • The default binary selfie_segmenter.tflite category mask is inverted (MediaPipe emits 0 = person / 255 = background for single-channel models); the pipeline assumed != 0 → person on both platforms. The mask polarity is now keyed on the loaded model and normalized at extraction (multiclass masks were already correct and keep their class indices).
  • With setRotationDegrees, MediaPipe returns the mask with unrotated dims but upright (rotated, stretched) content; the dims-swap heuristic could not detect this third convention and sampled the mask sideways. The snapshot's recorded rotation is now authoritative on both platforms (normalized sampling is dims-agnostic, so both MediaPipe conventions map correctly).
  • Soft-mask EMA is now asymmetric (attack 0.7 / release 0.35, was 0.45 flat) and the Android solid-person threshold dropped 250 → 235: a person entering frame reaches full sharpness in ~2 segmentation results instead of ~7.
• Makeup detached from the face during head motion — the per-landmark extrapolation cap 0.18·faceW could not cover real on-device landmark staleness. Raised to 0.30·faceW (+ maxDt ceiling 0.18 → 0.22 s) on both platforms, and FACE detection is now exempt from the degrade-ladder stride throttle (drop-don't-queue already self-paces it at true detector latency; seg/pose keep the stride).
• Whole preview lagged — the frame-budget ladder escalated on the FIRST over-budget frame and took ~1 s to recover one level, so a single GC hiccup latched permanent degradation. Now requires 3 consecutive over-budget frames. The Android blur composite also precomputes its mask-sampling taps (removes per-pixel rotation math from the hot loop).
• Makeup quality — the v0.8 eyeshadow lash-stop clamp painted the darkest color2 at full density across the visible under-eye strip; pigment/shimmer now fade out below the upper-lash line. The eye-opening carve is a true boolean DIFFERENCE (Android Path.op, iOS 16+ CGPath.subtracting) instead of even-odd, killing inverted-overlap flicker slivers at the eye corners. Shimmer is shed at degrade level 2 instead of 1, so mid devices keep sparkle.
• Accessories sized/positioned wrong — the June head-width retune assumed the temple pair 127↔356 is ~30% wider than the cheek pair; both actually sit on the same face-oval silhouette ring, so every head item rendered ~15% small and low. headWidthK 1.08 → 1.22, now live-tunable from the diagnostics debug channel (setDebugTunables({'headWidthK': 1.22})). Accessories also stay fully opaque until tracking confidence drops below ⅔ (was a linear fade that ghosted on every routine detection miss).

Added

• setDebugTunables(Map<String, double>) — numeric diagnostics knobs (currently headWidthK), added as a concrete no-op-default method so existing VideoEffectsProcessor implementers keep compiling; setDebugFlags keeps its Map<String, bool> signature unchanged.

0.8.0 (deep-audit hardening — leaks, races, makeup correctness, parity)

A dedicated hardening release: a two-pass adversarial audit of the whole codebase (DEEP_AUDIT_2026-07-05.md) produced ~87 confirmed findings + 9 enhancements, all fixed across both platforms (Android Kotlin + iOS Swift). Both platforms build green and the parity/unit suites pass; the changed default paths (iOS bilateral smoothing, unified effect order) still want one on-device A/B pass.

Fixed

• Critical Android output-buffer leak — the re-encoded I420 output buffer was never released, so continuous effects leaked one native buffer per frame until OOM. The previous frame's buffer is now released at the top of the next onFrame.
• iOS cross-thread races — model paths, effect params, and the accessory image were read/written from the capture, result, and platform threads without synchronization; all three are now lock-protected (NSLock), matching the audit's race classes.
• Hot-restart / engine-detach teardown (both platforms) — a Flutter hot restart no longer strands enabled native sessions with live MediaPipe handles: stale sessions are swept on attach and engine teardown disposes everything.
• Eyeshadow no longer tints the open eye — the eye opening is carved out of the lid polygon with an even-odd fill, and the dark gradient stop now sits at the upper lash line instead of bleeding over the eyeball.
• Under-eye brightening + highlight placement are face-rotation-aware — the ellipses now rotate with the face axis instead of assuming an upright head.
• Cat/bunny ears anchor under head roll — head-top accessories follow the head axis (headTopAnchor) instead of drifting sideways at ~25° roll.
• Accessories fade with tracking confidence — overlays ease out with the same confidence fade as makeup instead of popping in/out on momentary track loss.
• Makeup geometry scales with face size, not frame size — feather radii, brush sizes and corner radii are based on the measured face width, so makeup no longer thins or blows out as the subject moves toward/away from the camera.
• Reduced pigment thinning on deeper skin tones — the catch-light gate is now region-local (each region's own bright outliers), so baseline luma no longer thins blush/lip/shadow pigment on darker skin.
• lengbai skin LUT rebuilt lossless — the shipped filter_skin.png was a corrupted JPEG-recompressed texture; rebuilt from source, plus 12 LUT PNGs stripped of stray color-profile chunks that could shift the grade on load.
• Dependency + packaging hygiene — livekit_client/flutter_webrtc ranges are bounded to the validated WebRTC pin set (an open range could hard-fail pod install or swap the org.webrtc class set); new android/consumer-rules.pro keeps MediaPipe/protobuf through consumer R8; CI publish hardened (OIDC token acquired only after tests, SHA-pinned actions, a packaging gate, and main-only dispatch).

Changed

• Canonical effect order unified across platforms: grade → background blur → makeup → warp → glow/grain/vignette. Android is the reference; iOS now composites its color-post tail over makeup and the warped face in the same order.
• iOS skin smoothing now defaults to the Android-parity bilateral path (useBilateralSmooth = true; the revert flag is retained via setDebugFlags — the on-device A/B is still the pending confirmation).
• iOS bloom/vignette/grain ported to Android's curves — shared amount-adaptive bloom knee (thresh = 0.72 − 0.30·amount), same vignette falloff, same grain response.
• Blend rounding unified — color blends round (+0.5) instead of truncating, killing a systematic half-LSB darkening on Android.
• Background-blur strength curve aligned — Android now maps amount → sigma·14 like iOS's CIGaussianBlur, and frames with no person in the mask are no longer blurred (previously the whole frame blurred when segmentation found nobody).

Added

• EffectsState.fromMap + VideoEffectsProcessor.apply(state) — round-trippable state serialization and a bulk one-push apply (saved looks, A/B switching).
• MakeupLook extended to the full engine surface: brows, eyeliner (+style), eyeshadowColor2 + shimmer, lipStyle, contour, highlight — authored looks can now set-or-clear every makeup field.
• Adaptive glow knee — bloom's bright-pass threshold now scales with the amount, so low glow reads as subtle diffusion instead of nothing.
• Film-grain midtone response + 2-px cells — grain peaks in midtones and fades in deep shadows/highlights, with 2-px grain cells at 720p+ so it reads like film, not per-pixel noise.
• Slider-drag throttling — setter bursts coalesce into one trailing-edge native push per 70 ms instead of a method-channel call per pointer-move.
• pub.dev screenshots (screenshots/, wired in pubspec.yaml).

Performance

• Per-frame allocation churn removed on the face path (scratch buffers reused).
• Accessory compositing is bounded to the accessory bbox instead of the full frame.
• Warp inner loop reordered — the cheap sub-pixel skip now runs before the expensive contour gate (millions of ops/frame saved when reshape is subtle).
• One shared detection downscale sample per frame feeds all detectors.
• Grain + vignette fused into a single read-modify-write pass (Android).
• Sliding-window mask blur on iOS (O(w) per row instead of O(w·r)).

Deferred (tracked, not in this release)

Tetrahedral LUT interpolation (cross-platform parity work), hair occlusion for accessories, optional model packaging (slimming the ~26 MB asset set), and on-device validation of the new default paths (iOS bilateral smoothing, unified order).

0.7.0 (tracking robustness + body pose + TikTok presets + diagnostics)

Four themes: make face tracking robust through losses and re-acquires, add body tracking (segmentation polish + an optional pose landmarker), push the filter stack to a TikTok-grade one-tap experience with 12 new looks, and make every device-gated toggle validatable from a new in-app diagnostics screen. All changes cross-platform (Android Kotlin + iOS Swift); the new portable math is locked by flutter test parity cases. Both platforms build green.

Face tracking — robustness (WS2)

• Coast velocity-glide + re-acquire blend (COAST_GLIDE, default on). On track loss the published pose now advances along an exponentially decaying velocity (0.7/cycle, total travel ≈ 3.3 velocity-intervals, displacement-capped at ±0.18·faceW from the last-good pose) with an advanced timestamp — so predicted() no longer freezes mid-coast. When the face is re-acquired the overlay morphs back over 3 detections instead of popping; a genuine subject jump (> 0.5·faceW) still snaps immediately.
• Size-scaled One-Euro cutoff (SIZE_SCALED_CUTOFF, default on): minCutoff scales with faceW / 0.25 (beta inversely), clamped 0.5–2 — near faces stop lagging, far faces stop shimmering, at the tuned behavior exactly for a mid-distance face.
• Pose-sign self-validation (AUTO_SIGN_LOCK + GEO_ROLL, default on). Roll is now geometric (temple-line atan2 in landmark space — no device-dependent sign convention at all). Yaw/pitch matrix signs are auto-locked at runtime by correlating against nose-offset proxies (30 votes, 0.7 agreement, 8° gate) so the gloss key-light and shimmer band move the right way on every device/camera; bake the static *_SIGNs after the diagnostics screen confirms.
• Anti-overshoot on direction reversal: the accel EMA is damped 4× on a per-channel velocity sign flip, and the ½·a·dt² term is additionally clamped to |v·dt| so a stale acceleration can never reverse the direction of travel.
• Hardening: renderTestFrame now refuses (returns null) while the session is enabled instead of racing the live grade scratch; SelfieSegmenter releases inFlight only after mask publish (same race class as the fixed M1).

Body tracking (WS3)

• Segmentation polish: sideways frames (portrait 90/270) are now segmented upright (SEG_ROTATE) with consumers self-adapting to either MediaPipe mask-space convention; the person mask is temporally EMA'd (MASK_EMA_SEG, α=0.45) into a soft 0–255 mask; and the background-blur edge is feathered (Android bilinear lerp, iOS mask blur) — no more hard blocky flickering cutout. Raw category mask kept for the face-skin gate.
• Optional body-pose tracking (EffectsState.poseTracking, default off): MediaPipe PoseLandmarker (33 landmarks, bundled ~5.8 MB lite model) behind the same LIVE_STREAM / Kalman+One-Euro / coast discipline as faces (pose-tuned constants; per-channel v·dt prediction capped on shoulder width — no rigid fit for an articulated body). Pull-based Dart surface: VideoEffectsProcessor.poseLandmarks() → PoseFrame for skeleton overlays; shed entirely at degrade level 4.

Filters — TikTok-grade look (WS4)

• 12 new in-house LUT filters (cold_brew, paris, ibiza_pop, y2k_chrome, vhs_warm, noir_soft, noir_high, peach_soda, malibu, espresso, icy_blue, rosewood) generated by the rebuilt in-repo tool/lutgen.py color-science pipeline, with previews rendered through the real glow/vignette post tail (tool/render_previews.py).
• One-tap filter presets (VideoEffectsPresets, applyPreset) — 12 curated looks (Clean Glow, Golden Hour, Paris, Peach Soda, Ibiza Pop, Soft Matte, Cold Brew, Tokyo Night, Classic Cinema, Y2K, VHS, Film Noir), each an atomic stack of LUT + intensity + smoothing + clarity + glow + grain + vignette + whitening in ONE native push. New leading Presets tab in VideoEffectsSheet.
• iOS operator unification (4.3): new claritySigned CIKernel runs Android's exact signed freq-sep clarity (out = orig + (orig − lp)·cl in sRGB) off the same bilateral low-pass; both bilateral operators gain a degrade fallback (level ≥ 2 → cheap paths) and are now runtime-flippable for the on-device A/B (useBilateralSmooth default still off pending the renderTestFrame verdict).

Diagnostics (WS5)

• Runtime debug channel: setDebugFlags (preferGpu with safe landmarker rebuild, debugPose, autoSignLock, coastGlide, sizeScaledCutoff, segRotate, maskEmaSeg, bilateralSmooth/bilateralClarity, resampleColorDiag), getDebugInfo (delegate, latency, pose + geometric proxies + locked signs, iOS centre-pixel colour sample), getSegMask.
• Example-app Diagnostics screen: render-parity runner (deterministic test frame + FrameDiff), perf capture with the §9 GO/NO-GO verdict, head-pose sign check, iOS colour check, segmentation-mask visualizer, GPU-delegate latency A/B, pose-skeleton overlay — every device-gated decision from IMPROVEMENT_PLAN_OSS.md is now a one-tap check.

Tests

• New test/parity/parity_tracking_math(.dart|_test.dart) (coast glide schedule, re-acquire weights, cutoff scale, geometric roll, sign-lock decision, reversal damping, accel clamp, segmentation rotation mapping) + claritySigned cases in parity_color_math + preset catalog/composition tests + poseTracking state tests.

0.6.0 (tracking accuracy + makeup-quality pass)

Three themes: make face overlays track the live face with no perceptible delay, lift makeup realism, and gate effects behind a paid, per-platform entitlement. All changes are cross-platform (Android Kotlin + iOS Swift) and the new portable math is locked by flutter test parity cases. Both platforms build green.

Video Effects entitlement (activation + billing)

• VideoEffectsProcessor.create({entitled}) — effects now run only when the customer has activated + paid for the current platform. When entitled is false the processor is a forced passthrough: init() skips the native attach and _apply() never calls setEffects (zero pixel work), and VideoEffectsSheet shows an "activate to unlock" banner with disabled controls. Defaults to true (backward compatible). The trusted value is the engine-signed videoEffects claim in the LiveKit token (resolved server-side against the requesting platform), surfaced by utd_stream_sdk / utd_live_room_kit — never a flag the host app invents. Cross-repo spec: VIDEO_EFFECTS_ENTITLEMENT_PLAN.md.

Face tracking — lower latency, more accurate

• Unified render-time predictor (FaceLandmarks.predicted, both platforms). The old per-channel pos + v·dt extrapolation is replaced by a four-part model (each toggleable for on-device A/B; all-off reproduces the legacy predictor):
  • Self-consistent base — blends the One-Euro display position toward the near-raw Kalman posterior as the face moves, so v·dt starts from the same frame the velocity refers to (kills rest→move onset lag; strict no-op at rest).
  • Damped acceleration — adds a ½·a·dt² term (a = EMA finite-difference of the Kalman velocity) so the overlay anticipates motion onset / direction reversal instead of trailing it. Faded out while coasting; capped to half the offset budget.
  • Rigid + residual — fits one 2D similarity transform (translate + rotate + uniform scale) over stable anchor landmarks and moves the whole mesh rigidly, adding only a tightly-capped per-landmark residual (blink/jaw/brow). Fast head turns now lag rigidly instead of shearing or "snapping back". Degenerate fits fall back to the per-channel path.
  • dt ≤ 0 hardening — a stalled/rounded camera timestamp no longer disables prediction (it falls back to a one-interval look-ahead).
• iOS detection no longer stalls the grade — the per-frame YUV→BGRA detection-input render moved to a dedicated low-priority CIContext, so it stops serializing behind the color-grade render on the capture thread (iOS detection is CPU-only, so this is real lag).

Makeup — realism

• Linear-light compositing ON (LINEAR_BLEND/linearBlend = true). Soft-Light / Multiply / Screen / Color blends now run in linear light (physically-correct mid-tones). Composite math is parity-locked; the only on-device follow-up is opacity calibration (translucent eyeshadow/blush may want a small bump in the Dart presets).
• Smooth contours — lips / eyes / eyeshadow / brows are densified with a centripetal Catmull-Rom spline so close-up makeup edges read as organic curves, not faceted polygons.
• Lip body fullness — the default full/satin lip holds fuller colour in the body and eases toward the vermillion border (no longer a flat sticker).
• Roll-stable eyeshadow — the crease→lash colour + density now follow the lid-local (roll-aligned) axis instead of screen-vertical Y, so they rotate with the head on a tilt.
• Steadier skin-tone gates — the per-face skin-luminance reference is bilinearly sampled (2×2) instead of one nearest pixel, de-noising the brow/contour/blush/shimmer gates.
• iOS↔Android colour parity — filterIntensity now blends in sRGB (matches Android's byte lerp; was linear), and film grain is an additive-monochrome kernel (matches Android; was Overlay-blended). A bilateral frequency-separation skin-smoothing kernel (operator-match with Android's, replacing CINoiseReduction) is implemented but gated off (useBilateralSmooth) pending one on-device renderTestFrame A/B.

0.5.0 (built-in beauty & filters UI)

A drop-in modal sheet so apps no longer hand-roll their own effect picker.

• New VideoEffectsSheet — VideoEffectsSheet.show(context, processor:) opens a self-contained beauty & filters bottom sheet that drives a VideoEffectsProcessor. Full parity with the catalogs: Beauty sliders (smoothing / whitening / background-blur), LUT color Filters + skin-tone presets, Makeup looks + per-region shades (eye color / lipstick / eyeshadow / blush), and Accessories (every VideoEffectsAccessories PNG overlay).
• Category tabs — the controls are grouped into four tabs (Beauty / Filters / Makeup / Accessories) instead of one long scroll, and the sheet is sized to 1/4 of the screen height (heightFactor, default 0.25; floored to a usable minimum so the tabs still fit on short phones).
• Zero extra deps — pure Flutter Material; no flutter_screenutil or room-kit theme. The accent color comes from Theme.of(context).colorScheme.primary; language (EN/AR) + RTL are picked up from the ambient Localizations/Directionality. Catalogs and preview assets are read straight from the kit, so future entries appear automatically.
• The sheet shows a "preview only" hint and previews the UI without altering the video when VideoEffectsProcessor.isSupported == false.

See EFFECTS_SHEET_MIGRATION.md for the upgrade/migration guide.

0.4.0 (TikTok-style filters — color post + skin retouch)

The first three slices of the "make filters look like TikTok" roadmap (see TIKTOK_FILTERS_PLAN.md): color post (LUT strength + glow/grain/vignette), skin retouch (frequency-separation smoothing + selective brightening), and makeup richness (gradient eyeshadow, lip styles, brows). All new effects are opt-in (default off / neutral).

Color post

• Fixed: filterIntensity was ignored natively. The LUT strength was plumbed from Dart all the way to applyState but dropped — Android hardcoded the LUT mix to 100% (CpuEffects / GlEffects) and iOS CIColorCube has no opacity. Now honored on both platforms (iOS dissolves the graded image over the original; Android blends per-pixel), so setFilter(key, intensity:) actually dials the grade 0..1.
• New color-post tail (the "TikTok look"), applied after the grade + face overlays:
  • Glow / bloom (Orton) — EffectsState.glow / setGlow — bright areas are blurred and screen-blended back for the dreamy beauty-filter diffusion. The single highest-impact add. (iOS: CoreImage highlight-biased blur + screen; Android: 1/4-res highlight bright-pass + separable box blur + screen.)
  • Film grain — EffectsState.grain / setGrain — animated per frame so it shimmers like real grain (monochrome/tonal).
  • Vignette — EffectsState.vignette / setVignette — gentle radial corner darkening to focus the face.
• Three new TikTok base-grade LUTs: clean_girl (warm low-contrast "clean girl glow"), cool_aesthetic (VSCO-A6 cool faded), latte (muted-brown latte trend). Filter catalog now 46.
• Tuned three in-house LUTs: softer teal_orange (less "radioactive skin"), rolled-highlight golden_hour, tightened faded_film. Previews regenerated.

Skin retouch

• Frequency-separation skin smoothing — replaces the old "plastic" smoothing.
  • Android (CpuEffects): the smoother is now a pure edge-preserving low-pass (luminance-weighted bilateral, widened to 3 rings / 24 taps); gradeRgb recombines lowpass + (orig-lowpass)*textureDetail, skin-gated by the YCbCr mask so only skin is smoothed (not hair/background).
  • iOS (LutVideoProcessor): the plastic CIGaussianBlur is replaced with edge-preserving CINoiseReduction (+ a texture-limited mild blur at high strength), so pores/edges survive.
  • New EffectsState.textureDetail / setTextureDetail (0 = plastic, 1 = full texture; default 0.5).
• Signed clarity / local contrast — EffectsState.clarity / setClarity (-1..1): positive = punchy pop (unsharp), negative = soft "clean girl" skin.
• Selective brightening (landmark-masked, both platforms in FaceEffects):
  • Teeth whitening — teethWhitening / setTeethWhitening: desaturates yellow + lifts inside the inner-lip opening, gated to bright low-saturation pixels (gums/tongue/lips spared).
  • Eye-white (sclera) brightening — eyeBrightening / setEyeBrightening: brightens + de-reddens the sclera (eye opening minus iris), EAR-gated so it hides on blink.
  • Under-eye brightening — underEyeBrightening / setUnderEyeBrightening: lifts the tear-trough via a heavily-feathered ellipse below each eye.
• Example app: new Filter-strength, Glow, Grain, Vignette, Keep-texture, Clarity (signed), and Retouch (teeth / eye / under-eye) sliders.
• Minor: iOS bloom (glow) blur radius is now orientation-stable (scales by the short side) to better match the Android bloom spread.

Makeup richness

• Two-tone gradient eyeshadow + shimmer — setEyeshadow(hex, color2:, shimmer:) / EffectsState.eyeshadowColor2 + eyeshadowShimmer. The pigment now lerps from the crease color (top) to a darker lash-line color (bottom) instead of one flat tone; shimmer adds a luminance-gated satin sheen (no random noise, so it tracks the lid).
• Lip styles + better gloss — setLipstick(hex, ..., style:) / EffectsState.lipStyle: full (uniform), ombre (deeper at the outer edge), blurred (soft "just-bitten" stain, matte). The specular sheen is now a directional wet band just below the lip centre rather than a flat additive term.
• Brow fill/darken — setBrows(hex, opacity:) / EffectsState.brows + browOpacity: Multiply toward a brow color, hair-gated (only darkens pixels darker than skin) so it fills/defines brows without painting the skin between hairs. (Brow landmark polygons are candidate indices — validate on device.)
• Example app: new Lip-style chips, Eyeshadow shimmer slider + 2-tone toggle, and a Brows color picker.

Sculpt + eyeliner

• Eyeliner — setEyeliner(hex, opacity:, style:) / EffectsState.eyeliner + eyelinerOpacity + eyelinerStyle. A procedural tapered sliver along the upper lash line (thin at the inner corner, thicker toward the outer), blended Multiply toward the liner color and hidden on blink (EAR-gated). style: 'wing' adds an up-and-out flick at the outer corner; 'classic' is the plain stroke.
• Contour + highlight (dodge/burn sculpt) — setContour / setHighlight / EffectsState.contour + highlight. Soft oriented-ellipse brushes derived from reliable landmarks: contour burns a cool-gray shadow under the cheekbones (face-side → mouth-corner) and along the nose sides; highlight dodges a warm-white sheen on the nose bridge, cheekbone tops, cupid's bow, chin and brow-bone centre. Both are low-opacity, very soft, and skin-gated (won't paint hair/nostrils/lashes). Applied as a base layer under the makeup overlays.
• Example app: new Sculpt section (Contour / Highlight sliders) and an Eyeliner color picker + classic/wing chips.
• ⚠️ Eyeliner lash-line + brow polygons use candidate MediaPipe indices — validate on device; the EAR/hair gates keep an imperfect polygon safe.

Reshape (liquify)

• Landmark-driven face reshape — setEyeEnlarge / setFaceSlim / setNoseNarrow / setLipPlump / setChinShorten (+ matching EffectsState fields). A real-time CPU backward-map warp: for each output pixel inside a feathered face ellipse, a small sampling offset is accumulated from the active ops, gated by the ellipse mask (so background/hair never drag), clamped to <10% of the face width, then the frame is bilinear-resampled from an unmodified copy at the displaced location. Runs after the makeup pass so makeup/accessories warp along with the face. Eye-enlarge + lip-plump are radial bulges; nose-narrow / face-slim compress toward a centre line; chin-shorten lifts the chin.
• Same CPU algorithm on iOS (FaceWarp.swift) and Android (FaceWarp.kt) — exact parity, no new GPU backend (iOS has no CoreImage warp primitive). Consumes the One-Euro-smoothed landmarks so it doesn't jitter; sub-pixel offsets are skipped so everything outside the warp stays pixel-exact.
• Example app: new Reshape section with five sliders.
• ⚠️ Conservative caps + the elliptical face gate keep the warp safe; tune the feel on a real device.

Deferred to a follow-up: iOS skin-tone dual-LUT parity (needs a CIColorKernel skin mask); a FACE_OVAL polygon gate (vs the current ellipse) if the ellipse proves too loose; and multi-feature MLS coherence (the current model is per-feature additive offsets — fine for independent features at small deltas).

0.3.0 (more accessories, LUTs, curated makeup looks)

• Accessory art upgraded to realistic 3D renders. Replaced the placeholder glasses / crown / cat_ears PNGs with transparent, high-quality renders (generated via Vertex AI Imagen, matted with rembg).
• Six new accessories: sunglasses, flowerCrown, partyHat, bunnyEars, mustache, halo. Each is wired end-to-end:
  • Dart: EffectsState toggles + VideoEffectsProcessor setters (setSunglasses, setFlowerCrown, setPartyHat, setBunnyEars, setMustache, setHalo) + VideoEffectsAccessories catalog entries.
  • Native Android/iOS: lazy-loaded bitmaps and landmark-anchored draw passes (e.g. mustache at the philtrum, halo floating above the crown of the head).
• Calibration note: the new accessories ship with sensible widthScale / topFrac defaults mirroring the existing ones; fine-tune per device in FaceEffects.draw* if needed.
• 16 new color filters (LUTs) generated with color science as clean 64-level square LUTs (correct on both the Android GL and iOS CIColorCube paths): teal_orange, golden_hour, sunset_glow, vibrant_pop, crisp_clean, vintage_warm, faded_film, matte_fade, moody_blue, cold_winter, dreamy_pink, emerald, cyberpunk, sepia, noir_bw, silver_bw (filter catalog now 43).
• All filter previews re-rendered at 512² (was 256²) from a new high-quality reference portrait (Imagen), including the 5 skin-tone previews (dual-LUT skin mask). Existing LUTs were audited — all valid; none rebuilt (infrared white-clips by design).
• Curated makeup catalog (VideoEffectsMakeup): realistic lipstick / eyeshadow / blush / eye-color shades plus 6 coordinated full looks (natural, glam, soft_rose, bold_berry, sunkissed, smoky_night), each with an AI-rendered preview thumbnail. Apply a whole look in one call with VideoEffectsProcessor.setMakeupLook.
• More realistic makeup rendering: new lip finish (matte / satin / gloss, default satin) via setLipstick(..., finish:) + EffectsState.lipFinish, wired through Android & iOS drawLips; softer Gaussian-style blush falloff on both platforms (removes the visible ring).

0.2.0 (face effects — eye color, makeup, accessories, background blur)

• New face-effect pass driven by MediaPipe FaceLandmarker (478 landmarks), adapted into the existing in-place TrackProcessor frame seam (no per-frame bytes cross the method channel):
  • Eye color — iris recolor (landmarks 468–477) with a soft radial falloff.
  • Makeup — lipstick / eyeshadow / blush (landmark-region fills + gradients).
  • Accessories — glasses / crown / cat-ears PNG overlays, scaled/rotated from the temple + nose landmarks.
• Background blur finally implemented (the long-declared backgroundBlur field): MediaPipe selfie segmentation → blur the background, keep the person sharp. Android = CPU downscale box-blur over the frame; iOS = CIBlendWithMask.
• Detection runs async (LIVE_STREAM) off the capture thread with a cached, atomically-published landmark/mask snapshot — onFrame runs under the lock that gates frame delivery, so it never blocks the stream (≈1 frame of lag).
• Dart API (additive, backward compatible): flat EffectsState fields (eyeColor/lipstick/eyeshadow/blush + *Opacity, glasses/crown/ catEars) and VideoEffectsProcessor setters (setEyeColor, setLipstick, setEyeshadow, setBlush, setGlasses, setCrown, setCatEars). New VideoEffectsAccessories catalog.
• Assets: bundles the Apache-2.0 MediaPipe models (assets/models/ face_landmarker.task + selfie_segmenter.tflite, ≈4 MB) and placeholder accessory art (assets/accessories/*.png — replace with production artwork).
• Android minSdk raised 21 → 24 (required by MediaPipe Tasks Vision).
• Graceful degradation: if the MediaPipe native lib can't load (e.g. a pre-API-28 32-bit armeabi-v7a device, where its .so needs aligned_alloc), the providers set a permanent failed flag — face effects / background blur become a clean no-op (no crash, no per-frame retry) while LUT / smoothing / whitening / skin-tone keep working. Use an arm64-v8a device (or API 28+) for the MediaPipe effects; for production ship arm64-only or minSdkVersion 28.
• On-device fixes (Android): overlay rotation — FaceEffects.map() is now a plain normalized→pixel scale (MediaPipe Tasks already back-projects landmarks to the original image; the earlier inverse-rotation made overlays appear rotated/ offset). Builds on both platforms (example debug APK + iOS-simulator app); remaining items to validate: framerate at 720p and seg-mask edges (see NATIVE.md).

0.1.0 (GPU pipeline + M2)

• Android: zero-copy OpenGL ES pipeline (GlEffects + EffectsVideoProcessor) replacing the CPU/I420 path — OES→FBO passes, per-frame output texture, holds 30fps. Effects: square-LUT color filter, bilateral skin smoothing, whitening lift, and skin-tone (dual-LUT with a YCbCr skin mask).
• iOS M2: CoreImage chain extended with Gaussian/dissolve smoothing + brightness whitening (skin-tone deferred — needs a CoreImage skin mask).
• Dart: EffectsState.skinColor + VideoEffectsProcessor.setSkinColor + VideoEffectsSkinTones catalog (5 presets). SkinColor textures now declared as assets (assets/skin/**).
• Android Kotlin verified to compile in the app Gradle context.

0.1.0 (native LUT)

• Native in-place frame seam: registers an ExternalVideoFrameProcessing (Android) / ExternalVideoProcessingDelegate (iOS) on the flutter_webrtc LocalVideoTrack (found by id via FlutterWebRTCPlugin.sharedSingleton), so effects appear in both the local preview and the encoded stream. processedTrack stays null by design.
• LUT color filters implemented: Android CPU/I420 LUT; iOS CoreImage CIColorCubeWithColorSpace. Loads assets/luts/<key>.png (512² square LUTs).
• 13 bundled filters (ColorfulStyle + whitening/rosy/clarity) from a third-party beauty-effects resource bundle (LUTs only — no third-party native engine).
• Android plugin verified to compile in the app Gradle context (webrtc-sdk 144.7559.01 + auto-wired :flutter_webrtc). iOS pending on-device build.
• SkinColor textures staged under assets/skin/** (blend documented in NATIVE.md; shader not yet implemented).

0.1.0 (scaffold)

• Initial scaffold.
• Public Dart API: VideoEffectsProcessor (a livekit_client TrackProcessor<VideoProcessorOptions>), EffectsState, and the utd_video_effects_kit/control method-channel contract.
• Safe-passthrough behavior: until the native GPU pipeline lands, isSupported is false and processedTrack is null, so the original camera frames flow unmodified and video always publishes.
• Android (Kotlin) + iOS (Swift) plugin scaffolds implementing the channel contract as passthrough, with documented Milestone-0/1 TODOs.
• Integrates with utd_live_room_kit via its UTDLiveRoomConfig.buildVideoProcessor factory — no dependency from the kit onto this package.