mlx_audio 0.2.2
mlx_audio: ^0.2.2 copied to clipboard
On-device speech for Apple Silicon: Parakeet-TDT and Whisper speech-to-text and PocketTTS text-to-speech, built on the mlx array framework via mlx-c.
Changelog #
0.2.2 - 2026-08-06 #
- Require
mlx ^0.2.0:SmartTurnModelcallssetCacheLimit, which first shipped in mlx 0.2.0. The published 0.2.1 declared^0.1.0and could resolve an mlx that lacks the symbol. - Changelog correction: the Smart Turn v3.2 entries below shipped in 0.2.1 (they were mis-filed under Unreleased).
0.2.1 - 2026-07-29 #
-
Added Smart Turn v3.2 turn detection:
SmartTurn.load()resolves the pinnedmlx-community/smart-turn-v3checkpoint, caps MLX's buffer cache at 64 MB, and warms the Metal kernels on 1 s of silence;SmartTurnModel.predictEndpointscores one ≤ 8 s window of 16 kHz mono audio and returns the sigmoid probability plus a strictprobability > thresholdverdict. -
Added
dart run mlx_audio:turn_replay(parity/bench/score) — the replay harness, emitting the same window/probs schema as the ectos Swift harness so the two can be diffed directly or fed to the shared fp32 ONNX reference (tools/smart-turn-parity.py). -
Parity: max
|Δp| = 0.0000vs the fp32 ONNX reference and0.0038vs the Swift oracle over the committed two-fixture corpus (gate:< 0.02). -
Added revision pinning to
resolveOrDownloadModel: the newrevisionparameter (defaultmain, unchanged behaviour) is threaded into both the tree listing and every download URL, so a repo can be resolved at an exact commit. -
Added SHA256-verified downloads through the new
pinnedparameter withPinnedFile/PinnedRepo: each pinned file is hashed as a temp before it is renamed into place, a mismatch raises the new typedHubIntegrityExceptionand leaves nothing behind, and a warm cache is recognised by exact byte count. -
Added
kSmartTurnWeights— the pinnedmlx-community/smart-turn-v3(Smart Turn v3.2) revision and per-file hash manifest.
0.2.0 - 2026-07-26 #
- Added interface-level STT generation parameters, typed inference progress, and cooperative cancellation.
- Added deterministic, idempotent model cleanup through
close(). - Added true incremental PocketTTS PCM callbacks while preserving the complete
TtsOutputreturn value.
0.1.0 - 2026-07-19 #
Initial release: on-device speech models for Apple Silicon, built on the mlx
package.
- Speech-to-text:
- Parakeet-TDT — log-mel front-end → Conformer encoder → RNN-T
predict/joint → TDT greedy decode, loading the real
mlx-community/parakeet-tdt-0.6b-v2checkpoint. - Whisper — mel front-end → encoder → greedy KV-cache decode with >30 s tiled chunking; loads HuggingFace fp32, mlx-whisper fp16, and 4-bit / 8-bit quantized safetensors, with automatic tokenizer fallback by vocab size.
- Parakeet-TDT — log-mel front-end → Conformer encoder → RNN-T
predict/joint → TDT greedy decode, loading the real
- Text-to-speech: PocketTTS — FlowLM backbone + flow-matching latent sampler → Mimi neural codec, synthesizing a 24 kHz waveform from text with eight predefined voices.
- DSP / audio: STFT + log-mel front-end, float32 WAV I/O, resampling, and mono-16 kHz loading.
- Model management: a Hugging Face hub client that snapshots repos into a
local cache (
resolveOrDownloadModel,HubProgress). - Eval utilities: transcript normalization and word-error-rate.
- CLIs:
mlx_audio:transcribe(STT) andmlx_audio:say(TTS).