mlx_audio 0.2.2 copy "mlx_audio: ^0.2.2" to clipboard
mlx_audio: ^0.2.2 copied to clipboard

PlatformmacOS

On-device speech for Apple Silicon: Parakeet-TDT and Whisper speech-to-text and PocketTTS text-to-speech, built on the mlx array framework via mlx-c.

Changelog #

0.2.2 - 2026-08-06 #

  • Require mlx ^0.2.0: SmartTurnModel calls setCacheLimit, which first shipped in mlx 0.2.0. The published 0.2.1 declared ^0.1.0 and could resolve an mlx that lacks the symbol.
  • Changelog correction: the Smart Turn v3.2 entries below shipped in 0.2.1 (they were mis-filed under Unreleased).

0.2.1 - 2026-07-29 #

  • Added Smart Turn v3.2 turn detection: SmartTurn.load() resolves the pinned mlx-community/smart-turn-v3 checkpoint, caps MLX's buffer cache at 64 MB, and warms the Metal kernels on 1 s of silence; SmartTurnModel.predictEndpoint scores one ≤ 8 s window of 16 kHz mono audio and returns the sigmoid probability plus a strict probability > threshold verdict.

  • Added dart run mlx_audio:turn_replay (parity / bench / score) — the replay harness, emitting the same window/probs schema as the ectos Swift harness so the two can be diffed directly or fed to the shared fp32 ONNX reference (tools/smart-turn-parity.py).

  • Parity: max |Δp| = 0.0000 vs the fp32 ONNX reference and 0.0038 vs the Swift oracle over the committed two-fixture corpus (gate: < 0.02).

  • Added revision pinning to resolveOrDownloadModel: the new revision parameter (default main, unchanged behaviour) is threaded into both the tree listing and every download URL, so a repo can be resolved at an exact commit.

  • Added SHA256-verified downloads through the new pinned parameter with PinnedFile / PinnedRepo: each pinned file is hashed as a temp before it is renamed into place, a mismatch raises the new typed HubIntegrityException and leaves nothing behind, and a warm cache is recognised by exact byte count.

  • Added kSmartTurnWeights — the pinned mlx-community/smart-turn-v3 (Smart Turn v3.2) revision and per-file hash manifest.

0.2.0 - 2026-07-26 #

  • Added interface-level STT generation parameters, typed inference progress, and cooperative cancellation.
  • Added deterministic, idempotent model cleanup through close().
  • Added true incremental PocketTTS PCM callbacks while preserving the complete TtsOutput return value.

0.1.0 - 2026-07-19 #

Initial release: on-device speech models for Apple Silicon, built on the mlx package.

  • Speech-to-text:
    • Parakeet-TDT — log-mel front-end → Conformer encoder → RNN-T predict/joint → TDT greedy decode, loading the real mlx-community/parakeet-tdt-0.6b-v2 checkpoint.
    • Whisper — mel front-end → encoder → greedy KV-cache decode with >30 s tiled chunking; loads HuggingFace fp32, mlx-whisper fp16, and 4-bit / 8-bit quantized safetensors, with automatic tokenizer fallback by vocab size.
  • Text-to-speech: PocketTTS — FlowLM backbone + flow-matching latent sampler → Mimi neural codec, synthesizing a 24 kHz waveform from text with eight predefined voices.
  • DSP / audio: STFT + log-mel front-end, float32 WAV I/O, resampling, and mono-16 kHz loading.
  • Model management: a Hugging Face hub client that snapshots repos into a local cache (resolveOrDownloadModel, HubProgress).
  • Eval utilities: transcript normalization and word-error-rate.
  • CLIs: mlx_audio:transcribe (STT) and mlx_audio:say (TTS).
1
likes
160
points
236
downloads

Documentation

API reference

Publisher

verified publisherksh.dev

Weekly Downloads

On-device speech for Apple Silicon: Parakeet-TDT and Whisper speech-to-text and PocketTTS text-to-speech, built on the mlx array framework via mlx-c.

Repository (GitHub)
View/report issues

Topics

#speech-to-text #text-to-speech #whisper #mlx #on-device-ai

License

MIT (license)

Dependencies

args, crypto, http, mlx, path, wav

More

Packages that depend on mlx_audio