trial(tts/moss-tts-nano): MOSS-TTS-Nano-100M + Audio-Tokenizer-Nano to CoreML - #92
Open
Alex-Wengg wants to merge 2 commits into
Open
trial(tts/moss-tts-nano): MOSS-TTS-Nano-100M + Audio-Tokenizer-Nano to CoreML#92Alex-Wengg wants to merge 2 commits into
Alex-Wengg wants to merge 2 commits into
Conversation
…o CoreML Six mlpackages (macOS 14 / iOS 17): Prefill T512/M1024, Step M1024, fused Frame (17 local-transformer passes + in-graph inverse-CDF sampling), streaming CodecStep (shift-window KV, constant relative-RoPE tables), full CodecDecoder (RangeDim <=125 frames) and fp32 CodecEncoder. Parity: wrapper vs upstream 1e-5 level; fp16 greedy replay 370/375 frames token-exact over 30 s; codec step 56.6 dB SNR (GPU); encoder exact only in fp32. Warm latency on M5 Pro: step 7.6 + frame 5.3 + codec step 5.2 ms per 80 ms frame. Parakeet WER on two phrases 8.3 % (CoreML) vs 10.1 % (PyTorch). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018UHoFANi4DcvU6TzyTPnHH
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018UHoFANi4DcvU6TzyTPnHH
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
CoreML conversion of OpenMOSS-Team/MOSS-TTS-Nano-100M (0.1B multilingual streaming TTS with zero-shot voice cloning, 20 languages, 48 kHz stereo) and its codec MOSS-Audio-Tokenizer-Nano (22M). Apache-2.0 upstream.
Six mlpackages, all macOS 14 / iOS 17 (no
StateType):Prefill-T512-M1024,Step-M1024— GPT-2 12L/768d global LM, pass-through KV[12,1,12,1024,64]Frame— fused 1-layer local transformer (17 unrolled passes) + 16 codebook heads, with top-k / top-p / temperature / repetition-penalty sampling in-graph from host uniforms (inverse-CDF), greedy flagCodecStep— streaming per-frame decoder, shift-window KV caches (500/800/1200/1600), constant relative-RoPE tables, no runtime trigCodecDecoder— flexible-length batch decoder (≤125 frames)CodecEncoder(fp32) — voice-prompt encoder; fp16 loses 34 % of codes through the residual LFQValidation
en_2.wav): CoreML 8.3 % WER vs upstream PyTorch 10.1 %Notes
input_lengths,apply_ropedoes2 / Don a traced size (patched to static D for export). Details inTRIALS.md.FluidInference/moss-tts-nano-coreml; Swift backend follows in FluidAudio.🤖 Generated with Claude Code
https://claude.ai/code/session_018UHoFANi4DcvU6TzyTPnHH