feat(tts/chatterbox-nano): extended ~30 s output capacity + honest budget errors

[Alex-Wengg]

Closes #924

Problem

As #924 documents (and this PR's investigation confirmed against the shipped safetensors), the raw caps hide the voice's own footprint:

	Raw shape	Voice footprint	Usable budget
Input text	512-token prefill	376 cond rows + BOS	≤135 BPE tokens (~500–550 chars)
Generated audio	500-token flow bucket	250 prompt + 3 silence	≤247 tokens ≈ 9.9 s

The output cap binds first in practice. The buckets are conversion-time static-shape choices, not model limits — the checkpoint's wpe is [8196, 768] and the decode model's M1536 KV cache already supports ~1020 generated tokens, so only the S3Gen pair needed a larger export.

Changes

• ChatterboxNanoOutputCapacity — .standard (N500/T1000, ≈9.9 s) or .extended (N1000/T2000, ≈29.9 s), chosen at ChatterboxNanoManager(outputCapacity:). The extended pair is a separate ~280 MB download from FluidInference/chatterbox-nano-coreml (exported in feat(tts/chatterbox-nano): --bucket arg for S3Gen export; N1000/T2000 extended pair mobius#93, fp16 parity in the N500 class) and roughly doubles flow/vocoder latency per call, so it stays opt-in. T3 prefill/decode are shared.
• Fail fast + honest errors — decode throws the moment the generation budget is exhausted instead of decoding to EOS first; textTooLong/generationTooLong now report the usable budget (text tokens vs prefill-minus-conditioning; generated tokens vs bucket-minus-prompt, with seconds) for both Nano and Multilingual.
• Docs — effective-budget table in Documentation/TTS/Chatterbox.md + doc comments on the constants; CLI --extended-output.
• Tests — budget math, per-capacity model sets, repo variant mapping.

Verification (M5 Pro, release build)

• 95-word text, --extended-output: 25.6 s WAV, verbatim Parakeet round-trip
• Same text, .standard: fails fast with generationTooLong(tokens: 248, max: 247) (previously decoded to EOS before throwing)
• Short text, .standard: unchanged (3.0 s, same seed)

[github-actions]

Parakeet EOU Benchmark Results ✅

Status: Benchmark passed
Chunk Size: 320ms
Files Tested: 100/100

Performance Metrics

Metric	Value	Description
WER (Avg)	7.03%	Average Word Error Rate
WER (Med)	4.17%	Median Word Error Rate
RTFx	9.76x	Real-time factor (higher = faster)
Total Audio	470.6s	Total audio duration processed
Total Time	50.6s	Total processing time

Streaming Metrics

Metric	Value	Description
Avg Chunk Time	0.051s	Average chunk processing time
Max Chunk Time	0.101s	Maximum chunk processing time
EOU Detections	0	Total End-of-Utterance detections

_{Test runtime: 0m58s • 09/14/2026, 04:59 PM EST}

_{RTFx = Real-Time Factor (higher is better) • Processing includes: Model inference, audio preprocessing, state management, and file I/O}

[github-actions]

Supertonic3 Smoke Test ✅

Check	Result
Build	✅
Model download (incl. VectorEstimatorVariants/ int4 buckets)	✅
Model load	✅
Synthesis pipeline (--ve-variant int4)	✅
Output WAV	✅ (364.7 KB)

_{Runtime: 0m21s}

_{Note: CI VMs lack a physical Neural Engine; the ANE-bucketed VectorEstimator falls back to CPU here. This validates download + variant resolution + synthesis, not ANE residency/perf.}

[github-actions]

Offline VBx Pipeline Results

Speaker Diarization Performance (VBx Batch Mode)

Optimal clustering with Hungarian algorithm for maximum accuracy

Metric	Value	Target	Status	Description
DER	10.4%	<20%	✅	Diarization Error Rate (lower is better)
RTFx	10.58x	>1.0x	✅	Real-Time Factor (higher is faster)

Offline VBx Pipeline Timing Breakdown

Time spent in each stage of batch diarization

Stage	Time (s)	%	Description
Model Download	20.773	20.9	Fetching diarization models
Model Compile	8.903	9.0	CoreML compilation
Audio Load	0.094	0.1	Loading audio file
Segmentation	29.909	30.2	VAD + speech detection
Embedding	98.929	99.7	Speaker embedding extraction
Clustering (VBx)	0.107	0.1	Hungarian algorithm + VBx clustering
Total	99.187	100	Full VBx pipeline

Speaker Diarization Research Comparison

Offline VBx achieves competitive accuracy with batch processing

Method	DER	Mode	Description
FluidAudio (Offline)	10.4%	VBx Batch	On-device CoreML with optimal clustering
FluidAudio (Streaming)	17.7%	Chunk-based	First-occurrence speaker mapping
Research baseline	18-30%	Various	Standard dataset performance

Pipeline Details:

• Mode: Offline VBx with Hungarian algorithm for optimal speaker-to-cluster assignment
• Segmentation: VAD-based voice activity detection
• Embeddings: WeSpeaker-compatible speaker embeddings
• Clustering: PowerSet with VBx refinement
• Accuracy: Higher than streaming due to optimal post-hoc mapping

_{🎯 Offline VBx Test • AMI Corpus ES2004a • 1049.0s meeting audio • 128.9s processing • Test runtime: 2m 13s • 09/14/2026, 05:08 PM EST}

[github-actions]

VAD Benchmark Results

Performance Comparison

Dataset	Accuracy	Precision	Recall	F1-Score	RTFx	Files
MUSAN	94.0%	89.3%	100.0%	94.3%	400.6x faster	50
VOiCES	94.0%	89.3%	100.0%	94.3%	463.9x faster	50

Dataset Details

• MUSAN: Music, Speech, and Noise dataset - standard VAD evaluation
• VOiCES: Voices Obscured in Complex Environmental Settings - tests robustness in real-world conditions

✅: Average F1-Score above 70%

[github-actions]

Sortformer High-Latency Benchmark Results

ES2004a Performance (30.4s latency config)

Metric	Value	Target	Status
DER	30.3%	<35%	✅
Miss Rate	28.2%	-	-
False Alarm	0.9%	-	-
Speaker Error	1.2%	-	-
RTFx	16.4x	>1.0x	✅
Speakers	4/4	-	-

_{Sortformer High-Latency • ES2004a • Runtime: 2m 55s • 2026-09-14T21:15:08.628Z}

[github-actions]

PocketTTS Smoke Test ✅

Check	Result
Build	✅
Model download	✅
Model load	✅
Synthesis pipeline	✅
Output WAV	✅ (153.8 KB)

_{Runtime: 0m6s}

_{Note: PocketTTS uses CoreML MLState (macOS 15) KV cache + Mimi streaming state. CI VM lacks physical GPU — audio quality and performance may differ from Apple Silicon.}

[github-actions]

ASR Benchmark Results ✅

Status: All benchmarks passed

Parakeet v3 (multilingual)

Dataset	WER Avg	WER Med	RTFx	Status
test-clean	0.57%	0.00%	5.71x	✅
test-other	1.19%	0.00%	3.52x	✅

Parakeet v2 (English-optimized)

Dataset	WER Avg	WER Med	RTFx	Status
test-clean	0.80%	0.00%	5.83x	✅
test-other	1.00%	0.00%	3.36x	✅

Streaming (v3)

Metric	Value	Description
WER	0.00%	Word Error Rate in streaming mode
RTFx	0.65x	Streaming real-time factor
Avg Chunk Time	1.408s	Average time to process each chunk
Max Chunk Time	1.758s	Maximum chunk processing time
First Token	1.777s	Latency to first transcription token
Total Chunks	31	Number of chunks processed

Streaming (v2)

Metric	Value	Description
WER	0.00%	Word Error Rate in streaming mode
RTFx	0.50x	Streaming real-time factor
Avg Chunk Time	1.827s	Average time to process each chunk
Max Chunk Time	3.049s	Maximum chunk processing time
First Token	1.864s	Latency to first transcription token
Total Chunks	31	Number of chunks processed

_{Streaming tests use 5 files with 0.5s chunks to simulate real-time audio streaming}

_{25 files per dataset • Test runtime: 7m50s • 09/14/2026, 05:06 PM EST}

_{RTFx = Real-Time Factor (higher is better) • Calculated as: Total audio duration ÷ Total processing time
Processing time includes: Model inference on Apple Neural Engine, audio preprocessing, state resets between files, token-to-text conversion, and file I/O
Example: RTFx of 2.0x means 10 seconds of audio processed in 5 seconds (2x faster than real-time)}

Expected RTFx Performance on Physical M1 Hardware:

• M1 Mac: ~28x (clean), ~25x (other)
• CI shows ~0.5-3x due to virtualization limitations

_{Testing methodology follows HuggingFace Open ASR Leaderboard}

[github-actions]

Speaker Diarization Benchmark Results

Speaker Diarization Performance

Evaluating "who spoke when" detection accuracy

Metric	Value	Target	Status	Description
DER	15.1%	<30%	✅	Diarization Error Rate (lower is better)
JER	24.9%	<25%	✅	Jaccard Error Rate
RTFx	19.50x	>1.0x	✅	Real-Time Factor (higher is faster)

Diarization Pipeline Timing Breakdown

Time spent in each stage of speaker diarization

Stage	Time (s)	%	Description
Model Download	11.076	20.6	Fetching diarization models
Model Compile	4.747	8.8	CoreML compilation
Audio Load	0.055	0.1	Loading audio file
Segmentation	16.135	30.0	Detecting speech regions
Embedding	26.891	50.0	Extracting speaker voices
Clustering	10.757	20.0	Grouping same speakers
Total	53.805	100	Full pipeline

Speaker Diarization Research Comparison

Research baselines typically achieve 18-30% DER on standard datasets

Method	DER	Notes
FluidAudio	15.1%	On-device CoreML
Research baseline	18-30%	Standard dataset performance

Note: RTFx shown above is from GitHub Actions runner. On Apple Silicon with ANE:

• M2 MacBook Air (2022): Runs at 150 RTFx real-time
• Performance scales with Apple Neural Engine capabilities

_{🎯 Speaker Diarization Test • AMI Corpus ES2004a • 1049.0s meeting audio • 53.8s diarization time • Test runtime: 3m 34s • 09/14/2026, 05:14 PM EST}