feat: add mmx speech transcribe for speech-to-text (asr-1.0) - #262
Merged
Merged
Conversation
Adds a speech-to-text command on top of POST /v1/speech_to_text: mmx speech transcribe --file <path> [--language zh] [--response-format srt --out a.srt] - uploads the audio as multipart/form-data, reusing the existing region base URL and API key resolution - --response-format json (default) / verbose_json / srt / vtt - --language is sent as a request header, matching the API contract; the same value sent as a form field is accepted but silently ignored by the API - --stream pipes incremental text (json only); --out writes the result to a file - rejects audio above the documented 50 MB limit locally instead of uploading it only to come back as HTTP 413 - registers speech recognize as an alias, and documents the command in README, README_CN, SDK.md, ERRORS.md, the agent skill, and the CLI design tree
Blind review found one hard contract bug and several consistency issues: - SDK: `response_format: 'srt' | 'vtt'` returned a subtitle document, but the SDK parsed the body as JSON and failed. It now resolves to that document as a string through a dedicated overload, matching what the CLI already did. - SDK now enforces the documented 50 MB limit for both paths and Blobs, so the limit no longer lives only in the command. - The multipart text parts are built once by `sttFormFields()` and shared by the CLI (dry-run preview and request) and the SDK, instead of assembling the same field group twice in each. - Subtitle detection moved next to the format lists in `utils/stt.ts` instead of string comparisons spread across the command. - `--response-format` now reports `Invalid response format` rather than the audio-format wording borrowed from TTS. - The streaming path reports the audio duration on stderr, which was previously discarded in text output mode. - Documented the `speech recognize` alias, and clarified in the skill that piped `--stream` output needs `--output text` (non-TTY defaults to json). - Tests reuse the shared mock-server helpers and cover the new SDK subtitle path, the SDK size guard, and the streamed duration.
Second review round, second axis (API contract fidelity): - The stream loop now stops on the API's final `finish` event, and warns on stderr when a stream ends without it, instead of silently returning a possibly truncated transcript. - `--out` write failures no longer surface as a raw filesystem error for the disk-full case; ERRORS.md documents both outcomes. - The SDK validates `response_format` too, so the CLI and SDK reject the same values instead of only the CLI doing so. - Documentation no longer implies the 500 s limit is checked locally (only the 50 MB size limit is), and says `n_speakers` / `segments` need `--output json`. - `validateSttFileSize` takes a display label rather than a path, since the SDK passes a Blob filename; `utils/stt.ts` no longer borrows `formatList` from the TTS module.
Final blind review flagged that stopping at `finish: true` leaves the response body undrained, which can hold the connection (and the process) open. The body is now cancelled in a `finally`, matching the idiom already used by `agent/verify.ts`. Also records why `--timestamp-level` and `--model` are passed through instead of validated locally, so the omission reads as a decision rather than a gap.
…utput fidelity Review follow-ups on speech transcribe: - SDK validation now raises SDKError, not CLIError: the shared stt validators take the caller's error class, so consumers narrowing on SDKError no longer miss format/size/stream-combo rejections. - The SDK stream generator ends at the API's final event (finish=true) and releases the SSE body, mirroring the CLI fix; breaking out early is safe. CLI and SDK now share one submission path (submitSttForm), so the SSE / subtitle / json dispatch cannot drift between layers. - Stream deltas are assembled by index per the API contract: the CLI warns on out-of-order events and the json result is index-sorted. - srt/vtt saved with --out are byte-exact; only stdout keeps the trailing-newline convention. - mapApiError keys speech-to-text 422 (sensitive audio) off the endpoint + status instead of message text, and gives 413 a dedicated size-limit message with exit code 2. - withStubbedFetch extracted to test/helpers; help table alignment; SDK.md and ERRORS.md document the new behaviour.
Wzdhehe
added a commit
to Wzdhehe/mmx-asr-cli
that referenced
this pull request
Sep 19, 2026
…nt plus review fixes (commit 82f8d96)
Contributor
Author
|
Pushed
|
9 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
mmxcan synthesize speech but cannot transcribe it. The platform exposesPOST /v1/speech_to_text(ASR,asr-1.0), so Token Plan users have no way to reach it from the CLI — you have to drop tocurlwith a raw HTTP call. This adds the missing command:What it does
multipart/form-datato/v1/speech_to_text, reusing the existing region base URL and API key resolution.--model,--response-format,--language, and--timestamp-levelmap 1:1 to the API fields.--response-format json(default) prints the transcript,verbose_jsonreturns speakers and per-segment timestamps, andsrt/vttreturn subtitle documents (printed or saved verbatim).--streamprints incremental text;--out <path>writes the result to a file.sdk.speech.transcribe({ file, language }), with overloads for streaming (AsyncGenerator<SpeechToTextStreamEvent>) and subtitle formats (string).speech recognizeas an alias.Notes for reviewers
languageis sent as a request header. The API documents it as anin: headerparameter. Verified against the live API: a form field is silently ignored (an invalid tagzzstill returns HTTP 200 plus a transcript), while the header is honoured (zz→ HTTP 400invalid language "zz": must be a valid BCP-47 language tag). A test pins the header placement so a future refactor cannot regress it.50 MB is checked locally.
src/utils/stt.tsrejects oversized audio before the upload, because a 500 s 48 kHz stereo WAV (~92 MB) would otherwise be uploaded in full only to come back as HTTP 413. The server stays the authority for the 500 s duration limit and codec support — duration is not knowable without decoding the file, so that error is surfaced verbatim from the API.--streamis opt-in.mmx text chatauto-streams in a TTY; for ASR the whole transcript arrives in one response, so streaming is left to an explicit flag and the default path stays single and predictable. Behaviour while streaming followstext chat: in text mode the deltas go to stdout; when stdout is not a terminal (i.e.--output json) they accumulate into one JSON result instead.--streamwith--outerrors rather than silently ignoring--out, so nobody assumes a file was written. The hint points at--output textfor piping, since non-TTY stdout defaults to json.--modeland--timestamp-levelare passed through unvalidated while--response-formatis validated locally. That is deliberate: the CLI interprets the response format (it decides whether the body is a subtitle document), whereas the other two are opaque to it, and the API validates them with a clear message. It also means a new timestamp level does not need a CLI release.Verification
bun run typecheckbun run lintbun testtest/agent/*andtest/files/download.test.tsare pre-existing Windows-only failures that reproduce unmodified onbfbb4cbbun run buildbun build src/main.ts --compilesmoke binary runs--versionThe command was also exercised end-to-end against the live API on the
cnregion with real audio: default json,verbose_json+ word timestamps,srtto file (valid UTF-8, no BOM),--stream, therecognizealias, a positional audio path, and the error paths (missing--file, nonexistent path, unsupported format,--stream+srt,--stream+--out, non-audio input, invalid language tag) — all exiting with the documented codes (2 for usage, 1 for API errors).The change went through two rounds of independent blind review (repo standards/smells, and API-contract fidelity), plus a final single blind pass. Findings that were fixed:
srt/vttbodies as JSON and failed — those formats now resolve to the subtitle document;response_format, so the CLI and SDK disagreed;sttFormFields();finishand warns when a stream ends without it, rather than silently returning a possibly truncated transcript;--outdisk-full failures and a missingERRORS.mdrow;n_speakers/segmentsrequire--output json.Deliberately kept (raised in review, decided against) — all documented above: no local duration check, arrival-order delta assembly (
text chatbehaves the same; SSE over one connection is ordered), pass-through--model/--timestamp-level,--stream+--outmutually exclusive, and explicit--streamfollowing the CLI-wide non-TTY → json output rule.One review concern turned out to be a non-issue when checked against the live API: the SDK names Blob inputs
audio(no extension), and the API does not depend on the multipart filename —audio,audio.bin, andprobe.mp3all return the same transcript.Need help on this PR? Tag
@codesmith-botwith what you need. Autofix is disabled.