From f2e1f55fd46ace3565f02e841c21d3226627608f Mon Sep 17 00:00:00 2001 From: Wzdhehe Date: Sat, 19 Sep 2026 17:51:20 +0800 Subject: [PATCH 1/6] feat: add mmx speech transcribe for speech-to-text (asr-1.0) Adds a speech-to-text command on top of POST /v1/speech_to_text: mmx speech transcribe --file [--language zh] [--response-format srt --out a.srt] - uploads the audio as multipart/form-data, reusing the existing region base URL and API key resolution - --response-format json (default) / verbose_json / srt / vtt - --language is sent as a request header, matching the API contract; the same value sent as a form field is accepted but silently ignored by the API - --stream pipes incremental text (json only); --out writes the result to a file - rejects audio above the documented 50 MB limit locally instead of uploading it only to come back as HTTP 413 - registers speech recognize as an alias, and documents the command in README, README_CN, SDK.md, ERRORS.md, the agent skill, and the CLI design tree --- ERRORS.md | 14 + README.md | 18 +- README_CN.md | 17 +- SDK.md | 13 + docs/cli-design.md | 3 +- skill/SKILL.md | 47 ++- src/client/endpoints.ts | 4 + src/commands/help.ts | 1 + src/commands/speech/transcribe.ts | 183 ++++++++++ src/registry.ts | 5 +- src/sdk/speech/index.ts | 90 ++++- src/types/api.ts | 50 +++ src/utils/stt.ts | 41 +++ test/commands/aliases.test.ts | 7 + test/commands/speech/transcribe.test.ts | 436 ++++++++++++++++++++++++ test/sdk/speech.test.ts | 135 +++++++- test/utils/stt.test.ts | 39 +++ 17 files changed, 1091 insertions(+), 12 deletions(-) create mode 100644 src/commands/speech/transcribe.ts create mode 100644 src/utils/stt.ts create mode 100644 test/commands/speech/transcribe.test.ts create mode 100644 test/utils/stt.test.ts diff --git a/ERRORS.md b/ERRORS.md index 9ea439a3..bc40ca33 100644 --- a/ERRORS.md +++ b/ERRORS.md @@ -123,6 +123,20 @@ This document lists all error scenarios and the messages users will see. All errors fall under [Network Errors](#networkerrors). +### `mmx speech transcribe` + +| Scenario | Error Message | +|---|---| +| No `--file` (and no positional path) in non-interactive mode | `Missing required argument: --file` | +| Audio file not found | `File not found: ${fullPath}` | +| Invalid `--response-format` | `Invalid audio format "${fmt}". Supported: json, verbose_json, srt, vtt` | +| Audio file above 50 MB | `Audio file is ${size} MB; speech-to-text allows at most 50 MB: ${fullPath}` | +| `--stream` with a response format other than `json` | `response_format "${fmt}" cannot be combined with stream=true; streaming returns incremental json only.` | +| `--stream` together with `--out` | `--stream and --out cannot be combined.` | + +Audio longer than 500 seconds and unsupported/corrupt audio are rejected by the API; the +server message is surfaced verbatim (e.g. `API error: invalid params, ... (HTTP 400)`). + --- ## Vision Commands diff --git a/README.md b/README.md index fd88f9ee..bf2c0bfd 100644 --- a/README.md +++ b/README.md @@ -20,7 +20,7 @@ - **Text** — Multi-turn chat, streaming, system prompts, JSON output - **Image** — Text-to-image with aspect ratio and batch controls - **Video** — Async video generation with progress tracking -- **Speech** — TTS with 30+ voices, speed control, streaming playback +- **Speech** — TTS with 30+ voices, speed control, streaming playback; speech-to-text transcription (json, verbose_json, srt, vtt) - **Vision** — Image understanding and description - **Search** — Web search powered by MiniMax - **Dual Region** — Seamless Global (`api.minimax.io`) and CN (`api.minimaxi.com`) support @@ -54,6 +54,7 @@ mmx auth login --api-key sk-xxxxx mmx text chat --message "What is MiniMax?" mmx image "A cat in a spacesuit" mmx speech synthesize --text "Hello!" --out hello.mp3 +mmx speech transcribe --file meeting.mp3 mmx video generate --prompt "Ocean waves at sunset" mmx search "MiniMax AI latest news" mmx vision photo.jpg @@ -116,6 +117,21 @@ echo "Breaking news" | mmx speech synthesize --text-file - --out news.mp3 mmx speech voices ``` +```bash +# Speech-to-text (asr-1.0) +mmx speech transcribe --file meeting.mp3 +mmx speech transcribe --file call.mp3 --language zh +mmx speech transcribe --file talk.mp3 --response-format verbose_json --output json +mmx speech transcribe --file talk.mp3 --response-format srt --out talk.srt +mmx speech transcribe --file long.mp3 --stream +``` + +`mmx speech transcribe` accepts wav, aiff, flac, m4a, mp3, aac, opus, and ogg files up to +50 MB and 500 seconds; larger files are rejected locally before upload. Omitting +`--language` enables mixed-language recognition. `--response-format json` (default) prints the +transcript, `verbose_json` adds speakers and per-segment timestamps, and `srt` / `vtt` return +subtitle documents. `--stream` prints incremental text and requires `json`. + ### `mmx vision` ```bash diff --git a/README_CN.md b/README_CN.md index 5b7aa521..63fd452c 100644 --- a/README_CN.md +++ b/README_CN.md @@ -20,7 +20,7 @@ - **文本对话** — 多轮对话、流式输出、系统提示词、JSON 格式输出 - **图像生成** — 文生图，支持比例和批量控制 - **视频生成** — 异步生成，进度追踪 -- **语音合成** — 30+ 音色、语速调节、流式播放 +- **语音** — 语音合成（30+ 音色、语速调节、流式播放）与语音识别（音频转文字，支持 json、verbose_json、srt、vtt） - **图像理解** — 图片描述与识别 - **网络搜索** — MiniMax 搜索引擎 - **双区域** — 国际版（`api.minimax.io`）和国内版（`api.minimaxi.com`）自动切换 @@ -54,6 +54,7 @@ mmx auth login --api-key sk-xxxxx mmx text chat --message "你好，MiniMax！" mmx image "一只穿宇航服的猫" mmx speech synthesize --text "你好！" --out hello.mp3 +mmx speech transcribe --file meeting.mp3 mmx video generate --prompt "海浪拍打礁石" mmx search "MiniMax AI 最新动态" mmx vision photo.jpg @@ -111,6 +112,20 @@ echo "头条新闻" | mmx speech synthesize --text-file - --out news.mp3 mmx speech voices ``` +```bash +# 语音识别（asr-1.0） +mmx speech transcribe --file meeting.mp3 +mmx speech transcribe --file call.mp3 --language zh +mmx speech transcribe --file talk.mp3 --response-format verbose_json --output json +mmx speech transcribe --file talk.mp3 --response-format srt --out talk.srt +mmx speech transcribe --file long.mp3 --stream +``` + +`mmx speech transcribe` 支持 wav、aiff、flac、m4a、mp3、aac、opus、ogg 格式，音频不超过 +50 MB、时长不超过 500 秒；超出时会在上传前直接报错。不传 `--language` 时启用混合语言识别。 +`--response-format json`（默认）输出转写文本，`verbose_json` 附带说话人标识与分段 +时间戳，`srt` / `vtt` 直接返回字幕文档。`--stream` 逐段输出文本，仅支持 `json`。 + ### `mmx vision` ```bash diff --git a/SDK.md b/SDK.md index a42f3b9a..6782b2b8 100644 --- a/SDK.md +++ b/SDK.md @@ -120,6 +120,19 @@ for await (const chunk of stream) { // List voices const voices = await sdk.speech.voices(); const englishVoices = await sdk.speech.voices('en'); + +// Speech-to-text +const transcript = await sdk.speech.transcribe({ + file: './meeting.mp3', + language: 'zh', +}); +console.log(transcript.text, transcript.duration); + +// Speech-to-text, streamed +const deltas = await sdk.speech.transcribe({ file: './meeting.mp3', stream: true }); +for await (const event of deltas) { + process.stdout.write(event.delta); +} ``` ### Vision diff --git a/docs/cli-design.md b/docs/cli-design.md index cc883ea8..fe539c71 100644 --- a/docs/cli-design.md +++ b/docs/cli-design.md @@ -20,7 +20,8 @@ mmx ├── text │ └── chat Send a chat completion (M3) ├── speech -│ └── synthesize Synchronous TTS, ≤10k chars +│ ├── synthesize Synchronous TTS, ≤10k chars +│ └── transcribe Speech-to-text, ≤50 MB / ≤500 s (asr-1.0) ├── image │ └── generate Generate images (image-01) ├── video diff --git a/skill/SKILL.md b/skill/SKILL.md index 2cb831ee..690024d3 100644 --- a/skill/SKILL.md +++ b/skill/SKILL.md @@ -1,11 +1,11 @@ --- name: mmx-cli -description: Use mmx to generate text, images, video, and speech via the MiniMax AI platform. Use when the user wants to create media content, chat with MiniMax models, perform web search, or manage MiniMax API resources from the terminal. +description: Use mmx to generate text, images, video, and speech, and to transcribe audio, via the MiniMax AI platform. Use when the user wants to create media content, chat with MiniMax models, transcribe audio to text, perform web search, or manage MiniMax API resources from the terminal. --- # MiniMax CLI — Agent Skill Guide -Use `mmx` to generate text, images, video, speech, and perform web search via the MiniMax AI platform. +Use `mmx` to generate text, images, video, speech, transcribe audio, and perform web search via the MiniMax AI platform. ## Prerequisites @@ -205,6 +205,49 @@ echo "Breaking news." | mmx speech synthesize --text-file - --out news.mp3 --- +### speech transcribe + +Speech-to-text. Default model: `asr-1.0`. Accepts wav, aiff, flac, m4a, mp3, aac, opus, and +ogg files up to 50 MB and 500 seconds. + +```bash +mmx speech transcribe --file [flags] +``` + +| Flag | Type | Description | +|---|---|---| +| `--file ` | string | Audio file to transcribe (required; also accepted as a positional argument) | +| `--model ` | string | `asr-1.0` (default) | +| `--response-format ` | string | `json` (default), `verbose_json`, `srt`, `vtt` | +| `--language ` | string | BCP-47 language hint (`zh`, `en`, `ja`, ...). Omit for automatic/mixed-language detection | +| `--timestamp-level ` | string | `sentence` (default) or `word`; applies to `verbose_json` / `srt` / `vtt` | +| `--stream` | boolean | Stream incremental text to stdout. Requires `--response-format json` | +| `--out ` | string | Write the result to a file instead of stdout | + +```bash +mmx speech transcribe --file meeting.mp3 +# stdout: transcript text + +mmx speech transcribe --file call.mp3 --language zh --output json +# stdout: {"text":"...","duration":12.3,"trace_id":"..."} + +mmx speech transcribe --file talk.mp3 --response-format verbose_json --timestamp-level word --output json +# stdout: adds n_speakers and word-level segments + +mmx speech transcribe --file talk.mp3 --response-format srt --out talk.srt +# saves talk.srt; stdout: {"saved":".../talk.srt"} + +mmx speech transcribe --file long.mp3 --stream +# stdout: text as it is recognized, no trailing metadata +``` + +Notes: +- `--stream` cannot be combined with `--out`; redirect stdout instead. +- `srt` / `vtt` results are subtitle documents and are printed or saved verbatim. +- Without `--out`, `--output json` prints the full API response for `json` / `verbose_json`. + +--- + ### vision describe Image understanding via VLM. Provide either `--image` or `--file-id`, not both. diff --git a/src/client/endpoints.ts b/src/client/endpoints.ts index b91382a1..9b60f833 100644 --- a/src/client/endpoints.ts +++ b/src/client/endpoints.ts @@ -6,6 +6,10 @@ export function speechEndpoint(baseUrl: string): string { return `${baseUrl}/v1/t2a_v2`; } +export function speechToTextEndpoint(baseUrl: string): string { + return `${baseUrl}/v1/speech_to_text`; +} + export function voicesEndpoint(baseUrl: string): string { return `${baseUrl}/v1/get_voice`; } diff --git a/src/commands/help.ts b/src/commands/help.ts index df58640b..ae6a1e8a 100644 --- a/src/commands/help.ts +++ b/src/commands/help.ts @@ -12,6 +12,7 @@ interface ApiRef { const API_REFS: ApiRef[] = [ { command: 'mmx text chat', title: 'Text Generation (Chat Completion)', path: '/docs/api-reference/text-post' }, { command: 'mmx speech synthesize', title: 'Speech T2A (Text-to-Audio)', path: '/docs/api-reference/speech-t2a-http' }, + { command: 'mmx speech transcribe', title: 'Speech STT (Speech-to-Text)', path: '/docs/api-reference/speech-to-text' }, { command: 'mmx image generate', title: 'Image Generation (T2I / I2I)', path: '/docs/api-reference/image-generation-t2i' }, { command: 'mmx video generate', title: 'Video Generation (T2V / I2V / S2V)', path: '/docs/api-reference/video-generation' }, { command: 'mmx search query', title: 'Web Search', path: '/docs/api-reference/web-search' }, diff --git a/src/commands/speech/transcribe.ts b/src/commands/speech/transcribe.ts new file mode 100644 index 00000000..e3ba0a9c --- /dev/null +++ b/src/commands/speech/transcribe.ts @@ -0,0 +1,183 @@ +import { readFileSync, statSync, writeFileSync } from 'node:fs'; +import { basename, resolve } from 'node:path'; +import { defineCommand } from '../../command'; +import { CLIError } from '../../errors/base'; +import { ExitCode } from '../../errors/codes'; +import { request, requestJson } from '../../client/http'; +import { parseSSE } from '../../client/stream'; +import { speechToTextEndpoint } from '../../client/endpoints'; +import { resolveFileUploadPath } from '../../files/upload'; +import { detectOutputFormat, dryRun, formatOutput } from '../../output/formatter'; +import { formatList, validateAudioFormat } from '../../utils/audio-formats'; +import { + STT_DEFAULT_MODEL, + STT_RESPONSE_FORMATS, + sttStreamFormatConflict, + validateSttFileSize, +} from '../../utils/stt'; +import { promptOrFail } from '../../utils/prompt'; +import type { Config } from '../../config/schema'; +import type { GlobalFlags } from '../../types/flags'; +import type { SpeechToTextResponse, SpeechToTextStreamEvent } from '../../types/api'; + +/** srt/vtt come back as subtitle documents, not as JSON. */ +function isSubtitleFormat(responseFormat: string): boolean { + return responseFormat === 'srt' || responseFormat === 'vtt'; +} + +/** Results are emitted verbatim, only guaranteeing a final newline. */ +function withTrailingNewline(value: string): string { + return value.endsWith('\n') ? value : `${value}\n`; +} + +export default defineCommand({ + name: 'speech transcribe', + description: 'Transcribe an audio file to text (asr-1.0)', + apiDocs: '/docs/api-reference/speech-to-text', + usage: 'mmx speech transcribe --file [flags]', + options: [ + { flag: '--file ', description: 'Audio file to transcribe (mp3, wav, m4a, flac, aac, opus, ogg, aiff)', required: true }, + { flag: '--model ', description: `Model ID (default: ${STT_DEFAULT_MODEL})` }, + { flag: '--response-format ', description: `Transcription format: ${formatList(STT_RESPONSE_FORMATS)} (default: json)` }, + { flag: '--language ', description: 'BCP-47 language hint (zh, en, ja, ...); omit for automatic detection' }, + { flag: '--timestamp-level ', description: 'Timestamp granularity: sentence, word (verbose_json / srt / vtt only)' }, + { flag: '--stream', description: 'Stream incremental text to stdout (json only)' }, + { flag: '--out ', description: 'Write the result to a file instead of stdout' }, + ], + examples: [ + 'mmx speech transcribe --file meeting.mp3', + 'mmx speech transcribe --file call.mp3 --language zh', + 'mmx speech transcribe --file talk.mp3 --response-format verbose_json --output json', + 'mmx speech transcribe --file talk.mp3 --response-format srt --out talk.srt', + 'mmx speech transcribe --file long.mp3 --stream', + ], + async run(config: Config, flags: GlobalFlags) { + const fileInput = (flags.file ?? (flags._positional as string[] | undefined)?.[0]) as string | undefined; + const filePath = await promptOrFail({ + value: fileInput, + message: 'Enter audio file path:', + cancelMessage: 'Transcription cancelled.', + flagName: 'file', + usageHint: 'mmx speech transcribe --file ', + nonInteractive: config.nonInteractive, + }); + + const fullPath = resolveFileUploadPath( + filePath, + (path) => new CLIError(`File not found: ${path}`, ExitCode.USAGE), + ); + + const model = (flags.model as string) || STT_DEFAULT_MODEL; + const responseFormat = (flags.responseFormat as string) || 'json'; + const language = (flags.language as string) || undefined; + const timestampLevel = (flags.timestampLevel as string) || undefined; + const stream = flags.stream === true; + const outPath = flags.out ? resolve(flags.out as string) : undefined; + + validateAudioFormat(responseFormat, STT_RESPONSE_FORMATS); + validateSttFileSize(fullPath, statSync(fullPath).size); + + const streamConflict = sttStreamFormatConflict(responseFormat, stream); + if (streamConflict) throw new CLIError(streamConflict, ExitCode.USAGE); + if (stream && outPath) { + throw new CLIError( + '--stream and --out cannot be combined.', + ExitCode.USAGE, + 'Redirect stdout instead: mmx speech transcribe --file --stream > transcript.txt', + ); + } + + const preview: Record = { model, response_format: responseFormat, file: fullPath }; + if (language) preview.language = language; + if (timestampLevel) preview.timestamp_level = timestampLevel; + if (stream) preview.stream = true; + if (dryRun(config, preview)) return; + + const form = new FormData(); + form.append('model', model); + form.append('file', new Blob([readFileSync(fullPath)]), basename(fullPath)); + form.append('response_format', responseFormat); + if (timestampLevel) form.append('timestamp_level', timestampLevel); + if (stream) form.append('stream', 'true'); + + // `language` only takes effect as a request header: the API accepts (and + // ignores) the same value as a form field. + const headers: Record = {}; + if (language) headers.language = language; + + const url = speechToTextEndpoint(config.baseUrl); + const format = detectOutputFormat(config.output); + + if (stream) { + const res = await request(config, { url, method: 'POST', body: form, headers, stream: true }); + + const contentType = res.headers.get('content-type') || ''; + if (!contentType.includes('text/event-stream')) { + throw new CLIError( + `Expected SSE stream but got content-type "${contentType}". Server may be experiencing issues.`, + ExitCode.GENERAL, + ); + } + + let text = ''; + let duration: number | undefined; + const toStdout = format !== 'json'; + for await (const event of parseSSE(res)) { + if (event.data === '[DONE]') break; + let chunk: SpeechToTextStreamEvent; + try { + chunk = JSON.parse(event.data) as SpeechToTextStreamEvent; + } catch (err) { + // Warn but keep going — partial text beats failing the whole run. + process.stderr.write(`[warning] Failed to parse stream chunk: ${err instanceof Error ? err.message : String(err)}\n`); + continue; + } + if (chunk.delta) { + text += chunk.delta; + if (toStdout) process.stdout.write(chunk.delta); + } + if (chunk.finish) duration = chunk.duration; + } + + if (toStdout) { + process.stdout.write('\n'); + } else { + process.stdout.write(withTrailingNewline(formatOutput({ text, duration }, format))); + } + return; + } + + if (!config.quiet) process.stderr.write(`[Model: ${model}]\n`); + + let payload: string; + let duration: number | undefined; + + if (isSubtitleFormat(responseFormat)) { + const res = await request(config, { url, method: 'POST', body: form, headers }); + payload = await res.text(); + } else { + const response = await requestJson(config, { + url, + method: 'POST', + body: form, + headers, + }); + duration = response.duration; + payload = format === 'json' ? formatOutput(response, format) : response.text; + } + + if (outPath) { + writeFileSync(outPath, withTrailingNewline(payload), 'utf-8'); + if (config.quiet) { + console.log(outPath); + } else { + const saved: Record = { saved: outPath }; + if (duration !== undefined) saved.duration = duration; + console.log(formatOutput(saved, format)); + } + return; + } + + process.stdout.write(withTrailingNewline(payload)); + }, +}); diff --git a/src/registry.ts b/src/registry.ts index 25ae0012..3e3eac0c 100644 --- a/src/registry.ts +++ b/src/registry.ts @@ -10,6 +10,7 @@ import authLogout from './commands/auth/logout'; import textChat from './commands/text/chat'; import textRepl from './commands/text/repl'; import speechSynthesize from './commands/speech/synthesize'; +import speechTranscribe from './commands/speech/transcribe'; import speechVoices from './commands/speech/voices'; import imageGenerate from './commands/image/generate'; import videoGenerate from './commands/video/generate'; @@ -212,7 +213,7 @@ ${b('Usage:')} mmx [flags] ${b('Resources:')} ${a('auth')} ${d('Authentication (login, status, refresh, logout)')} ${a('text')} ${d('Text generation (chat)')} - ${a('speech')} ${d('Speech synthesis (synthesize, voices)')} + ${a('speech')} ${d('Speech synthesis and transcription (synthesize, voices, transcribe)')} ${a('image')} ${d('Image generation (generate)')} ${a('video')} ${d('Video generation (generate, task get, download)')} ${a('search')} ${d('Web search (query)')} @@ -296,6 +297,8 @@ export const registry = new CommandRegistry({ 'text repl': textRepl, 'speech synthesize': speechSynthesize, 'speech generate': speechSynthesize, + 'speech transcribe': speechTranscribe, + 'speech recognize': speechTranscribe, 'speech voices': speechVoices, 'image generate': imageGenerate, 'video generate': videoGenerate, diff --git a/src/sdk/speech/index.ts b/src/sdk/speech/index.ts index 5c4844cf..f898507e 100644 --- a/src/sdk/speech/index.ts +++ b/src/sdk/speech/index.ts @@ -1,13 +1,34 @@ -import { existsSync, mkdirSync, writeFileSync } from 'node:fs'; -import { resolve, dirname } from 'node:path'; +import { existsSync, mkdirSync, readFileSync, writeFileSync } from 'node:fs'; +import { basename, resolve, dirname } from 'node:path'; import { Client } from "../client"; -import { speechEndpoint, voicesEndpoint } from "../../client/endpoints"; -import { SpeechRequest, SpeechResponse, VoiceListResponse } from "../../types/api"; +import { speechEndpoint, speechToTextEndpoint, voicesEndpoint } from "../../client/endpoints"; +import { + SpeechRequest, + SpeechResponse, + SpeechToTextRequest, + SpeechToTextResponse, + SpeechToTextStreamEvent, + VoiceListResponse, +} from "../../types/api"; import { filterByLanguage } from "../../commands/speech/voices"; import { SDKError } from "../../errors/base"; import { ExitCode } from "../../errors/codes"; import { toMerged } from "es-toolkit/object"; import { ModelPartial } from "../types"; +import { STT_DEFAULT_MODEL, sttStreamFormatConflict } from "../../utils/stt"; + +export type TranscribeParams = ModelPartial & { + /** Local audio path, or a Blob/File to send as-is. */ + file: string | Blob; + /** + * BCP-47 language hint (e.g. `zh`, `en`). Sent as the `language` request + * header: the API ignores it when passed as a form field, even though its + * curl example uses `-F "language=zh"`. + */ + language?: string; +}; + +export type TranscribeStreamParams = TranscribeParams & { stream: true }; function hexToBuffer(hex: string): Buffer { if (!/^[0-9a-fA-F]*$/.test(hex)) { @@ -24,6 +45,20 @@ function defaultFilename(prefix: string, ext: string): string { return `${prefix}_${ts}.${ext}`; } +/** Resolve a transcription input into the multipart `file` part. */ +function prepareAudioUpload(file: string | Blob): { blob: Blob; filename: string } { + if (typeof file === 'string') { + const fullPath = resolve(file); + if (!existsSync(fullPath)) { + throw new SDKError(`File not found: ${fullPath}`, ExitCode.USAGE); + } + return { blob: new Blob([readFileSync(fullPath)]), filename: basename(fullPath) }; + } + + const name = (file as File).name; + return { blob: file, filename: typeof name === 'string' && name ? name : 'audio' }; +} + export class SpeechSDK extends Client { async synthesize(request: ModelPartial & { stream: true }): Promise>; async synthesize(request: ModelPartial): Promise; @@ -73,6 +108,53 @@ export class SpeechSDK extends Client { return voices; } + /** + * Transcribe an audio file (speech-to-text). + * + * `json` and `verbose_json` resolve to a structured response; SRT and VTT are + * returned by the API as subtitle documents, so `response_format` is applied + * as-is and the raw body is surfaced by the caller. The audio is uploaded as + * `multipart/form-data`, and `language` travels as a request header — the API + * ignores it when sent as a form field. + */ + async transcribe(params: TranscribeStreamParams): Promise>; + async transcribe(params: TranscribeParams): Promise; + async transcribe( + params: TranscribeParams, + ): Promise> { + const { file, language, model, response_format, timestamp_level, stream } = params; + + if (!file) { + throw new SDKError('file is required', ExitCode.USAGE); + } + + const responseFormat = response_format ?? 'json'; + const streamConflict = sttStreamFormatConflict(responseFormat, stream === true); + if (streamConflict) { + throw new SDKError(streamConflict, ExitCode.USAGE); + } + + const { blob, filename } = prepareAudioUpload(file); + + const form = new FormData(); + form.append('model', model ?? STT_DEFAULT_MODEL); + form.append('file', blob, filename); + form.append('response_format', responseFormat); + if (timestamp_level) form.append('timestamp_level', timestamp_level); + if (stream) form.append('stream', 'true'); + + const url = speechToTextEndpoint(this.config.baseUrl); + const headers: Record = {}; + if (language) headers.language = language; + + if (stream) { + const res = await this.request({ url, method: 'POST', body: form, headers, stream: true }); + return this.streamSSE(res); + } + + return this.requestJson({ url, method: 'POST', body: form, headers }); + } + /** * Save synthesized speech audio to a file. Decodes the hex-encoded audio * from the API response and writes it to disk. Creates intermediate diff --git a/src/types/api.ts b/src/types/api.ts index 67c26361..e1517c01 100644 --- a/src/types/api.ts +++ b/src/types/api.ts @@ -134,6 +134,56 @@ export interface SpeechResponse { }; } +// ---- Speech / STT (speech-to-text) ---- + +/** + * Result shape for `POST /v1/speech_to_text`. + * `json` → text + duration; `verbose_json` → adds `n_speakers` and `segments` + * (both also returned as SRT / VTT documents by the same endpoint). + */ +export type SpeechToTextFormat = 'json' | 'verbose_json' | 'srt' | 'vtt'; + +/** Timestamp granularity for `verbose_json` / `srt` / `vtt` results. */ +export type SpeechToTextTimestampLevel = 'sentence' | 'word'; + +/** + * Multipart body for `POST /v1/speech_to_text`. `file` is sent as a form part; + * unlike the other fields here, `language` travels as a request header — see + * `transcribe()` in `src/sdk/speech`. + */ +export interface SpeechToTextRequest { + model: string; + response_format?: SpeechToTextFormat; + timestamp_level?: SpeechToTextTimestampLevel; + stream?: boolean; +} + +/** One timestamped unit (sentence, or word when `timestamp_level=word`). */ +export interface SpeechToTextSegment { + id: number; + start: number; + end: number; + speaker: string; + text: string; +} + +export interface SpeechToTextResponse { + text: string; + duration: number; + n_speakers?: number; + segments?: SpeechToTextSegment[]; + trace_id?: string; +} + +/** One `data:` event of the `stream=true` SSE response. */ +export interface SpeechToTextStreamEvent { + index: number; + delta: string; + finish: boolean; + /** Total audio duration in seconds; only present on the final event. */ + duration?: number; +} + // ---- Voice List ---- export interface SystemVoiceInfo { diff --git a/src/utils/stt.ts b/src/utils/stt.ts new file mode 100644 index 00000000..2d955d27 --- /dev/null +++ b/src/utils/stt.ts @@ -0,0 +1,41 @@ +import { CLIError } from '../errors/base'; +import { ExitCode } from '../errors/codes'; +import type { SpeechToTextFormat } from '../types/api'; + +/** The only model `POST /v1/speech_to_text` exposes today. */ +export const STT_DEFAULT_MODEL = 'asr-1.0'; + +/** `response_format` values accepted by the API. */ +export const STT_RESPONSE_FORMATS: readonly SpeechToTextFormat[] = [ + 'json', + 'verbose_json', + 'srt', + 'vtt', +]; + +/** + * Documented upload limit for speech-to-text. Checked before the request so an + * oversized file fails locally instead of being uploaded only to come back as + * HTTP 413 — an uncompressed 500 s 48 kHz stereo WAV is ~92 MB. + */ +export const STT_MAX_FILE_BYTES = 50 * 1024 * 1024; + +export function validateSttFileSize(filePath: string, sizeBytes: number): void { + if (sizeBytes > STT_MAX_FILE_BYTES) { + throw new CLIError( + `Audio file is ${(sizeBytes / 1024 / 1024).toFixed(1)} MB; speech-to-text allows at most ${STT_MAX_FILE_BYTES / 1024 / 1024} MB: ${filePath}`, + ExitCode.USAGE, + 'Re-encode to compressed mono audio (e.g. mp3 / aac) or split it into smaller files.', + ); + } +} + +/** + * `stream=true` is only accepted together with `response_format=json`, so the + * combination is rejected before the audio is uploaded. Returns the message + * instead of throwing so the CLI and SDK can each raise their own error type. + */ +export function sttStreamFormatConflict(responseFormat: string, stream: boolean): string | undefined { + if (!stream || responseFormat === 'json') return undefined; + return `response_format "${responseFormat}" cannot be combined with stream=true; streaming returns incremental json only.`; +} diff --git a/test/commands/aliases.test.ts b/test/commands/aliases.test.ts index 93416a2b..61cd0d5e 100644 --- a/test/commands/aliases.test.ts +++ b/test/commands/aliases.test.ts @@ -21,6 +21,13 @@ describe('command aliases', () => { expect(generate.command).toBe(synthesize.command); }); + it('resolves "speech recognize" same as "speech transcribe"', () => { + const recognize = registry.resolve(['speech', 'recognize']); + const transcribe = registry.resolve(['speech', 'transcribe']); + expect(recognize.command).toBe(transcribe.command); + expect(transcribe.command.name).toBe('speech transcribe'); + }); + it('resolves file storage commands', () => { expect(registry.resolve(['file', 'upload']).command.name).toBe('file upload'); expect(registry.resolve(['file', 'list']).command.name).toBe('file list'); diff --git a/test/commands/speech/transcribe.test.ts b/test/commands/speech/transcribe.test.ts new file mode 100644 index 00000000..1d3cec9d --- /dev/null +++ b/test/commands/speech/transcribe.test.ts @@ -0,0 +1,436 @@ +import { describe, it, expect } from 'bun:test'; +import { mkdtempSync, readFileSync, rmSync, truncateSync, writeFileSync } from 'fs'; +import { tmpdir } from 'os'; +import { join } from 'path'; +import { default as transcribeCommand } from '../../../src/commands/speech/transcribe'; +import { STT_MAX_FILE_BYTES } from '../../../src/utils/stt'; + +const baseConfig = { + apiKey: 'test-key', + region: 'global' as const, + baseUrl: 'https://api.mmx.io', + output: 'text' as const, + timeout: 10, + verbose: false, + quiet: false, + noColor: true, + yes: false, + dryRun: false, + nonInteractive: true, + async: false, +}; + +const baseFlags = { + quiet: false, + verbose: false, + noColor: true, + yes: false, + dryRun: false, + help: false, + nonInteractive: true, + async: false, +}; + +async function captureStdout(fn: () => Promise): Promise { + const originalWrite = process.stdout.write; + let output = ''; + process.stdout.write = ((chunk: string | Uint8Array) => { + output += typeof chunk === 'string' ? chunk : Buffer.from(chunk).toString('utf-8'); + return true; + }) as typeof process.stdout.write; + + try { + await fn(); + return output; + } finally { + process.stdout.write = originalWrite; + } +} + +async function captureLog(fn: () => Promise): Promise { + const originalLog = console.log; + let output = ''; + console.log = (msg: unknown) => { output += String(msg); }; + + try { + await fn(); + return output; + } finally { + console.log = originalLog; + } +} + +/** Run `fn` against a stubbed fetch and hand back what the command sent. */ +async function withStubbedFetch( + respond: () => Response, + fn: (sent: { url: string; init: RequestInit | undefined }) => Promise, +): Promise { + const originalFetch = globalThis.fetch; + const sent = { url: '', init: undefined as RequestInit | undefined }; + globalThis.fetch = (async (input, init) => { + sent.url = String(input); + sent.init = init; + return respond(); + }) as typeof fetch; + + try { + await fn(sent); + } finally { + globalThis.fetch = originalFetch; + } +} + +function jsonResponse(body: unknown): Response { + return new Response(JSON.stringify(body), { + status: 200, + headers: { 'Content-Type': 'application/json' }, + }); +} + +function makeTempAudio(contents = 'fake audio bytes'): { dir: string; filePath: string } { + const dir = mkdtempSync(join(tmpdir(), 'mmx-transcribe-test-')); + const filePath = join(dir, 'fixture.mp3'); + writeFileSync(filePath, contents); + return { dir, filePath }; +} + +describe('speech transcribe command', () => { + it('has correct name', () => { + expect(transcribeCommand.name).toBe('speech transcribe'); + }); + + it('requires --file in non-interactive mode', async () => { + await expect( + transcribeCommand.execute(baseConfig, baseFlags), + ).rejects.toThrow('Missing required argument: --file'); + }); + + it('throws when the audio file does not exist', async () => { + try { + await transcribeCommand.execute(baseConfig, { + ...baseFlags, + file: '/tmp/nonexistent-audio-xxxxx.mp3', + }); + throw new Error('Expected transcribe to reject'); + } catch (error) { + expect(error).toMatchObject({ + name: 'CLIError', + message: expect.stringContaining('File not found'), + exitCode: 2, + }); + } + }); + + it('rejects an invalid --response-format before uploading anything', async () => { + const { dir, filePath } = makeTempAudio(); + + try { + await expect( + transcribeCommand.execute(baseConfig, { ...baseFlags, file: filePath, responseFormat: 'txt' }), + ).rejects.toThrow(/Invalid audio format "txt"/); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); + + it('rejects an audio file above the 50 MB limit before uploading', async () => { + const { dir, filePath } = makeTempAudio(); + + try { + truncateSync(filePath, STT_MAX_FILE_BYTES + 1); + await expect( + transcribeCommand.execute(baseConfig, { ...baseFlags, file: filePath }), + ).rejects.toThrow(/speech-to-text allows at most 50 MB/); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); + + it('rejects --stream combined with a non-json response format', async () => { + const { dir, filePath } = makeTempAudio(); + + try { + await expect( + transcribeCommand.execute(baseConfig, { + ...baseFlags, + file: filePath, + stream: true, + responseFormat: 'srt', + }), + ).rejects.toThrow(/cannot be combined with stream=true/); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); + + it('rejects --stream combined with --out', async () => { + const { dir, filePath } = makeTempAudio(); + + try { + await expect( + transcribeCommand.execute(baseConfig, { + ...baseFlags, + file: filePath, + stream: true, + out: join(dir, 'out.txt'), + }), + ).rejects.toThrow(/--stream and --out cannot be combined/); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); + + it('shows the multipart fields in dry-run without network access', async () => { + const { dir, filePath } = makeTempAudio(); + + try { + const captured = await captureLog(async () => { + await transcribeCommand.execute( + { ...baseConfig, output: 'json' as const, dryRun: true }, + { ...baseFlags, dryRun: true, file: filePath, language: 'zh' }, + ); + }); + + const request = JSON.parse(captured).request; + expect(request.model).toBe('asr-1.0'); + expect(request.response_format).toBe('json'); + expect(request.language).toBe('zh'); + expect(request.file).toBe(filePath); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); + + it('uploads the audio as multipart and prints the transcript', async () => { + const { dir, filePath } = makeTempAudio('fixture audio'); + + try { + await withStubbedFetch( + () => jsonResponse({ text: 'hello world', duration: 1.5, trace_id: 'trace-1' }), + async (sent) => { + const captured = await captureStdout(async () => { + await transcribeCommand.execute(baseConfig, { ...baseFlags, file: filePath }); + }); + + expect(sent.url).toBe('https://api.mmx.io/v1/speech_to_text'); + expect(sent.init?.method).toBe('POST'); + expect(sent.init?.body).toBeInstanceOf(FormData); + + const body = sent.init?.body as FormData; + expect(body.get('model')).toBe('asr-1.0'); + expect(body.get('response_format')).toBe('json'); + expect(body.get('stream')).toBeNull(); + + const uploaded = body.get('file'); + expect(uploaded).toBeInstanceOf(Blob); + expect((uploaded as File).name).toBe('fixture.mp3'); + expect(await (uploaded as Blob).text()).toBe('fixture audio'); + + expect(captured).toBe('hello world\n'); + }, + ); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); + + it('sends --language as a request header, never as a form field', async () => { + const { dir, filePath } = makeTempAudio(); + + try { + await withStubbedFetch( + () => jsonResponse({ text: '你好', duration: 1 }), + async (sent) => { + await captureStdout(async () => { + await transcribeCommand.execute(baseConfig, { + ...baseFlags, + file: filePath, + language: 'zh', + }); + }); + + expect(sent.init?.headers).toMatchObject({ + Authorization: 'Bearer test-key', + language: 'zh', + }); + expect((sent.init?.body as FormData).get('language')).toBeNull(); + }, + ); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); + + it('passes --model and --timestamp-level through as form fields', async () => { + const { dir, filePath } = makeTempAudio(); + + try { + await withStubbedFetch( + () => jsonResponse({ text: 'hi', duration: 1, n_speakers: 1, segments: [] }), + async (sent) => { + await captureStdout(async () => { + await transcribeCommand.execute(baseConfig, { + ...baseFlags, + file: filePath, + model: 'asr-1.0', + responseFormat: 'verbose_json', + timestampLevel: 'word', + }); + }); + + const body = sent.init?.body as FormData; + expect(body.get('response_format')).toBe('verbose_json'); + expect(body.get('timestamp_level')).toBe('word'); + }, + ); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); + + it('prints the full response under --output json', async () => { + const { dir, filePath } = makeTempAudio(); + + try { + await withStubbedFetch( + () => jsonResponse({ + text: 'hi', + duration: 2, + n_speakers: 2, + segments: [{ id: 0, start: 0, end: 2, speaker: 'S1', text: 'hi' }], + }), + async () => { + const captured = await captureStdout(async () => { + await transcribeCommand.execute( + { ...baseConfig, output: 'json' as const }, + { ...baseFlags, file: filePath, responseFormat: 'verbose_json' }, + ); + }); + + const parsed = JSON.parse(captured); + expect(parsed.n_speakers).toBe(2); + expect(parsed.segments[0].speaker).toBe('S1'); + }, + ); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); + + it('prints srt subtitles verbatim', async () => { + const { dir, filePath } = makeTempAudio(); + const srt = '1\n00:00:00,080 --> 00:00:04,540\nhello\n\n'; + + try { + await withStubbedFetch( + () => new Response(srt, { status: 200, headers: { 'Content-Type': 'text/plain' } }), + async (sent) => { + const captured = await captureStdout(async () => { + await transcribeCommand.execute(baseConfig, { + ...baseFlags, + file: filePath, + responseFormat: 'srt', + }); + }); + + expect((sent.init?.body as FormData).get('response_format')).toBe('srt'); + expect(captured).toBe(srt); + }, + ); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); + + it('writes the result to --out and reports the saved path', async () => { + const { dir, filePath } = makeTempAudio(); + const outPath = join(dir, 'transcript.txt'); + + try { + await withStubbedFetch( + () => jsonResponse({ text: 'saved transcript', duration: 3.5 }), + async () => { + const captured = await captureLog(async () => { + await transcribeCommand.execute(baseConfig, { + ...baseFlags, + file: filePath, + out: outPath, + }); + }); + + expect(readFileSync(outPath, 'utf-8')).toBe('saved transcript\n'); + expect(captured).toContain('saved'); + expect(captured).toContain('3.5'); + }, + ); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); + + it('streams incremental text events to stdout', async () => { + const { dir, filePath } = makeTempAudio(); + const sse = [ + 'data: {"index":0,"delta":"你好","finish":false}', + '', + 'data: {"index":1,"delta":"世界","finish":false}', + '', + 'data: {"index":2,"delta":"","finish":true,"duration":2.25}', + '', + '', + ].join('\n'); + + try { + await withStubbedFetch( + () => new Response(sse, { status: 200, headers: { 'Content-Type': 'text/event-stream' } }), + async (sent) => { + const captured = await captureStdout(async () => { + await transcribeCommand.execute(baseConfig, { + ...baseFlags, + file: filePath, + stream: true, + }); + }); + + expect((sent.init?.body as FormData).get('stream')).toBe('true'); + expect(captured).toBe('你好世界\n'); + }, + ); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); + + it('accumulates streamed text into a single json result under --output json', async () => { + const { dir, filePath } = makeTempAudio(); + const sse = [ + 'data: {"index":0,"delta":"par","finish":false}', + '', + 'data: {"index":1,"delta":"tial","finish":false}', + '', + 'data: {"index":2,"delta":"","finish":true,"duration":1.75}', + '', + '', + ].join('\n'); + + try { + await withStubbedFetch( + () => new Response(sse, { status: 200, headers: { 'Content-Type': 'text/event-stream' } }), + async () => { + const captured = await captureStdout(async () => { + await transcribeCommand.execute( + { ...baseConfig, output: 'json' as const }, + { ...baseFlags, file: filePath, stream: true }, + ); + }); + + const parsed = JSON.parse(captured); + expect(parsed.text).toBe('partial'); + expect(parsed.duration).toBe(1.75); + }, + ); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); +}); diff --git a/test/sdk/speech.test.ts b/test/sdk/speech.test.ts index 7f8a886e..22867e3a 100644 --- a/test/sdk/speech.test.ts +++ b/test/sdk/speech.test.ts @@ -2,10 +2,10 @@ import { describe, it, expect, afterEach } from 'bun:test'; import { createMockServer, jsonResponse, type MockServer } from '../helpers/mock-server'; import { MiniMaxSDK } from '../../src/sdk'; import { SpeechSDK } from '../../src/sdk/speech'; -import { existsSync, unlinkSync, readFileSync } from 'node:fs'; +import { existsSync, mkdtempSync, rmSync, unlinkSync, readFileSync, writeFileSync } from 'node:fs'; import { join } from 'node:path'; import { tmpdir } from 'node:os'; -import type { SpeechResponse } from '../../src/types/api'; +import type { SpeechResponse, SpeechToTextStreamEvent } from '../../src/types/api'; function makeSpeechResponse(hexAudio?: string): SpeechResponse { return { @@ -118,3 +118,134 @@ describe('SpeechSDK.validateParams', () => { await expect(sdk.synthesize({ text: '' })).rejects.toThrow('text is required'); }); }); + +describe('SpeechSDK.transcribe', () => { + const sdk = new SpeechSDK({ apiKey: 'sk-test', baseUrl: 'https://api.mmx.io' }); + + async function withStubbedFetch( + respond: () => Response, + fn: (sent: { url: string; init: RequestInit | undefined }) => Promise, + ): Promise { + const originalFetch = globalThis.fetch; + const sent = { url: '', init: undefined as RequestInit | undefined }; + globalThis.fetch = (async (input, init) => { + sent.url = String(input); + sent.init = init; + return respond(); + }) as typeof fetch; + + try { + await fn(sent); + } finally { + globalThis.fetch = originalFetch; + } + } + + function withTempAudio(contents: string): { filePath: string; cleanup: () => void } { + const dir = mkdtempSync(join(tmpdir(), 'mmx-asr-sdk-')); + const filePath = join(dir, 'clip.mp3'); + writeFileSync(filePath, contents); + return { filePath, cleanup: () => rmSync(dir, { recursive: true, force: true }) }; + } + + it('uploads the audio as multipart and returns the transcript', async () => { + const { filePath, cleanup } = withTempAudio('sdk audio'); + + try { + await withStubbedFetch( + () => new Response(JSON.stringify({ text: 'transcribed', duration: 2.5 }), { + status: 200, + headers: { 'Content-Type': 'application/json' }, + }), + async (sent) => { + const result = await sdk.transcribe({ file: filePath, language: 'zh' }); + + expect(sent.url).toBe('https://api.mmx.io/v1/speech_to_text'); + expect(sent.init?.method).toBe('POST'); + expect(sent.init?.headers).toMatchObject({ language: 'zh' }); + + const body = sent.init?.body as FormData; + expect(body.get('model')).toBe('asr-1.0'); + expect(body.get('response_format')).toBe('json'); + expect(body.get('language')).toBeNull(); + + const uploaded = body.get('file'); + expect((uploaded as File).name).toBe('clip.mp3'); + expect(await (uploaded as Blob).text()).toBe('sdk audio'); + + expect(result.text).toBe('transcribed'); + expect(result.duration).toBe(2.5); + }, + ); + } finally { + cleanup(); + } + }); + + it('accepts a Blob instead of a path', async () => { + await withStubbedFetch( + () => new Response(JSON.stringify({ text: 'blob input', duration: 1 }), { + status: 200, + headers: { 'Content-Type': 'application/json' }, + }), + async (sent) => { + const result = await sdk.transcribe({ file: new Blob(['blob audio']) }); + + expect((sent.init?.body as FormData).get('file')).toBeInstanceOf(Blob); + expect(result.text).toBe('blob input'); + }, + ); + }); + + it('yields streamed events when stream is enabled', async () => { + const { filePath, cleanup } = withTempAudio('streamed audio'); + const sse = [ + 'data: {"index":0,"delta":"a","finish":false}', + '', + 'data: {"index":1,"delta":"","finish":true,"duration":9.5}', + '', + '', + ].join('\n'); + + try { + await withStubbedFetch( + () => new Response(sse, { status: 200, headers: { 'Content-Type': 'text/event-stream' } }), + async (sent) => { + const events: SpeechToTextStreamEvent[] = []; + for await (const event of await sdk.transcribe({ file: filePath, stream: true })) { + events.push(event); + } + + expect((sent.init?.body as FormData).get('stream')).toBe('true'); + expect(events).toHaveLength(2); + expect(events[0]!.delta).toBe('a'); + expect(events[1]!.duration).toBe(9.5); + }, + ); + } finally { + cleanup(); + } + }); + + it('throws when file is missing', async () => { + await expect(sdk.transcribe({ file: '' })).rejects.toThrow('file is required'); + }); + + it('throws when the file does not exist', async () => { + await expect( + sdk.transcribe({ file: '/tmp/does-not-exist-xxxxx.mp3' }), + ).rejects.toThrow('File not found'); + }); + + it('throws when stream is combined with a non-json response format', async () => { + const { filePath, cleanup } = withTempAudio('audio'); + + try { + await expect( + sdk.transcribe({ file: filePath, stream: true, response_format: 'srt' }), + ).rejects.toThrow(/cannot be combined with stream=true/); + } finally { + cleanup(); + } + }); +}); diff --git a/test/utils/stt.test.ts b/test/utils/stt.test.ts new file mode 100644 index 00000000..6a541164 --- /dev/null +++ b/test/utils/stt.test.ts @@ -0,0 +1,39 @@ +import { describe, it, expect } from 'bun:test'; +import { + STT_DEFAULT_MODEL, + STT_MAX_FILE_BYTES, + sttStreamFormatConflict, + validateSttFileSize, +} from '../../src/utils/stt'; + +describe('stt', () => { + it('exposes asr-1.0 as the default model', () => { + expect(STT_DEFAULT_MODEL).toBe('asr-1.0'); + }); + + describe('validateSttFileSize', () => { + it('accepts a file exactly at the limit', () => { + expect(() => validateSttFileSize('clip.mp3', STT_MAX_FILE_BYTES)).not.toThrow(); + }); + + it('rejects a file above the limit', () => { + expect(() => validateSttFileSize('clip.wav', STT_MAX_FILE_BYTES + 1)) + .toThrow(/at most 50 MB/); + }); + }); + + describe('sttStreamFormatConflict', () => { + it('allows json while streaming', () => { + expect(sttStreamFormatConflict('json', true)).toBeUndefined(); + }); + + it.each(['verbose_json', 'srt', 'vtt'])( + 'reports %s as incompatible with streaming', + (format) => expect(sttStreamFormatConflict(format, true)).toMatch(/cannot be combined with stream=true/), + ); + + it('ignores the response format when not streaming', () => { + expect(sttStreamFormatConflict('srt', false)).toBeUndefined(); + }); + }); +}); From f95e24050634ce532b97045a0ea7e2473ead5f3d Mon Sep 17 00:00:00 2001 From: Wzdhehe Date: Sat, 19 Sep 2026 17:59:56 +0800 Subject: [PATCH 2/6] fix(speech): address review findings on speech transcribe Blind review found one hard contract bug and several consistency issues: - SDK: `response_format: 'srt' | 'vtt'` returned a subtitle document, but the SDK parsed the body as JSON and failed. It now resolves to that document as a string through a dedicated overload, matching what the CLI already did. - SDK now enforces the documented 50 MB limit for both paths and Blobs, so the limit no longer lives only in the command. - The multipart text parts are built once by `sttFormFields()` and shared by the CLI (dry-run preview and request) and the SDK, instead of assembling the same field group twice in each. - Subtitle detection moved next to the format lists in `utils/stt.ts` instead of string comparisons spread across the command. - `--response-format` now reports `Invalid response format` rather than the audio-format wording borrowed from TTS. - The streaming path reports the audio duration on stderr, which was previously discarded in text output mode. - Documented the `speech recognize` alias, and clarified in the skill that piped `--stream` output needs `--output text` (non-TTY defaults to json). - Tests reuse the shared mock-server helpers and cover the new SDK subtitle path, the SDK size guard, and the streamed duration. --- ERRORS.md | 10 ++-- README.md | 4 +- README_CN.md | 4 +- skill/SKILL.md | 10 +++- src/commands/speech/transcribe.ts | 54 ++++++++++------- src/sdk/speech/index.ts | 62 +++++++++++++------- src/utils/stt.ts | 77 ++++++++++++++++++++----- test/commands/speech/transcribe.test.ts | 75 +++++++++++++----------- test/sdk/speech.test.ts | 58 +++++++++++++------ test/utils/stt.test.ts | 77 ++++++++++++++++++++----- 10 files changed, 303 insertions(+), 128 deletions(-) diff --git a/ERRORS.md b/ERRORS.md index bc40ca33..9c810074 100644 --- a/ERRORS.md +++ b/ERRORS.md @@ -119,17 +119,13 @@ This document lists all error scenarios and the messages users will see. | `--out` path no write permission | `Permission denied: cannot write to "${outPath}".` | | Disk full | `Disk full — cannot write audio file.` | -### `mmx speech voices` - -All errors fall under [Network Errors](#networkerrors). - ### `mmx speech transcribe` | Scenario | Error Message | |---|---| | No `--file` (and no positional path) in non-interactive mode | `Missing required argument: --file` | | Audio file not found | `File not found: ${fullPath}` | -| Invalid `--response-format` | `Invalid audio format "${fmt}". Supported: json, verbose_json, srt, vtt` | +| Invalid `--response-format` | `Invalid response format "${fmt}". Supported: json, verbose_json, srt, vtt` | | Audio file above 50 MB | `Audio file is ${size} MB; speech-to-text allows at most 50 MB: ${fullPath}` | | `--stream` with a response format other than `json` | `response_format "${fmt}" cannot be combined with stream=true; streaming returns incremental json only.` | | `--stream` together with `--out` | `--stream and --out cannot be combined.` | @@ -137,6 +133,10 @@ All errors fall under [Network Errors](#networkerrors). Audio longer than 500 seconds and unsupported/corrupt audio are rejected by the API; the server message is surfaced verbatim (e.g. `API error: invalid params, ... (HTTP 400)`). +### `mmx speech voices` + +All errors fall under [Network Errors](#networkerrors). + --- ## Vision Commands diff --git a/README.md b/README.md index bf2c0bfd..27be66a7 100644 --- a/README.md +++ b/README.md @@ -130,7 +130,9 @@ mmx speech transcribe --file long.mp3 --stream 50 MB and 500 seconds; larger files are rejected locally before upload. Omitting `--language` enables mixed-language recognition. `--response-format json` (default) prints the transcript, `verbose_json` adds speakers and per-segment timestamps, and `srt` / `vtt` return -subtitle documents. `--stream` prints incremental text and requires `json`. +subtitle documents. `--stream` prints incremental text and requires `json`; when stdout is not +a terminal it accumulates into a single JSON result unless `--output text` is passed. +`mmx speech recognize` is an alias for `mmx speech transcribe`. ### `mmx vision` diff --git a/README_CN.md b/README_CN.md index 63fd452c..300445f7 100644 --- a/README_CN.md +++ b/README_CN.md @@ -124,7 +124,9 @@ mmx speech transcribe --file long.mp3 --stream `mmx speech transcribe` 支持 wav、aiff、flac、m4a、mp3、aac、opus、ogg 格式，音频不超过 50 MB、时长不超过 500 秒；超出时会在上传前直接报错。不传 `--language` 时启用混合语言识别。 `--response-format json`（默认）输出转写文本，`verbose_json` 附带说话人标识与分段 -时间戳，`srt` / `vtt` 直接返回字幕文档。`--stream` 逐段输出文本，仅支持 `json`。 +时间戳，`srt` / `vtt` 直接返回字幕文档。`--stream` 逐段输出文本，仅支持 `json`；当 stdout +不是终端时会汇总为单个 JSON 结果，除非显式传 `--output text`。 +`mmx speech recognize` 是 `mmx speech transcribe` 的别名。 ### `mmx vision` diff --git a/skill/SKILL.md b/skill/SKILL.md index 690024d3..5b292f16 100644 --- a/skill/SKILL.md +++ b/skill/SKILL.md @@ -238,13 +238,21 @@ mmx speech transcribe --file talk.mp3 --response-format srt --out talk.srt # saves talk.srt; stdout: {"saved":".../talk.srt"} mmx speech transcribe --file long.mp3 --stream -# stdout: text as it is recognized, no trailing metadata +# stdout: text as it is recognized (add --output text when piping) ``` Notes: +- `mmx speech recognize` is an alias for `mmx speech transcribe`. - `--stream` cannot be combined with `--out`; redirect stdout instead. +- `--stream` prints deltas only in text output mode. stdout that is not a terminal defaults to + `json` (as everywhere else in this CLI), which accumulates the streamed text into one JSON + result — pass `--output text` to pipe streamed text, e.g. + `mmx speech transcribe --file long.mp3 --stream --output text > transcript.txt`. - `srt` / `vtt` results are subtitle documents and are printed or saved verbatim. - Without `--out`, `--output json` prints the full API response for `json` / `verbose_json`. +- Input validation (missing file, unsupported format, over 50 MB, `--stream` with a non-json + format) fails before anything is uploaded; the API stays the authority for the 500 s duration + limit and codec support. --- diff --git a/src/commands/speech/transcribe.ts b/src/commands/speech/transcribe.ts index e3ba0a9c..6113bf9a 100644 --- a/src/commands/speech/transcribe.ts +++ b/src/commands/speech/transcribe.ts @@ -8,22 +8,25 @@ import { parseSSE } from '../../client/stream'; import { speechToTextEndpoint } from '../../client/endpoints'; import { resolveFileUploadPath } from '../../files/upload'; import { detectOutputFormat, dryRun, formatOutput } from '../../output/formatter'; -import { formatList, validateAudioFormat } from '../../utils/audio-formats'; +import { formatList } from '../../utils/audio-formats'; import { STT_DEFAULT_MODEL, STT_RESPONSE_FORMATS, - sttStreamFormatConflict, + isSubtitleFormat, + sttFormFields, validateSttFileSize, + validateSttResponseFormat, + validateSttStreaming, } from '../../utils/stt'; import { promptOrFail } from '../../utils/prompt'; import type { Config } from '../../config/schema'; import type { GlobalFlags } from '../../types/flags'; -import type { SpeechToTextResponse, SpeechToTextStreamEvent } from '../../types/api'; - -/** srt/vtt come back as subtitle documents, not as JSON. */ -function isSubtitleFormat(responseFormat: string): boolean { - return responseFormat === 'srt' || responseFormat === 'vtt'; -} +import type { + SpeechToTextFormat, + SpeechToTextResponse, + SpeechToTextStreamEvent, + SpeechToTextTimestampLevel, +} from '../../types/api'; /** Results are emitted verbatim, only guaranteeing a final newline. */ function withTrailingNewline(value: string): string { @@ -74,31 +77,34 @@ export default defineCommand({ const stream = flags.stream === true; const outPath = flags.out ? resolve(flags.out as string) : undefined; - validateAudioFormat(responseFormat, STT_RESPONSE_FORMATS); + validateSttResponseFormat(responseFormat); validateSttFileSize(fullPath, statSync(fullPath).size); + validateSttStreaming(responseFormat, stream); - const streamConflict = sttStreamFormatConflict(responseFormat, stream); - if (streamConflict) throw new CLIError(streamConflict, ExitCode.USAGE); if (stream && outPath) { throw new CLIError( '--stream and --out cannot be combined.', ExitCode.USAGE, - 'Redirect stdout instead: mmx speech transcribe --file --stream > transcript.txt', + 'Redirect stdout instead: mmx speech transcribe --file --stream --output text > transcript.txt', ); } - const preview: Record = { model, response_format: responseFormat, file: fullPath }; + // One source for the multipart text parts, so the dry-run preview and the + // request cannot drift apart. + const fields = sttFormFields({ + model, + response_format: responseFormat as SpeechToTextFormat, + timestamp_level: timestampLevel as SpeechToTextTimestampLevel | undefined, + stream, + }); + + const preview: Record = { ...fields, file: fullPath }; if (language) preview.language = language; - if (timestampLevel) preview.timestamp_level = timestampLevel; - if (stream) preview.stream = true; if (dryRun(config, preview)) return; const form = new FormData(); - form.append('model', model); + for (const [field, value] of Object.entries(fields)) form.append(field, value); form.append('file', new Blob([readFileSync(fullPath)]), basename(fullPath)); - form.append('response_format', responseFormat); - if (timestampLevel) form.append('timestamp_level', timestampLevel); - if (stream) form.append('stream', 'true'); // `language` only takes effect as a request header: the API accepts (and // ignores) the same value as a form field. @@ -108,6 +114,8 @@ export default defineCommand({ const url = speechToTextEndpoint(config.baseUrl); const format = detectOutputFormat(config.output); + if (!config.quiet) process.stderr.write(`[Model: ${model}]\n`); + if (stream) { const res = await request(config, { url, method: 'POST', body: form, headers, stream: true }); @@ -139,6 +147,12 @@ export default defineCommand({ if (chunk.finish) duration = chunk.duration; } + // Only the final event carries the audio duration, so it is reported once + // the stream is drained. + if (!config.quiet && duration !== undefined) { + process.stderr.write(`[Duration: ${duration}s]\n`); + } + if (toStdout) { process.stdout.write('\n'); } else { @@ -147,8 +161,6 @@ export default defineCommand({ return; } - if (!config.quiet) process.stderr.write(`[Model: ${model}]\n`); - let payload: string; let duration: number | undefined; diff --git a/src/sdk/speech/index.ts b/src/sdk/speech/index.ts index f898507e..a40a4554 100644 --- a/src/sdk/speech/index.ts +++ b/src/sdk/speech/index.ts @@ -1,10 +1,11 @@ -import { existsSync, mkdirSync, readFileSync, writeFileSync } from 'node:fs'; +import { existsSync, mkdirSync, readFileSync, statSync, writeFileSync } from 'node:fs'; import { basename, resolve, dirname } from 'node:path'; import { Client } from "../client"; import { speechEndpoint, speechToTextEndpoint, voicesEndpoint } from "../../client/endpoints"; import { SpeechRequest, SpeechResponse, + SpeechToTextFormat, SpeechToTextRequest, SpeechToTextResponse, SpeechToTextStreamEvent, @@ -15,21 +16,29 @@ import { SDKError } from "../../errors/base"; import { ExitCode } from "../../errors/codes"; import { toMerged } from "es-toolkit/object"; import { ModelPartial } from "../types"; -import { STT_DEFAULT_MODEL, sttStreamFormatConflict } from "../../utils/stt"; - +import { + isSubtitleFormat, + sttFormFields, + validateSttFileSize, + validateSttStreaming, +} from "../../utils/stt"; export type TranscribeParams = ModelPartial & { /** Local audio path, or a Blob/File to send as-is. */ file: string | Blob; /** * BCP-47 language hint (e.g. `zh`, `en`). Sent as the `language` request - * header: the API ignores it when passed as a form field, even though its - * curl example uses `-F "language=zh"`. + * header — the API accepts and ignores the same value as a form field. */ language?: string; }; export type TranscribeStreamParams = TranscribeParams & { stream: true }; +/** Subtitle formats come back as documents, not as JSON. */ +export type TranscribeSubtitleParams = TranscribeParams & { + response_format: Extract; +}; + function hexToBuffer(hex: string): Buffer { if (!/^[0-9a-fA-F]*$/.test(hex)) { throw new SDKError('API returned invalid audio data (not valid hex).', ExitCode.GENERAL); @@ -45,6 +54,12 @@ function defaultFilename(prefix: string, ext: string): string { return `${prefix}_${ts}.${ext}`; } +/** The uploaded filename, preferring a real File name over a placeholder. */ +function blobFilename(file: Blob): string { + const name = (file as File).name; + return typeof name === 'string' && name ? name : 'audio'; +} + /** Resolve a transcription input into the multipart `file` part. */ function prepareAudioUpload(file: string | Blob): { blob: Blob; filename: string } { if (typeof file === 'string') { @@ -52,11 +67,13 @@ function prepareAudioUpload(file: string | Blob): { blob: Blob; filename: string if (!existsSync(fullPath)) { throw new SDKError(`File not found: ${fullPath}`, ExitCode.USAGE); } + validateSttFileSize(fullPath, statSync(fullPath).size); return { blob: new Blob([readFileSync(fullPath)]), filename: basename(fullPath) }; } - const name = (file as File).name; - return { blob: file, filename: typeof name === 'string' && name ? name : 'audio' }; + const filename = blobFilename(file); + validateSttFileSize(filename, file.size); + return { blob: file, filename }; } export class SpeechSDK extends Client { @@ -111,17 +128,19 @@ export class SpeechSDK extends Client { /** * Transcribe an audio file (speech-to-text). * - * `json` and `verbose_json` resolve to a structured response; SRT and VTT are - * returned by the API as subtitle documents, so `response_format` is applied - * as-is and the raw body is surfaced by the caller. The audio is uploaded as + * `json` and `verbose_json` resolve to a structured response; `srt` and `vtt` + * are returned by the API as subtitle documents and resolve to that document + * as a string. `stream: true` resolves to a stream of incremental text events + * and is only valid with `response_format: 'json'`. The audio is uploaded as * `multipart/form-data`, and `language` travels as a request header — the API - * ignores it when sent as a form field. + * accepts and ignores the same value as a form field. */ async transcribe(params: TranscribeStreamParams): Promise>; + async transcribe(params: TranscribeSubtitleParams): Promise; async transcribe(params: TranscribeParams): Promise; async transcribe( params: TranscribeParams, - ): Promise> { + ): Promise> { const { file, language, model, response_format, timestamp_level, stream } = params; if (!file) { @@ -129,19 +148,17 @@ export class SpeechSDK extends Client { } const responseFormat = response_format ?? 'json'; - const streamConflict = sttStreamFormatConflict(responseFormat, stream === true); - if (streamConflict) { - throw new SDKError(streamConflict, ExitCode.USAGE); - } + validateSttStreaming(responseFormat, stream === true); const { blob, filename } = prepareAudioUpload(file); const form = new FormData(); - form.append('model', model ?? STT_DEFAULT_MODEL); + for (const [field, value] of Object.entries( + sttFormFields({ model, response_format: responseFormat, timestamp_level, stream }), + )) { + form.append(field, value); + } form.append('file', blob, filename); - form.append('response_format', responseFormat); - if (timestamp_level) form.append('timestamp_level', timestamp_level); - if (stream) form.append('stream', 'true'); const url = speechToTextEndpoint(this.config.baseUrl); const headers: Record = {}; @@ -152,6 +169,11 @@ export class SpeechSDK extends Client { return this.streamSSE(res); } + if (isSubtitleFormat(responseFormat)) { + const res = await this.request({ url, method: 'POST', body: form, headers }); + return await res.text(); + } + return this.requestJson({ url, method: 'POST', body: form, headers }); } diff --git a/src/utils/stt.ts b/src/utils/stt.ts index 2d955d27..a84be067 100644 --- a/src/utils/stt.ts +++ b/src/utils/stt.ts @@ -1,6 +1,7 @@ import { CLIError } from '../errors/base'; import { ExitCode } from '../errors/codes'; -import type { SpeechToTextFormat } from '../types/api'; +import { formatList } from './audio-formats'; +import type { SpeechToTextFormat, SpeechToTextTimestampLevel } from '../types/api'; /** The only model `POST /v1/speech_to_text` exposes today. */ export const STT_DEFAULT_MODEL = 'asr-1.0'; @@ -13,29 +14,79 @@ export const STT_RESPONSE_FORMATS: readonly SpeechToTextFormat[] = [ 'vtt', ]; +/** Formats the API returns as subtitle documents rather than as JSON. */ +export const STT_SUBTITLE_FORMATS: readonly SpeechToTextFormat[] = ['srt', 'vtt']; + +/** `stream=true` carries incremental json only. */ +export const STT_STREAM_FORMAT: SpeechToTextFormat = 'json'; + /** - * Documented upload limit for speech-to-text. Checked before the request so an - * oversized file fails locally instead of being uploaded only to come back as - * HTTP 413 — an uncompressed 500 s 48 kHz stereo WAV is ~92 MB. + * Documented upload limit. Checked before the request so an oversized file + * fails locally instead of being uploaded only to come back as HTTP 413 — an + * uncompressed 500 s 48 kHz stereo WAV is ~92 MB. */ export const STT_MAX_FILE_BYTES = 50 * 1024 * 1024; -export function validateSttFileSize(filePath: string, sizeBytes: number): void { - if (sizeBytes > STT_MAX_FILE_BYTES) { +/** The text parts of the multipart request; the audio travels as `file`. */ +export interface SttFields { + model?: string; + response_format?: SpeechToTextFormat; + timestamp_level?: SpeechToTextTimestampLevel; + stream?: boolean; +} + +/** Whether a response format is returned as a subtitle document. */ +export function isSubtitleFormat(format: string): boolean { + return (STT_SUBTITLE_FORMATS as readonly string[]).includes(format); +} + +/** + * The multipart text parts, in the order the API documents them, so the CLI and + * the SDK cannot drift over field names or encodings. + */ +export function sttFormFields({ + model, + response_format, + timestamp_level, + stream, +}: SttFields): Record { + const fields: Record = { + model: model ?? STT_DEFAULT_MODEL, + response_format: response_format ?? STT_STREAM_FORMAT, + }; + if (timestamp_level) fields.timestamp_level = timestamp_level; + if (stream) fields.stream = 'true'; + return fields; +} + +export function validateSttResponseFormat(format: string): void { + if (!(STT_RESPONSE_FORMATS as readonly string[]).includes(format)) { throw new CLIError( - `Audio file is ${(sizeBytes / 1024 / 1024).toFixed(1)} MB; speech-to-text allows at most ${STT_MAX_FILE_BYTES / 1024 / 1024} MB: ${filePath}`, + `Invalid response format "${format}". Supported: ${formatList(STT_RESPONSE_FORMATS)}`, ExitCode.USAGE, - 'Re-encode to compressed mono audio (e.g. mp3 / aac) or split it into smaller files.', ); } } /** * `stream=true` is only accepted together with `response_format=json`, so the - * combination is rejected before the audio is uploaded. Returns the message - * instead of throwing so the CLI and SDK can each raise their own error type. + * combination is rejected before the audio is uploaded. */ -export function sttStreamFormatConflict(responseFormat: string, stream: boolean): string | undefined { - if (!stream || responseFormat === 'json') return undefined; - return `response_format "${responseFormat}" cannot be combined with stream=true; streaming returns incremental json only.`; +export function validateSttStreaming(responseFormat: string, stream: boolean): void { + if (stream && responseFormat !== STT_STREAM_FORMAT) { + throw new CLIError( + `response_format "${responseFormat}" cannot be combined with stream=true; streaming returns incremental json only.`, + ExitCode.USAGE, + ); + } +} + +export function validateSttFileSize(filePath: string, sizeBytes: number): void { + if (sizeBytes > STT_MAX_FILE_BYTES) { + throw new CLIError( + `Audio file is ${(sizeBytes / 1024 / 1024).toFixed(1)} MB; speech-to-text allows at most ${STT_MAX_FILE_BYTES / 1024 / 1024} MB: ${filePath}`, + ExitCode.USAGE, + 'Re-encode to compressed mono audio (e.g. mp3 / aac) or split it into smaller files.', + ); + } } diff --git a/test/commands/speech/transcribe.test.ts b/test/commands/speech/transcribe.test.ts index 1d3cec9d..7ff25842 100644 --- a/test/commands/speech/transcribe.test.ts +++ b/test/commands/speech/transcribe.test.ts @@ -4,6 +4,7 @@ import { tmpdir } from 'os'; import { join } from 'path'; import { default as transcribeCommand } from '../../../src/commands/speech/transcribe'; import { STT_MAX_FILE_BYTES } from '../../../src/utils/stt'; +import { jsonResponse, sseResponse } from '../../helpers/mock-server'; const baseConfig = { apiKey: 'test-key', @@ -60,6 +61,22 @@ async function captureLog(fn: () => Promise): Promise { } } +async function captureStderr(fn: () => Promise): Promise { + const originalWrite = process.stderr.write; + let output = ''; + process.stderr.write = ((chunk: string | Uint8Array) => { + output += typeof chunk === 'string' ? chunk : Buffer.from(chunk).toString('utf-8'); + return true; + }) as typeof process.stderr.write; + + try { + await fn(); + return output; + } finally { + process.stderr.write = originalWrite; + } +} + /** Run `fn` against a stubbed fetch and hand back what the command sent. */ async function withStubbedFetch( respond: () => Response, @@ -80,13 +97,6 @@ async function withStubbedFetch( } } -function jsonResponse(body: unknown): Response { - return new Response(JSON.stringify(body), { - status: 200, - headers: { 'Content-Type': 'application/json' }, - }); -} - function makeTempAudio(contents = 'fake audio bytes'): { dir: string; filePath: string } { const dir = mkdtempSync(join(tmpdir(), 'mmx-transcribe-test-')); const filePath = join(dir, 'fixture.mp3'); @@ -127,7 +137,7 @@ describe('speech transcribe command', () => { try { await expect( transcribeCommand.execute(baseConfig, { ...baseFlags, file: filePath, responseFormat: 'txt' }), - ).rejects.toThrow(/Invalid audio format "txt"/); + ).rejects.toThrow(/Invalid response format "txt"/); } finally { rmSync(dir, { recursive: true, force: true }); } @@ -368,32 +378,33 @@ describe('speech transcribe command', () => { } }); - it('streams incremental text events to stdout', async () => { + it('streams incremental text events to stdout and reports the duration', async () => { const { dir, filePath } = makeTempAudio(); - const sse = [ - 'data: {"index":0,"delta":"你好","finish":false}', - '', - 'data: {"index":1,"delta":"世界","finish":false}', - '', - 'data: {"index":2,"delta":"","finish":true,"duration":2.25}', - '', - '', - ].join('\n'); + const response = sseResponse([ + { data: '{"index":0,"delta":"你好","finish":false}' }, + { data: '{"index":1,"delta":"世界","finish":false}' }, + { data: '{"index":2,"delta":"","finish":true,"duration":2.25}' }, + ]); try { await withStubbedFetch( - () => new Response(sse, { status: 200, headers: { 'Content-Type': 'text/event-stream' } }), + () => response, async (sent) => { - const captured = await captureStdout(async () => { - await transcribeCommand.execute(baseConfig, { - ...baseFlags, - file: filePath, - stream: true, + let captured = ''; + const stderr = await captureStderr(async () => { + captured = await captureStdout(async () => { + await transcribeCommand.execute(baseConfig, { + ...baseFlags, + file: filePath, + stream: true, + }); }); }); expect((sent.init?.body as FormData).get('stream')).toBe('true'); expect(captured).toBe('你好世界\n'); + expect(stderr).toContain('[Model: asr-1.0]'); + expect(stderr).toContain('[Duration: 2.25s]'); }, ); } finally { @@ -403,19 +414,15 @@ describe('speech transcribe command', () => { it('accumulates streamed text into a single json result under --output json', async () => { const { dir, filePath } = makeTempAudio(); - const sse = [ - 'data: {"index":0,"delta":"par","finish":false}', - '', - 'data: {"index":1,"delta":"tial","finish":false}', - '', - 'data: {"index":2,"delta":"","finish":true,"duration":1.75}', - '', - '', - ].join('\n'); + const response = sseResponse([ + { data: '{"index":0,"delta":"par","finish":false}' }, + { data: '{"index":1,"delta":"tial","finish":false}' }, + { data: '{"index":2,"delta":"","finish":true,"duration":1.75}' }, + ]); try { await withStubbedFetch( - () => new Response(sse, { status: 200, headers: { 'Content-Type': 'text/event-stream' } }), + () => response, async () => { const captured = await captureStdout(async () => { await transcribeCommand.execute( diff --git a/test/sdk/speech.test.ts b/test/sdk/speech.test.ts index 22867e3a..d3a740ba 100644 --- a/test/sdk/speech.test.ts +++ b/test/sdk/speech.test.ts @@ -1,11 +1,12 @@ import { describe, it, expect, afterEach } from 'bun:test'; -import { createMockServer, jsonResponse, type MockServer } from '../helpers/mock-server'; +import { createMockServer, jsonResponse, sseResponse, type MockServer } from '../helpers/mock-server'; import { MiniMaxSDK } from '../../src/sdk'; import { SpeechSDK } from '../../src/sdk/speech'; -import { existsSync, mkdtempSync, rmSync, unlinkSync, readFileSync, writeFileSync } from 'node:fs'; +import { existsSync, mkdtempSync, rmSync, truncateSync, unlinkSync, readFileSync, writeFileSync } from 'node:fs'; import { join } from 'node:path'; import { tmpdir } from 'node:os'; import type { SpeechResponse, SpeechToTextStreamEvent } from '../../src/types/api'; +import { STT_MAX_FILE_BYTES } from '../../src/utils/stt'; function makeSpeechResponse(hexAudio?: string): SpeechResponse { return { @@ -153,10 +154,7 @@ describe('SpeechSDK.transcribe', () => { try { await withStubbedFetch( - () => new Response(JSON.stringify({ text: 'transcribed', duration: 2.5 }), { - status: 200, - headers: { 'Content-Type': 'application/json' }, - }), + () => jsonResponse({ text: 'transcribed', duration: 2.5 }), async (sent) => { const result = await sdk.transcribe({ file: filePath, language: 'zh' }); @@ -184,10 +182,7 @@ describe('SpeechSDK.transcribe', () => { it('accepts a Blob instead of a path', async () => { await withStubbedFetch( - () => new Response(JSON.stringify({ text: 'blob input', duration: 1 }), { - status: 200, - headers: { 'Content-Type': 'application/json' }, - }), + () => jsonResponse({ text: 'blob input', duration: 1 }), async (sent) => { const result = await sdk.transcribe({ file: new Blob(['blob audio']) }); @@ -197,19 +192,46 @@ describe('SpeechSDK.transcribe', () => { ); }); + it('returns the subtitle document for srt, which the API sends as text', async () => { + const { filePath, cleanup } = withTempAudio('srt audio'); + const srt = '1\n00:00:00,080 --> 00:00:04,540\nhello\n\n'; + + try { + await withStubbedFetch( + () => new Response(srt, { status: 200, headers: { 'Content-Type': 'text/plain' } }), + async (sent) => { + const document = await sdk.transcribe({ file: filePath, response_format: 'srt' }); + + expect((sent.init?.body as FormData).get('response_format')).toBe('srt'); + expect(document).toBe(srt); + }, + ); + } finally { + cleanup(); + } + }); + + it('rejects audio above the documented 50 MB limit', async () => { + const { filePath, cleanup } = withTempAudio('oversized audio'); + + try { + truncateSync(filePath, STT_MAX_FILE_BYTES + 1); + await expect(sdk.transcribe({ file: filePath })).rejects.toThrow(/at most 50 MB/); + } finally { + cleanup(); + } + }); + it('yields streamed events when stream is enabled', async () => { const { filePath, cleanup } = withTempAudio('streamed audio'); - const sse = [ - 'data: {"index":0,"delta":"a","finish":false}', - '', - 'data: {"index":1,"delta":"","finish":true,"duration":9.5}', - '', - '', - ].join('\n'); + const response = sseResponse([ + { data: '{"index":0,"delta":"a","finish":false}' }, + { data: '{"index":1,"delta":"","finish":true,"duration":9.5}' }, + ]); try { await withStubbedFetch( - () => new Response(sse, { status: 200, headers: { 'Content-Type': 'text/event-stream' } }), + () => response, async (sent) => { const events: SpeechToTextStreamEvent[] = []; for await (const event of await sdk.transcribe({ file: filePath, stream: true })) { diff --git a/test/utils/stt.test.ts b/test/utils/stt.test.ts index 6a541164..0449a060 100644 --- a/test/utils/stt.test.ts +++ b/test/utils/stt.test.ts @@ -2,8 +2,11 @@ import { describe, it, expect } from 'bun:test'; import { STT_DEFAULT_MODEL, STT_MAX_FILE_BYTES, - sttStreamFormatConflict, + isSubtitleFormat, + sttFormFields, validateSttFileSize, + validateSttResponseFormat, + validateSttStreaming, } from '../../src/utils/stt'; describe('stt', () => { @@ -11,29 +14,75 @@ describe('stt', () => { expect(STT_DEFAULT_MODEL).toBe('asr-1.0'); }); - describe('validateSttFileSize', () => { - it('accepts a file exactly at the limit', () => { - expect(() => validateSttFileSize('clip.mp3', STT_MAX_FILE_BYTES)).not.toThrow(); + describe('sttFormFields', () => { + it('defaults to asr-1.0 and json', () => { + expect(sttFormFields({})).toEqual({ model: 'asr-1.0', response_format: 'json' }); }); - it('rejects a file above the limit', () => { - expect(() => validateSttFileSize('clip.wav', STT_MAX_FILE_BYTES + 1)) - .toThrow(/at most 50 MB/); + it('encodes stream as the multipart string "true"', () => { + expect(sttFormFields({ stream: true })).toEqual({ + model: 'asr-1.0', + response_format: 'json', + stream: 'true', + }); + }); + + it('passes model, response_format, and timestamp_level through', () => { + expect( + sttFormFields({ model: 'asr-1.0', response_format: 'verbose_json', timestamp_level: 'word' }), + ).toEqual({ + model: 'asr-1.0', + response_format: 'verbose_json', + timestamp_level: 'word', + }); + }); + }); + + describe('isSubtitleFormat', () => { + it.each(['srt', 'vtt'])('treats %s as a subtitle document', (format) => { + expect(isSubtitleFormat(format)).toBe(true); + }); + + it.each(['json', 'verbose_json'])('treats %s as json', (format) => { + expect(isSubtitleFormat(format)).toBe(false); + }); + }); + + describe('validateSttResponseFormat', () => { + it.each(['json', 'verbose_json', 'srt', 'vtt'])('accepts %s', (format) => { + expect(() => validateSttResponseFormat(format)).not.toThrow(); + }); + + it('rejects an unknown format, naming the response format', () => { + expect(() => validateSttResponseFormat('txt')).toThrow( + /Invalid response format "txt". Supported: json, verbose_json, srt, vtt/, + ); }); }); - describe('sttStreamFormatConflict', () => { + describe('validateSttStreaming', () => { it('allows json while streaming', () => { - expect(sttStreamFormatConflict('json', true)).toBeUndefined(); + expect(() => validateSttStreaming('json', true)).not.toThrow(); }); - it.each(['verbose_json', 'srt', 'vtt'])( - 'reports %s as incompatible with streaming', - (format) => expect(sttStreamFormatConflict(format, true)).toMatch(/cannot be combined with stream=true/), - ); + it.each(['verbose_json', 'srt', 'vtt'])('rejects %s while streaming', (format) => { + expect(() => validateSttStreaming(format, true)) + .toThrow(/cannot be combined with stream=true/); + }); it('ignores the response format when not streaming', () => { - expect(sttStreamFormatConflict('srt', false)).toBeUndefined(); + expect(() => validateSttStreaming('srt', false)).not.toThrow(); + }); + }); + + describe('validateSttFileSize', () => { + it('accepts a file exactly at the limit', () => { + expect(() => validateSttFileSize('clip.mp3', STT_MAX_FILE_BYTES)).not.toThrow(); + }); + + it('rejects a file above the limit', () => { + expect(() => validateSttFileSize('clip.wav', STT_MAX_FILE_BYTES + 1)) + .toThrow(/at most 50 MB/); }); }); }); From 64dd423401d46f7346e1da34074729f18360fdf5 Mon Sep 17 00:00:00 2001 From: Wzdhehe Date: Sat, 19 Sep 2026 18:06:27 +0800 Subject: [PATCH 3/6] fix(speech): tighten transcribe stream and output handling Second review round, second axis (API contract fidelity): - The stream loop now stops on the API's final `finish` event, and warns on stderr when a stream ends without it, instead of silently returning a possibly truncated transcript. - `--out` write failures no longer surface as a raw filesystem error for the disk-full case; ERRORS.md documents both outcomes. - The SDK validates `response_format` too, so the CLI and SDK reject the same values instead of only the CLI doing so. - Documentation no longer implies the 500 s limit is checked locally (only the 50 MB size limit is), and says `n_speakers` / `segments` need `--output json`. - `validateSttFileSize` takes a display label rather than a path, since the SDK passes a Blob filename; `utils/stt.ts` no longer borrows `formatList` from the TTS module. --- ERRORS.md | 2 ++ README.md | 14 ++++++---- README_CN.md | 13 +++++---- skill/SKILL.md | 4 +++ src/commands/speech/transcribe.ts | 37 +++++++++++++++++++------ src/sdk/speech/index.ts | 15 ++++++++-- src/utils/stt.ts | 8 +++--- test/commands/speech/transcribe.test.ts | 30 ++++++++++++++++++++ test/sdk/speech.test.ts | 15 +++++++++- 9 files changed, 110 insertions(+), 28 deletions(-) diff --git a/ERRORS.md b/ERRORS.md index 9c810074..754efb08 100644 --- a/ERRORS.md +++ b/ERRORS.md @@ -129,6 +129,8 @@ This document lists all error scenarios and the messages users will see. | Audio file above 50 MB | `Audio file is ${size} MB; speech-to-text allows at most 50 MB: ${fullPath}` | | `--stream` with a response format other than `json` | `response_format "${fmt}" cannot be combined with stream=true; streaming returns incremental json only.` | | `--stream` together with `--out` | `--stream and --out cannot be combined.` | +| `--out` unwritable (permissions, missing directory) | `File system error: ${message}` | +| Disk full | `Disk full — cannot write transcript file.` | Audio longer than 500 seconds and unsupported/corrupt audio are rejected by the API; the server message is surfaced verbatim (e.g. `API error: invalid params, ... (HTTP 400)`). diff --git a/README.md b/README.md index 27be66a7..ace15c4b 100644 --- a/README.md +++ b/README.md @@ -127,12 +127,14 @@ mmx speech transcribe --file long.mp3 --stream ``` `mmx speech transcribe` accepts wav, aiff, flac, m4a, mp3, aac, opus, and ogg files up to -50 MB and 500 seconds; larger files are rejected locally before upload. Omitting -`--language` enables mixed-language recognition. `--response-format json` (default) prints the -transcript, `verbose_json` adds speakers and per-segment timestamps, and `srt` / `vtt` return -subtitle documents. `--stream` prints incremental text and requires `json`; when stdout is not -a terminal it accumulates into a single JSON result unless `--output text` is passed. -`mmx speech recognize` is an alias for `mmx speech transcribe`. +50 MB and 500 seconds. Audio above 50 MB is rejected locally before upload; the 500 second +limit is enforced by the API. Omitting `--language` enables mixed-language recognition. +`--response-format json` (default) prints the transcript, `verbose_json` adds speakers and +per-segment timestamps, and `srt` / `vtt` return subtitle documents. `n_speakers` and +`segments` are part of the response, so pass `--output json` to see them. `--stream` prints +incremental text and requires `json`; when stdout is not a terminal it accumulates into a single +JSON result unless `--output text` is passed. `mmx speech recognize` is an alias for +`mmx speech transcribe`. ### `mmx vision` diff --git a/README_CN.md b/README_CN.md index 300445f7..ed99c4df 100644 --- a/README_CN.md +++ b/README_CN.md @@ -121,12 +121,13 @@ mmx speech transcribe --file talk.mp3 --response-format srt --out talk.srt mmx speech transcribe --file long.mp3 --stream ``` -`mmx speech transcribe` 支持 wav、aiff、flac、m4a、mp3、aac、opus、ogg 格式，音频不超过 -50 MB、时长不超过 500 秒；超出时会在上传前直接报错。不传 `--language` 时启用混合语言识别。 -`--response-format json`（默认）输出转写文本，`verbose_json` 附带说话人标识与分段 -时间戳，`srt` / `vtt` 直接返回字幕文档。`--stream` 逐段输出文本，仅支持 `json`；当 stdout -不是终端时会汇总为单个 JSON 结果，除非显式传 `--output text`。 -`mmx speech recognize` 是 `mmx speech transcribe` 的别名。 +`mmx speech transcribe` 支持 wav、aiff、flac、m4a、mp3、aac、opus、ogg 格式，音频上限为 +50 MB / 500 秒。**超过 50 MB 会在上传前直接报错**；500 秒时长上限由服务端校验。 +不传 `--language` 时启用混合语言识别。`--response-format json`（默认）输出转写文本， +`verbose_json` 附带说话人标识与分段 时间戳，`srt` / `vtt` 直接返回字幕文档； +`n_speakers` 与 `segments` 属于响应字段，需加 `--output json` 才能看到。 +`--stream` 逐段输出文本，仅支持 `json`；当 stdout 不是终端时会汇总为单个 JSON 结果， +除非显式传 `--output text`。`mmx speech recognize` 是 `mmx speech transcribe` 的别名。 ### `mmx vision` diff --git a/skill/SKILL.md b/skill/SKILL.md index 5b292f16..d1d173c4 100644 --- a/skill/SKILL.md +++ b/skill/SKILL.md @@ -250,6 +250,10 @@ Notes: `mmx speech transcribe --file long.mp3 --stream --output text > transcript.txt`. - `srt` / `vtt` results are subtitle documents and are printed or saved verbatim. - Without `--out`, `--output json` prints the full API response for `json` / `verbose_json`. + Plain-text output prints the transcript only, so add `--output json` to read `n_speakers` + and `segments`. +- A stream that ends without the API's final event raises a warning on stderr, since the + transcript may be truncated. - Input validation (missing file, unsupported format, over 50 MB, `--stream` with a non-json format) fails before anything is uploaded; the API stays the authority for the 500 s duration limit and codec support. diff --git a/src/commands/speech/transcribe.ts b/src/commands/speech/transcribe.ts index 6113bf9a..46e87b95 100644 --- a/src/commands/speech/transcribe.ts +++ b/src/commands/speech/transcribe.ts @@ -8,7 +8,6 @@ import { parseSSE } from '../../client/stream'; import { speechToTextEndpoint } from '../../client/endpoints'; import { resolveFileUploadPath } from '../../files/upload'; import { detectOutputFormat, dryRun, formatOutput } from '../../output/formatter'; -import { formatList } from '../../utils/audio-formats'; import { STT_DEFAULT_MODEL, STT_RESPONSE_FORMATS, @@ -41,10 +40,10 @@ export default defineCommand({ options: [ { flag: '--file ', description: 'Audio file to transcribe (mp3, wav, m4a, flac, aac, opus, ogg, aiff)', required: true }, { flag: '--model ', description: `Model ID (default: ${STT_DEFAULT_MODEL})` }, - { flag: '--response-format ', description: `Transcription format: ${formatList(STT_RESPONSE_FORMATS)} (default: json)` }, + { flag: '--response-format ', description: `Transcription format: ${STT_RESPONSE_FORMATS.join(', ')} (default: json)` }, { flag: '--language ', description: 'BCP-47 language hint (zh, en, ja, ...); omit for automatic detection' }, { flag: '--timestamp-level ', description: 'Timestamp granularity: sentence, word (verbose_json / srt / vtt only)' }, - { flag: '--stream', description: 'Stream incremental text to stdout (json only)' }, + { flag: '--stream', description: 'Stream incremental text (json only; pair with --output text when piping)' }, { flag: '--out ', description: 'Write the result to a file instead of stdout' }, ], examples: [ @@ -129,6 +128,7 @@ export default defineCommand({ let text = ''; let duration: number | undefined; + let finished = false; const toStdout = format !== 'json'; for await (const event of parseSSE(res)) { if (event.data === '[DONE]') break; @@ -144,13 +144,15 @@ export default defineCommand({ text += chunk.delta; if (toStdout) process.stdout.write(chunk.delta); } - if (chunk.finish) duration = chunk.duration; + if (chunk.finish) { + duration = chunk.duration; + finished = true; + break; + } } - // Only the final event carries the audio duration, so it is reported once - // the stream is drained. - if (!config.quiet && duration !== undefined) { - process.stderr.write(`[Duration: ${duration}s]\n`); + if (!finished) { + process.stderr.write('[warning] Stream ended before the final event; the transcript may be incomplete.\n'); } if (toStdout) { @@ -158,6 +160,12 @@ export default defineCommand({ } else { process.stdout.write(withTrailingNewline(formatOutput({ text, duration }, format))); } + + // Only the final event carries the audio duration, so it is reported once + // the stream is drained — after stdout is closed out. + if (!config.quiet && duration !== undefined) { + process.stderr.write(`[Duration: ${duration}s]\n`); + } return; } @@ -179,7 +187,18 @@ export default defineCommand({ } if (outPath) { - writeFileSync(outPath, withTrailingNewline(payload), 'utf-8'); + try { + writeFileSync(outPath, withTrailingNewline(payload), 'utf-8'); + } catch (err) { + if ((err as NodeJS.ErrnoException).code === 'ENOSPC') { + throw new CLIError( + 'Disk full — cannot write transcript file.', + ExitCode.GENERAL, + 'Free up disk space and try again.', + ); + } + throw err; + } if (config.quiet) { console.log(outPath); } else { diff --git a/src/sdk/speech/index.ts b/src/sdk/speech/index.ts index a40a4554..3562337e 100644 --- a/src/sdk/speech/index.ts +++ b/src/sdk/speech/index.ts @@ -20,6 +20,7 @@ import { isSubtitleFormat, sttFormFields, validateSttFileSize, + validateSttResponseFormat, validateSttStreaming, } from "../../utils/stt"; export type TranscribeParams = ModelPartial & { @@ -147,8 +148,7 @@ export class SpeechSDK extends Client { throw new SDKError('file is required', ExitCode.USAGE); } - const responseFormat = response_format ?? 'json'; - validateSttStreaming(responseFormat, stream === true); + const responseFormat = this.validateTranscribeFormat(response_format, stream === true); const { blob, filename } = prepareAudioUpload(file); @@ -228,4 +228,15 @@ export class SpeechSDK extends Client { output_format: 'hex', }, params) as SpeechRequest; } + + /** Resolve the transcription response format, rejecting unusable values. */ + private validateTranscribeFormat( + responseFormat: SpeechToTextFormat | undefined, + stream: boolean, + ): SpeechToTextFormat { + const resolved = responseFormat ?? 'json'; + validateSttResponseFormat(resolved); + validateSttStreaming(resolved, stream); + return resolved; + } } diff --git a/src/utils/stt.ts b/src/utils/stt.ts index a84be067..78c34cf1 100644 --- a/src/utils/stt.ts +++ b/src/utils/stt.ts @@ -1,6 +1,5 @@ import { CLIError } from '../errors/base'; import { ExitCode } from '../errors/codes'; -import { formatList } from './audio-formats'; import type { SpeechToTextFormat, SpeechToTextTimestampLevel } from '../types/api'; /** The only model `POST /v1/speech_to_text` exposes today. */ @@ -62,7 +61,7 @@ export function sttFormFields({ export function validateSttResponseFormat(format: string): void { if (!(STT_RESPONSE_FORMATS as readonly string[]).includes(format)) { throw new CLIError( - `Invalid response format "${format}". Supported: ${formatList(STT_RESPONSE_FORMATS)}`, + `Invalid response format "${format}". Supported: ${STT_RESPONSE_FORMATS.join(', ')}`, ExitCode.USAGE, ); } @@ -81,10 +80,11 @@ export function validateSttStreaming(responseFormat: string, stream: boolean): v } } -export function validateSttFileSize(filePath: string, sizeBytes: number): void { +/** `source` is what the caller calls the audio: a path, or a Blob's filename. */ +export function validateSttFileSize(source: string, sizeBytes: number): void { if (sizeBytes > STT_MAX_FILE_BYTES) { throw new CLIError( - `Audio file is ${(sizeBytes / 1024 / 1024).toFixed(1)} MB; speech-to-text allows at most ${STT_MAX_FILE_BYTES / 1024 / 1024} MB: ${filePath}`, + `Audio file is ${(sizeBytes / 1024 / 1024).toFixed(1)} MB; speech-to-text allows at most ${STT_MAX_FILE_BYTES / 1024 / 1024} MB: ${source}`, ExitCode.USAGE, 'Re-encode to compressed mono audio (e.g. mp3 / aac) or split it into smaller files.', ); diff --git a/test/commands/speech/transcribe.test.ts b/test/commands/speech/transcribe.test.ts index 7ff25842..5ec312f5 100644 --- a/test/commands/speech/transcribe.test.ts +++ b/test/commands/speech/transcribe.test.ts @@ -412,6 +412,36 @@ describe('speech transcribe command', () => { } }); + it('warns on stderr when the stream ends without the final event', async () => { + const { dir, filePath } = makeTempAudio(); + const response = sseResponse([ + { data: '{"index":0,"delta":"truncated","finish":false}' }, + ]); + + try { + await withStubbedFetch( + () => response, + async () => { + let captured = ''; + const stderr = await captureStderr(async () => { + captured = await captureStdout(async () => { + await transcribeCommand.execute(baseConfig, { + ...baseFlags, + file: filePath, + stream: true, + }); + }); + }); + + expect(captured).toBe('truncated\n'); + expect(stderr).toContain('Stream ended before the final event'); + }, + ); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); + it('accumulates streamed text into a single json result under --output json', async () => { const { dir, filePath } = makeTempAudio(); const response = sseResponse([ diff --git a/test/sdk/speech.test.ts b/test/sdk/speech.test.ts index d3a740ba..5f6c16f1 100644 --- a/test/sdk/speech.test.ts +++ b/test/sdk/speech.test.ts @@ -5,7 +5,7 @@ import { SpeechSDK } from '../../src/sdk/speech'; import { existsSync, mkdtempSync, rmSync, truncateSync, unlinkSync, readFileSync, writeFileSync } from 'node:fs'; import { join } from 'node:path'; import { tmpdir } from 'node:os'; -import type { SpeechResponse, SpeechToTextStreamEvent } from '../../src/types/api'; +import type { SpeechResponse, SpeechToTextFormat, SpeechToTextStreamEvent } from '../../src/types/api'; import { STT_MAX_FILE_BYTES } from '../../src/utils/stt'; function makeSpeechResponse(hexAudio?: string): SpeechResponse { @@ -270,4 +270,17 @@ describe('SpeechSDK.transcribe', () => { cleanup(); } }); + + it('rejects an unknown response format before uploading', async () => { + const { filePath, cleanup } = withTempAudio('audio'); + const unknown = 'txt' as unknown as SpeechToTextFormat; + + try { + await expect( + sdk.transcribe({ file: filePath, response_format: unknown }), + ).rejects.toThrow(/Invalid response format "txt"/); + } finally { + cleanup(); + } + }); }); From dda4f86e10327803794355974c56f1552ba34ff9 Mon Sep 17 00:00:00 2001 From: Wzdhehe Date: Sat, 19 Sep 2026 18:12:11 +0800 Subject: [PATCH 4/6] fix(speech): release the SSE body after the final stream event Final blind review flagged that stopping at `finish: true` leaves the response body undrained, which can hold the connection (and the process) open. The body is now cancelled in a `finally`, matching the idiom already used by `agent/verify.ts`. Also records why `--timestamp-level` and `--model` are passed through instead of validated locally, so the omission reads as a decision rather than a gap. --- src/commands/speech/transcribe.ts | 46 +++++++++++++++++++------------ 1 file changed, 28 insertions(+), 18 deletions(-) diff --git a/src/commands/speech/transcribe.ts b/src/commands/speech/transcribe.ts index 46e87b95..13414fba 100644 --- a/src/commands/speech/transcribe.ts +++ b/src/commands/speech/transcribe.ts @@ -79,6 +79,10 @@ export default defineCommand({ validateSttResponseFormat(responseFormat); validateSttFileSize(fullPath, statSync(fullPath).size); validateSttStreaming(responseFormat, stream); + // --timestamp-level and --model are passed through, not validated here: the + // CLI never interprets them, and the API documents them as ignored (for + // timestamp_level with json) or server-validated, so a local allow-list + // would only go stale. if (stream && outPath) { throw new CLIError( @@ -130,25 +134,31 @@ export default defineCommand({ let duration: number | undefined; let finished = false; const toStdout = format !== 'json'; - for await (const event of parseSSE(res)) { - if (event.data === '[DONE]') break; - let chunk: SpeechToTextStreamEvent; - try { - chunk = JSON.parse(event.data) as SpeechToTextStreamEvent; - } catch (err) { - // Warn but keep going — partial text beats failing the whole run. - process.stderr.write(`[warning] Failed to parse stream chunk: ${err instanceof Error ? err.message : String(err)}\n`); - continue; - } - if (chunk.delta) { - text += chunk.delta; - if (toStdout) process.stdout.write(chunk.delta); - } - if (chunk.finish) { - duration = chunk.duration; - finished = true; - break; + try { + for await (const event of parseSSE(res)) { + if (event.data === '[DONE]') break; + let chunk: SpeechToTextStreamEvent; + try { + chunk = JSON.parse(event.data) as SpeechToTextStreamEvent; + } catch (err) { + // Warn but keep going — partial text beats failing the whole run. + process.stderr.write(`[warning] Failed to parse stream chunk: ${err instanceof Error ? err.message : String(err)}\n`); + continue; + } + if (chunk.delta) { + text += chunk.delta; + if (toStdout) process.stdout.write(chunk.delta); + } + if (chunk.finish) { + duration = chunk.duration; + finished = true; + break; + } } + } finally { + // Stopping at the final event leaves the SSE body undrained, which keeps + // the connection (and the process) alive; release it either way. + await res.body?.cancel().catch(() => undefined); } if (!finished) { From 82f8d9617e8ce0e76f77ad411e36ee066401d5f7 Mon Sep 17 00:00:00 2001 From: Wzdhehe Date: Sat, 19 Sep 2026 18:40:00 +0800 Subject: [PATCH 5/6] fix(speech): tighten transcribe error classes, stream contract, and output fidelity Review follow-ups on speech transcribe: - SDK validation now raises SDKError, not CLIError: the shared stt validators take the caller's error class, so consumers narrowing on SDKError no longer miss format/size/stream-combo rejections. - The SDK stream generator ends at the API's final event (finish=true) and releases the SSE body, mirroring the CLI fix; breaking out early is safe. CLI and SDK now share one submission path (submitSttForm), so the SSE / subtitle / json dispatch cannot drift between layers. - Stream deltas are assembled by index per the API contract: the CLI warns on out-of-order events and the json result is index-sorted. - srt/vtt saved with --out are byte-exact; only stdout keeps the trailing-newline convention. - mapApiError keys speech-to-text 422 (sensitive audio) off the endpoint + status instead of message text, and gives 413 a dedicated size-limit message with exit code 2. - withStubbedFetch extracted to test/helpers; help table alignment; SDK.md and ERRORS.md document the new behaviour. --- ERRORS.md | 2 + SDK.md | 5 +- src/commands/help.ts | 2 +- src/commands/speech/transcribe.ts | 53 ++++++---- src/errors/api.ts | 20 ++++ src/sdk/speech/index.ts | 54 +++++++--- src/utils/stt.ts | 77 ++++++++++++-- test/commands/speech/transcribe.test.ts | 79 ++++++++++---- test/errors/api.test.ts | 29 +++++ test/helpers/fetch-stub.ts | 19 ++++ test/sdk/speech.test.ts | 135 ++++++++++++++++++++---- test/utils/stt.test.ts | 24 +++++ 12 files changed, 415 insertions(+), 84 deletions(-) create mode 100644 test/helpers/fetch-stub.ts diff --git a/ERRORS.md b/ERRORS.md index 754efb08..88f8a4af 100644 --- a/ERRORS.md +++ b/ERRORS.md @@ -127,6 +127,8 @@ This document lists all error scenarios and the messages users will see. | Audio file not found | `File not found: ${fullPath}` | | Invalid `--response-format` | `Invalid response format "${fmt}". Supported: json, verbose_json, srt, vtt` | | Audio file above 50 MB | `Audio file is ${size} MB; speech-to-text allows at most 50 MB: ${fullPath}` | +| Server rejects the upload as too large (HTTP 413) | `Audio file exceeds the speech-to-text size limit (HTTP 413). ${message}` | +| Audio flagged by the sensitivity filter (HTTP 422) | `Input audio flagged by sensitivity filter (${message})` | | `--stream` with a response format other than `json` | `response_format "${fmt}" cannot be combined with stream=true; streaming returns incremental json only.` | | `--stream` together with `--out` | `--stream and --out cannot be combined.` | | `--out` unwritable (permissions, missing directory) | `File system error: ${message}` | diff --git a/SDK.md b/SDK.md index 6782b2b8..3239a86c 100644 --- a/SDK.md +++ b/SDK.md @@ -131,8 +131,11 @@ console.log(transcript.text, transcript.duration); // Speech-to-text, streamed const deltas = await sdk.speech.transcribe({ file: './meeting.mp3', stream: true }); for await (const event of deltas) { - process.stdout.write(event.delta); + process.stdout.write(event.delta); // concatenate delta values in `index` order } +// The generator ends at the API's final event (finish: true) and releases the +// connection, so breaking out of the loop early is safe. Unlike the CLI, which +// warns and continues, the SDK throws SDKError on a malformed stream chunk. ``` ### Vision diff --git a/src/commands/help.ts b/src/commands/help.ts index ae6a1e8a..25c5d702 100644 --- a/src/commands/help.ts +++ b/src/commands/help.ts @@ -12,7 +12,7 @@ interface ApiRef { const API_REFS: ApiRef[] = [ { command: 'mmx text chat', title: 'Text Generation (Chat Completion)', path: '/docs/api-reference/text-post' }, { command: 'mmx speech synthesize', title: 'Speech T2A (Text-to-Audio)', path: '/docs/api-reference/speech-t2a-http' }, - { command: 'mmx speech transcribe', title: 'Speech STT (Speech-to-Text)', path: '/docs/api-reference/speech-to-text' }, + { command: 'mmx speech transcribe', title: 'Speech STT (Speech-to-Text)', path: '/docs/api-reference/speech-to-text' }, { command: 'mmx image generate', title: 'Image Generation (T2I / I2I)', path: '/docs/api-reference/image-generation-t2i' }, { command: 'mmx video generate', title: 'Video Generation (T2V / I2V / S2V)', path: '/docs/api-reference/video-generation' }, { command: 'mmx search query', title: 'Web Search', path: '/docs/api-reference/web-search' }, diff --git a/src/commands/speech/transcribe.ts b/src/commands/speech/transcribe.ts index 13414fba..5a093273 100644 --- a/src/commands/speech/transcribe.ts +++ b/src/commands/speech/transcribe.ts @@ -3,7 +3,6 @@ import { basename, resolve } from 'node:path'; import { defineCommand } from '../../command'; import { CLIError } from '../../errors/base'; import { ExitCode } from '../../errors/codes'; -import { request, requestJson } from '../../client/http'; import { parseSSE } from '../../client/stream'; import { speechToTextEndpoint } from '../../client/endpoints'; import { resolveFileUploadPath } from '../../files/upload'; @@ -13,6 +12,7 @@ import { STT_RESPONSE_FORMATS, isSubtitleFormat, sttFormFields, + submitSttForm, validateSttFileSize, validateSttResponseFormat, validateSttStreaming, @@ -22,12 +22,14 @@ import type { Config } from '../../config/schema'; import type { GlobalFlags } from '../../types/flags'; import type { SpeechToTextFormat, - SpeechToTextResponse, SpeechToTextStreamEvent, SpeechToTextTimestampLevel, } from '../../types/api'; -/** Results are emitted verbatim, only guaranteeing a final newline. */ +/** + * Stdout only gets a guaranteed final newline (terminal convention); subtitle + * documents written with `--out` are byte-exact, per the API's own bytes. + */ function withTrailingNewline(value: string): string { return value.endsWith('\n') ? value : `${value}\n`; } @@ -119,8 +121,11 @@ export default defineCommand({ if (!config.quiet) process.stderr.write(`[Model: ${model}]\n`); - if (stream) { - const res = await request(config, { url, method: 'POST', body: form, headers, stream: true }); + // One submission for every format; the reply shape decides how it is read. + const submission = await submitSttForm({ config, url, form, headers, responseFormat, stream }); + + if (submission.kind === 'stream') { + const res = submission.res; const contentType = res.headers.get('content-type') || ''; if (!contentType.includes('text/event-stream')) { @@ -130,7 +135,10 @@ export default defineCommand({ ); } - let text = ''; + // The API numbers events from 0; deltas are concatenated by `index`, so a + // gap or repeat is surfaced instead of silently jumbling the transcript. + const parts: Array<{ index: number; delta: string }> = []; + let nextIndex = 0; let duration: number | undefined; let finished = false; const toStdout = format !== 'json'; @@ -145,8 +153,12 @@ export default defineCommand({ process.stderr.write(`[warning] Failed to parse stream chunk: ${err instanceof Error ? err.message : String(err)}\n`); continue; } + if (typeof chunk.index === 'number' && chunk.index !== nextIndex) { + process.stderr.write(`[warning] Stream events arrived out of order (expected index ${nextIndex}, got ${chunk.index}).\n`); + } + if (typeof chunk.index === 'number') nextIndex = chunk.index + 1; if (chunk.delta) { - text += chunk.delta; + parts.push({ index: typeof chunk.index === 'number' ? chunk.index : parts.length, delta: chunk.delta }); if (toStdout) process.stdout.write(chunk.delta); } if (chunk.finish) { @@ -165,6 +177,10 @@ export default defineCommand({ process.stderr.write('[warning] Stream ended before the final event; the transcript may be incomplete.\n'); } + // Assembled by index, per the API's contract — arrival order only shows + // through on stdout, where deltas are printed as they come. + const text = parts.sort((a, b) => a.index - b.index).map((part) => part.delta).join(''); + if (toStdout) { process.stdout.write('\n'); } else { @@ -182,23 +198,22 @@ export default defineCommand({ let payload: string; let duration: number | undefined; - if (isSubtitleFormat(responseFormat)) { - const res = await request(config, { url, method: 'POST', body: form, headers }); - payload = await res.text(); + if (submission.kind === 'subtitle') { + payload = submission.document; } else { - const response = await requestJson(config, { - url, - method: 'POST', - body: form, - headers, - }); - duration = response.duration; - payload = format === 'json' ? formatOutput(response, format) : response.text; + duration = submission.response.duration; + payload = format === 'json' ? formatOutput(submission.response, format) : submission.response.text; } if (outPath) { try { - writeFileSync(outPath, withTrailingNewline(payload), 'utf-8'); + // Subtitle documents are saved byte-exact; text transcripts keep the + // trailing-newline convention for plain-text files. + writeFileSync( + outPath, + isSubtitleFormat(responseFormat) ? payload : withTrailingNewline(payload), + 'utf-8', + ); } catch (err) { if ((err as NodeJS.ErrnoException).code === 'ENOSPC') { throw new CLIError( diff --git a/src/errors/api.ts b/src/errors/api.ts index 48fb3013..e91e43ec 100644 --- a/src/errors/api.ts +++ b/src/errors/api.ts @@ -69,6 +69,26 @@ export function mapApiError(status: number, body: ApiErrorBody, url?: string): C ); } + // The speech-to-text API uses 422 specifically for sensitive audio (1026). + // Its message language varies, so key off the endpoint + status rather than + // message text; other endpoints keep the heuristic below. + if (status === 422 && url?.includes('/speech_to_text')) { + return new CLIError( + `Input audio flagged by sensitivity filter${apiMsg ? ` (${apiMsg})` : ''}.`, + ExitCode.CONTENT_FILTER, + ); + } + + // The server-side counterpart of the local 50 MB pre-upload check; reached + // only if the two limits ever disagree. + if (status === 413 && url?.includes('/speech_to_text')) { + return new CLIError( + `Audio file exceeds the speech-to-text size limit (HTTP 413). ${apiMsg}`, + ExitCode.USAGE, + 'Re-encode to compressed mono audio (e.g. mp3 / aac) or split it into smaller files.', + ); + } + const isV2ContentFilter = status === 422 && errorType === 'unprocessable_entity_error' && diff --git a/src/sdk/speech/index.ts b/src/sdk/speech/index.ts index 3562337e..cbf0c959 100644 --- a/src/sdk/speech/index.ts +++ b/src/sdk/speech/index.ts @@ -17,8 +17,8 @@ import { ExitCode } from "../../errors/codes"; import { toMerged } from "es-toolkit/object"; import { ModelPartial } from "../types"; import { - isSubtitleFormat, sttFormFields, + submitSttForm, validateSttFileSize, validateSttResponseFormat, validateSttStreaming, @@ -68,12 +68,12 @@ function prepareAudioUpload(file: string | Blob): { blob: Blob; filename: string if (!existsSync(fullPath)) { throw new SDKError(`File not found: ${fullPath}`, ExitCode.USAGE); } - validateSttFileSize(fullPath, statSync(fullPath).size); + validateSttFileSize(fullPath, statSync(fullPath).size, SDKError); return { blob: new Blob([readFileSync(fullPath)]), filename: basename(fullPath) }; } const filename = blobFilename(file); - validateSttFileSize(filename, file.size); + validateSttFileSize(filename, file.size, SDKError); return { blob: file, filename }; } @@ -132,9 +132,11 @@ export class SpeechSDK extends Client { * `json` and `verbose_json` resolve to a structured response; `srt` and `vtt` * are returned by the API as subtitle documents and resolve to that document * as a string. `stream: true` resolves to a stream of incremental text events - * and is only valid with `response_format: 'json'`. The audio is uploaded as - * `multipart/form-data`, and `language` travels as a request header — the API - * accepts and ignores the same value as a form field. + * and is only valid with `response_format: 'json'`; the generator ends at the + * API's final event (`finish: true`) and releases the connection, so breaking + * out early is safe. Concatenate `delta` values in `index` order. The audio is + * uploaded as `multipart/form-data`, and `language` travels as a request + * header — the API accepts and ignores the same value as a form field. */ async transcribe(params: TranscribeStreamParams): Promise>; async transcribe(params: TranscribeSubtitleParams): Promise; @@ -164,17 +166,35 @@ export class SpeechSDK extends Client { const headers: Record = {}; if (language) headers.language = language; - if (stream) { - const res = await this.request({ url, method: 'POST', body: form, headers, stream: true }); - return this.streamSSE(res); - } + const submission = await submitSttForm({ + config: this.config, + url, + form, + headers, + responseFormat, + stream: stream === true, + }); - if (isSubtitleFormat(responseFormat)) { - const res = await this.request({ url, method: 'POST', body: form, headers }); - return await res.text(); - } + if (submission.kind === 'stream') return this.transcribeStream(submission.res); + if (submission.kind === 'subtitle') return submission.document; + return submission.response; + } - return this.requestJson({ url, method: 'POST', body: form, headers }); + /** + * Yield events until the API's final one. Stopping there leaves the SSE body + * undrained, which keeps the connection (and any open handles) alive; the + * `finally` releases it whether the stream ended on `finish` or the consumer + * broke out early. + */ + private async *transcribeStream(res: Response): AsyncGenerator { + try { + for await (const event of this.streamSSE(res)) { + yield event; + if (event.finish) break; + } + } finally { + await res.body?.cancel().catch(() => undefined); + } } /** @@ -235,8 +255,8 @@ export class SpeechSDK extends Client { stream: boolean, ): SpeechToTextFormat { const resolved = responseFormat ?? 'json'; - validateSttResponseFormat(resolved); - validateSttStreaming(resolved, stream); + validateSttResponseFormat(resolved, SDKError); + validateSttStreaming(resolved, stream, SDKError); return resolved; } } diff --git a/src/utils/stt.ts b/src/utils/stt.ts index 78c34cf1..4152a857 100644 --- a/src/utils/stt.ts +++ b/src/utils/stt.ts @@ -1,6 +1,15 @@ import { CLIError } from '../errors/base'; import { ExitCode } from '../errors/codes'; -import type { SpeechToTextFormat, SpeechToTextTimestampLevel } from '../types/api'; +import { request, requestJson } from '../client/http'; +import type { Config } from '../config/schema'; +import type { SpeechToTextFormat, SpeechToTextResponse, SpeechToTextTimestampLevel } from '../types/api'; + +/** + * The validators raise the caller's error class: the CLI passes nothing + * (defaults to `CLIError`), the SDK passes `SDKError` so consumers narrowing on + * it do not miss these. + */ +export type SttErrorCtor = new (message: string, exitCode: ExitCode, hint?: string) => CLIError; /** The only model `POST /v1/speech_to_text` exposes today. */ export const STT_DEFAULT_MODEL = 'asr-1.0'; @@ -58,9 +67,9 @@ export function sttFormFields({ return fields; } -export function validateSttResponseFormat(format: string): void { +export function validateSttResponseFormat(format: string, errorCtor: SttErrorCtor = CLIError): void { if (!(STT_RESPONSE_FORMATS as readonly string[]).includes(format)) { - throw new CLIError( + throw new errorCtor( `Invalid response format "${format}". Supported: ${STT_RESPONSE_FORMATS.join(', ')}`, ExitCode.USAGE, ); @@ -71,9 +80,13 @@ export function validateSttResponseFormat(format: string): void { * `stream=true` is only accepted together with `response_format=json`, so the * combination is rejected before the audio is uploaded. */ -export function validateSttStreaming(responseFormat: string, stream: boolean): void { +export function validateSttStreaming( + responseFormat: string, + stream: boolean, + errorCtor: SttErrorCtor = CLIError, +): void { if (stream && responseFormat !== STT_STREAM_FORMAT) { - throw new CLIError( + throw new errorCtor( `response_format "${responseFormat}" cannot be combined with stream=true; streaming returns incremental json only.`, ExitCode.USAGE, ); @@ -81,12 +94,62 @@ export function validateSttStreaming(responseFormat: string, stream: boolean): v } /** `source` is what the caller calls the audio: a path, or a Blob's filename. */ -export function validateSttFileSize(source: string, sizeBytes: number): void { +export function validateSttFileSize( + source: string, + sizeBytes: number, + errorCtor: SttErrorCtor = CLIError, +): void { if (sizeBytes > STT_MAX_FILE_BYTES) { - throw new CLIError( + throw new errorCtor( `Audio file is ${(sizeBytes / 1024 / 1024).toFixed(1)} MB; speech-to-text allows at most ${STT_MAX_FILE_BYTES / 1024 / 1024} MB: ${source}`, ExitCode.USAGE, 'Re-encode to compressed mono audio (e.g. mp3 / aac) or split it into smaller files.', ); } } + +/** The result of submitting a transcription, split by how the API replies. */ +export type SttSubmission = + | { kind: 'stream'; res: Response } + | { kind: 'subtitle'; document: string } + | { kind: 'json'; response: SpeechToTextResponse }; + +export interface SttSubmissionOpts { + config: Config; + url: string; + form: FormData; + headers: Record; + /** Already validated by the caller; decides how the reply is read. */ + responseFormat: string; + stream: boolean; +} + +/** + * Submit the multipart form once for the CLI and the SDK alike, so the + * three-way response handling (SSE / subtitle document / JSON) cannot drift + * between the two layers. + */ +export async function submitSttForm({ + config, + url, + form, + headers, + responseFormat, + stream, +}: SttSubmissionOpts): Promise { + if (stream) { + const res = await request(config, { url, method: 'POST', body: form, headers, stream: true }); + return { kind: 'stream', res }; + } + if (isSubtitleFormat(responseFormat)) { + const res = await request(config, { url, method: 'POST', body: form, headers }); + return { kind: 'subtitle', document: await res.text() }; + } + const response = await requestJson(config, { + url, + method: 'POST', + body: form, + headers, + }); + return { kind: 'json', response }; +} diff --git a/test/commands/speech/transcribe.test.ts b/test/commands/speech/transcribe.test.ts index 5ec312f5..d606acfa 100644 --- a/test/commands/speech/transcribe.test.ts +++ b/test/commands/speech/transcribe.test.ts @@ -5,6 +5,7 @@ import { join } from 'path'; import { default as transcribeCommand } from '../../../src/commands/speech/transcribe'; import { STT_MAX_FILE_BYTES } from '../../../src/utils/stt'; import { jsonResponse, sseResponse } from '../../helpers/mock-server'; +import { withStubbedFetch } from '../../helpers/fetch-stub'; const baseConfig = { apiKey: 'test-key', @@ -77,26 +78,6 @@ async function captureStderr(fn: () => Promise): Promise { } } -/** Run `fn` against a stubbed fetch and hand back what the command sent. */ -async function withStubbedFetch( - respond: () => Response, - fn: (sent: { url: string; init: RequestInit | undefined }) => Promise, -): Promise { - const originalFetch = globalThis.fetch; - const sent = { url: '', init: undefined as RequestInit | undefined }; - globalThis.fetch = (async (input, init) => { - sent.url = String(input); - sent.init = init; - return respond(); - }) as typeof fetch; - - try { - await fn(sent); - } finally { - globalThis.fetch = originalFetch; - } -} - function makeTempAudio(contents = 'fake audio bytes'): { dir: string; filePath: string } { const dir = mkdtempSync(join(tmpdir(), 'mmx-transcribe-test-')); const filePath = join(dir, 'fixture.mp3'); @@ -378,6 +359,33 @@ describe('speech transcribe command', () => { } }); + it('saves an srt document byte-exact, without an injected trailing newline', async () => { + const { dir, filePath } = makeTempAudio(); + const outPath = join(dir, 'talk.srt'); + // Deliberately no trailing newline: the file must keep the API's own bytes. + const srt = '1\n00:00:00,080 --> 00:00:04,540\nhello'; + + try { + await withStubbedFetch( + () => new Response(srt, { status: 200, headers: { 'Content-Type': 'text/plain' } }), + async () => { + await captureLog(async () => { + await transcribeCommand.execute(baseConfig, { + ...baseFlags, + file: filePath, + responseFormat: 'srt', + out: outPath, + }); + }); + + expect(readFileSync(outPath, 'utf-8')).toBe(srt); + }, + ); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); + it('streams incremental text events to stdout and reports the duration', async () => { const { dir, filePath } = makeTempAudio(); const response = sseResponse([ @@ -470,4 +478,35 @@ describe('speech transcribe command', () => { rmSync(dir, { recursive: true, force: true }); } }); + + it('assembles the transcript by index when events arrive out of order', async () => { + const { dir, filePath } = makeTempAudio(); + const response = sseResponse([ + { data: '{"index":1,"delta":"world","finish":false}' }, + { data: '{"index":0,"delta":"hello ","finish":false}' }, + { data: '{"index":2,"delta":"","finish":true,"duration":2}' }, + ]); + + try { + await withStubbedFetch( + () => response, + async () => { + let captured = ''; + const stderr = await captureStderr(async () => { + captured = await captureStdout(async () => { + await transcribeCommand.execute( + { ...baseConfig, output: 'json' as const }, + { ...baseFlags, file: filePath, stream: true }, + ); + }); + }); + + expect(JSON.parse(captured).text).toBe('hello world'); + expect(stderr).toContain('out of order'); + }, + ); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); }); diff --git a/test/errors/api.test.ts b/test/errors/api.test.ts index 6a56f329..1796259b 100644 --- a/test/errors/api.test.ts +++ b/test/errors/api.test.ts @@ -68,6 +68,35 @@ describe('mapApiError', () => { expect(err.exitCode).toBe(ExitCode.GENERAL); }); + it('maps speech-to-text 422 to CONTENT_FILTER without relying on message text', () => { + // The ASR API documents 422 as sensitive audio (1026); the message language + // varies, so the mapping keys off the endpoint, not the text. + const err = mapApiError(422, { + error: { + type: 'unprocessable_entity_error', + message: '音频校验未通过', + http_code: '422', + }, + }, 'https://api.minimax.cn/v1/speech_to_text'); + + expect(err.exitCode).toBe(ExitCode.CONTENT_FILTER); + expect(err.message).toContain('音频校验未通过'); + }); + + it('maps speech-to-text 413 to USAGE with the size-limit hint', () => { + const err = mapApiError(413, { + error: { + type: 'invalid_request_error', + message: 'body over 52428800 bytes', + http_code: '413', + }, + }, 'https://api.minimax.cn/v1/speech_to_text'); + + expect(err.exitCode).toBe(ExitCode.USAGE); + expect(err.message).toContain('size limit'); + expect(err.hint).toContain('Re-encode'); + }); + it('maps MiniMax quota code 1028', () => { const err = mapApiError(400, { base_resp: { status_code: 1028, status_msg: 'quota exhausted' } }); expect(err.exitCode).toBe(ExitCode.QUOTA); diff --git a/test/helpers/fetch-stub.ts b/test/helpers/fetch-stub.ts new file mode 100644 index 00000000..37da5706 --- /dev/null +++ b/test/helpers/fetch-stub.ts @@ -0,0 +1,19 @@ +/** Run `fn` against a stubbed fetch and hand back what the command sent. */ +export async function withStubbedFetch( + respond: () => Response, + fn: (sent: { url: string; init: RequestInit | undefined }) => Promise, +): Promise { + const originalFetch = globalThis.fetch; + const sent = { url: '', init: undefined as RequestInit | undefined }; + globalThis.fetch = (async (input, init) => { + sent.url = String(input); + sent.init = init; + return respond(); + }) as typeof fetch; + + try { + await fn(sent); + } finally { + globalThis.fetch = originalFetch; + } +} diff --git a/test/sdk/speech.test.ts b/test/sdk/speech.test.ts index 5f6c16f1..23694ac1 100644 --- a/test/sdk/speech.test.ts +++ b/test/sdk/speech.test.ts @@ -7,6 +7,8 @@ import { join } from 'node:path'; import { tmpdir } from 'node:os'; import type { SpeechResponse, SpeechToTextFormat, SpeechToTextStreamEvent } from '../../src/types/api'; import { STT_MAX_FILE_BYTES } from '../../src/utils/stt'; +import { withStubbedFetch } from '../helpers/fetch-stub'; +import { SDKError } from '../../src/errors/base'; function makeSpeechResponse(hexAudio?: string): SpeechResponse { return { @@ -123,25 +125,6 @@ describe('SpeechSDK.validateParams', () => { describe('SpeechSDK.transcribe', () => { const sdk = new SpeechSDK({ apiKey: 'sk-test', baseUrl: 'https://api.mmx.io' }); - async function withStubbedFetch( - respond: () => Response, - fn: (sent: { url: string; init: RequestInit | undefined }) => Promise, - ): Promise { - const originalFetch = globalThis.fetch; - const sent = { url: '', init: undefined as RequestInit | undefined }; - globalThis.fetch = (async (input, init) => { - sent.url = String(input); - sent.init = init; - return respond(); - }) as typeof fetch; - - try { - await fn(sent); - } finally { - globalThis.fetch = originalFetch; - } - } - function withTempAudio(contents: string): { filePath: string; cleanup: () => void } { const dir = mkdtempSync(join(tmpdir(), 'mmx-asr-sdk-')); const filePath = join(dir, 'clip.mp3'); @@ -249,6 +232,120 @@ describe('SpeechSDK.transcribe', () => { } }); + it('stops the stream at the final event, not at the end of the body', async () => { + const { filePath, cleanup } = withTempAudio('streamed audio'); + // An event after `finish` (and the helper's trailing [DONE]) must never + // reach the consumer: the API terminates the stream at finish=true. + const response = sseResponse([ + { data: '{"index":0,"delta":"done","finish":false}' }, + { data: '{"index":1,"delta":"","finish":true,"duration":3.25}' }, + { data: '{"index":2,"delta":"past the end","finish":false}' }, + ]); + + try { + await withStubbedFetch( + () => response, + async () => { + const events: SpeechToTextStreamEvent[] = []; + for await (const event of await sdk.transcribe({ file: filePath, stream: true })) { + events.push(event); + } + + expect(events).toHaveLength(2); + expect(events.at(-1)!.finish).toBe(true); + }, + ); + } finally { + cleanup(); + } + }); + + it('releases the SSE body when the consumer breaks out early', async () => { + const { filePath, cleanup } = withTempAudio('streamed audio'); + let cancelled = false; + const body = new ReadableStream({ + start(controller) { + controller.enqueue(new TextEncoder().encode( + 'data: {"index":0,"delta":"first","finish":false}\n\n' + + 'data: {"index":1,"delta":"","finish":true,"duration":1}\n\n', + )); + }, + cancel() { + cancelled = true; + }, + }); + const response = new Response(body, { + headers: { 'Content-Type': 'text/event-stream' }, + }); + + try { + await withStubbedFetch( + () => response, + async () => { + const seen: SpeechToTextStreamEvent[] = []; + for await (const event of await sdk.transcribe({ file: filePath, stream: true })) { + seen.push(event); + break; // consumer stops after the first event + } + expect(seen).toHaveLength(1); + await new Promise((resolve) => setTimeout(resolve, 0)); + expect(cancelled).toBe(true); + }, + ); + } finally { + cleanup(); + } + }); + + it('raises SDKError, not CLIError, for an invalid response format', async () => { + const { filePath, cleanup } = withTempAudio('audio'); + + try { + try { + await sdk.transcribe({ file: filePath, response_format: 'txt' as SpeechToTextFormat }); + throw new Error('Expected transcribe to reject'); + } catch (error) { + expect(error).toBeInstanceOf(SDKError); + expect((error as Error).message).toContain('Invalid response format "txt"'); + } + } finally { + cleanup(); + } + }); + + it('raises SDKError when stream is combined with a subtitle format', async () => { + const { filePath, cleanup } = withTempAudio('audio'); + + try { + try { + await sdk.transcribe({ file: filePath, stream: true, response_format: 'srt' }); + throw new Error('Expected transcribe to reject'); + } catch (error) { + expect(error).toBeInstanceOf(SDKError); + expect((error as Error).message).toContain('cannot be combined with stream=true'); + } + } finally { + cleanup(); + } + }); + + it('raises SDKError for audio above the 50 MB limit', async () => { + const { filePath, cleanup } = withTempAudio('oversized'); + truncateSync(filePath, STT_MAX_FILE_BYTES + 1); + + try { + try { + await sdk.transcribe({ file: filePath }); + throw new Error('Expected transcribe to reject'); + } catch (error) { + expect(error).toBeInstanceOf(SDKError); + expect((error as Error).message).toContain('at most 50 MB'); + } + } finally { + cleanup(); + } + }); + it('throws when file is missing', async () => { await expect(sdk.transcribe({ file: '' })).rejects.toThrow('file is required'); }); diff --git a/test/utils/stt.test.ts b/test/utils/stt.test.ts index 0449a060..9e4aa7ad 100644 --- a/test/utils/stt.test.ts +++ b/test/utils/stt.test.ts @@ -1,4 +1,5 @@ import { describe, it, expect } from 'bun:test'; +import { CLIError, SDKError } from '../../src/errors/base'; import { STT_DEFAULT_MODEL, STT_MAX_FILE_BYTES, @@ -85,4 +86,27 @@ describe('stt', () => { .toThrow(/at most 50 MB/); }); }); + + describe('error class', () => { + it('defaults to CLIError', () => { + try { + validateSttResponseFormat('txt'); + throw new Error('Expected validateSttResponseFormat to throw'); + } catch (error) { + expect(error).toBeInstanceOf(CLIError); + expect(error).not.toBeInstanceOf(SDKError); + } + }); + + it('raises the error class the caller passes, keeping message and exit code', () => { + try { + validateSttFileSize('clip.wav', STT_MAX_FILE_BYTES + 1, SDKError); + throw new Error('Expected validateSttFileSize to throw'); + } catch (error) { + expect(error).toBeInstanceOf(SDKError); + expect((error as SDKError).exitCode).toBe(2); + expect((error as SDKError).hint).toContain('Re-encode'); + } + }); + }); }); From 652e2f30dfee4073099d790dc598d8185d63afa9 Mon Sep 17 00:00:00 2001 From: saladday <1203511142@qq.com> Date: Sat, 19 Sep 2026 22:41:56 +0800 Subject: [PATCH 6/6] fix(speech): correct transcribe types and reject incomplete streams --- SDK.md | 2 ++ src/sdk/speech/index.ts | 28 +++++++++++++++++-- test/sdk/speech.test.ts | 61 +++++++++++++++++++++++++++++++++++++++-- 3 files changed, 85 insertions(+), 6 deletions(-) diff --git a/SDK.md b/SDK.md index 3239a86c..b6dbb5d2 100644 --- a/SDK.md +++ b/SDK.md @@ -136,6 +136,8 @@ for await (const event of deltas) { // The generator ends at the API's final event (finish: true) and releases the // connection, so breaking out of the loop early is safe. Unlike the CLI, which // warns and continues, the SDK throws SDKError on a malformed stream chunk. +// Non-SSE responses and streams ending without finish: true also throw SDKError +// during iteration; text received before an error may be incomplete. ``` ### Vision diff --git a/src/sdk/speech/index.ts b/src/sdk/speech/index.ts index cbf0c959..4ef6fcfb 100644 --- a/src/sdk/speech/index.ts +++ b/src/sdk/speech/index.ts @@ -33,11 +33,20 @@ export type TranscribeParams = ModelPartial & { language?: string; }; -export type TranscribeStreamParams = TranscribeParams & { stream: true }; +export type TranscribeStreamParams = TranscribeParams & { + stream: true; + response_format?: 'json'; +}; /** Subtitle formats come back as documents, not as JSON. */ export type TranscribeSubtitleParams = TranscribeParams & { response_format: Extract; + stream?: false; +}; + +export type TranscribeJsonParams = TranscribeParams & { + response_format?: Extract; + stream?: false; }; function hexToBuffer(hex: string): Buffer { @@ -140,7 +149,8 @@ export class SpeechSDK extends Client { */ async transcribe(params: TranscribeStreamParams): Promise>; async transcribe(params: TranscribeSubtitleParams): Promise; - async transcribe(params: TranscribeParams): Promise; + async transcribe(params: TranscribeJsonParams): Promise; + async transcribe(params: TranscribeParams): Promise>; async transcribe( params: TranscribeParams, ): Promise> { @@ -188,10 +198,22 @@ export class SpeechSDK extends Client { */ private async *transcribeStream(res: Response): AsyncGenerator { try { + const contentType = res.headers.get('content-type') || ''; + if (contentType.split(';', 1)[0]!.trim().toLowerCase() !== 'text/event-stream') { + throw new SDKError( + `Expected SSE stream but got content-type "${contentType}". Server may be experiencing issues.`, + ExitCode.GENERAL, + ); + } + for await (const event of this.streamSSE(res)) { yield event; - if (event.finish) break; + if (event.finish) return; } + throw new SDKError( + 'Stream ended before the final event; the transcript may be incomplete.', + ExitCode.GENERAL, + ); } finally { await res.body?.cancel().catch(() => undefined); } diff --git a/test/sdk/speech.test.ts b/test/sdk/speech.test.ts index 23694ac1..0f1120b4 100644 --- a/test/sdk/speech.test.ts +++ b/test/sdk/speech.test.ts @@ -1,11 +1,11 @@ -import { describe, it, expect, afterEach } from 'bun:test'; +import { describe, it, expect, expectTypeOf, afterEach } from 'bun:test'; import { createMockServer, jsonResponse, sseResponse, type MockServer } from '../helpers/mock-server'; import { MiniMaxSDK } from '../../src/sdk'; -import { SpeechSDK } from '../../src/sdk/speech'; +import { SpeechSDK, type TranscribeParams } from '../../src/sdk/speech'; import { existsSync, mkdtempSync, rmSync, truncateSync, unlinkSync, readFileSync, writeFileSync } from 'node:fs'; import { join } from 'node:path'; import { tmpdir } from 'node:os'; -import type { SpeechResponse, SpeechToTextFormat, SpeechToTextStreamEvent } from '../../src/types/api'; +import type { SpeechResponse, SpeechToTextFormat, SpeechToTextResponse, SpeechToTextStreamEvent } from '../../src/types/api'; import { STT_MAX_FILE_BYTES } from '../../src/utils/stt'; import { withStubbedFetch } from '../helpers/fetch-stub'; import { SDKError } from '../../src/errors/base'; @@ -205,6 +205,61 @@ describe('SpeechSDK.transcribe', () => { } }); + it('infers precise literal results and unions for dynamic transcription options', () => { + const inferResults = (params: TranscribeParams, stream: boolean, format: SpeechToTextFormat) => ({ + json: sdk.transcribe({ file: 'audio.mp3' }), + verbose: sdk.transcribe({ file: 'audio.mp3', response_format: 'verbose_json', stream: false }), + subtitle: sdk.transcribe({ file: 'audio.mp3', response_format: 'srt' }), + stream: sdk.transcribe({ file: 'audio.mp3', stream: true }), + dynamic: sdk.transcribe(params), + dynamicStream: sdk.transcribe({ file: 'audio.mp3', stream }), + dynamicFormat: sdk.transcribe({ file: 'audio.mp3', response_format: format }), + }); + type DynamicResult = Promise>; + expectTypeOf(inferResults).returns.toEqualTypeOf<{ + json: Promise; + verbose: Promise; + subtitle: Promise; + stream: Promise>; + dynamic: DynamicResult; + dynamicStream: DynamicResult; + dynamicFormat: DynamicResult; + }>(); + }); + + it('rejects a non-SSE stream response and releases its body', async () => { + let cancelled = false; + const response = new Response(new ReadableStream({ + start(controller) { + controller.enqueue(new TextEncoder().encode('{"text":"unexpected JSON"}')); + }, + cancel() { cancelled = true; }, + }), { headers: { 'Content-Type': 'application/json' } }); + + await withStubbedFetch(() => response, async () => { + const stream = await sdk.transcribe({ file: new Blob(['audio']), stream: true }); + await expect(stream.next()).rejects.toBeInstanceOf(SDKError); + expect(cancelled).toBe(true); + }); + }); + + for (const ending of ['eof', 'done'] as const) { + it(`rejects a stream ending with ${ending} before the final event`, async () => { + const partial = 'data: {"index":0,"delta":"partial","finish":false}\n\n'; + const response = new Response(partial + (ending === 'done' ? 'data: [DONE]\n\n' : ''), { + headers: { 'Content-Type': 'text/event-stream; charset=utf-8' }, + }); + await withStubbedFetch(() => response, async () => { + const stream = await sdk.transcribe({ file: new Blob(['audio']), stream: true }); + expect((await stream.next()).value?.delta).toBe('partial'); + await expect(stream.next()).rejects.toMatchObject({ + name: 'SDKError', + message: 'Stream ended before the final event; the transcript may be incomplete.', + }); + }); + }); + } + it('yields streamed events when stream is enabled', async () => { const { filePath, cleanup } = withTempAudio('streamed audio'); const response = sseResponse([