Skip to content

Accept a prompt query parameter on the live transcription route #618

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
scriptease wants to merge 1 commit into
base: main
Choose a base branch
from
+10 −2
Open

Accept a prompt query parameter on the live transcription route #618

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions app/server/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -480,6 +480,7 @@ Because the body carries audio rather than JSON, parameters are query parameters
| `channels` | `1` | interleaved channel count |
| `sample_format` | `s16le` | `s16le` or `f32le` |
| `language` | unset | passed through to the model |
| `prompt` | unset | URL-encoded recognition context (hotwords, spellings), same as the multipart `prompt` field |
| `busy_timeout_ms` | model policy | how long to wait for the model lock, as elsewhere; clamped by the configured ceiling, so a request can shorten its own wait but never weaken the guard |

```bash
Expand Down
9 changes: 7 additions & 2 deletions app/server/runtime.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3018,10 +3018,15 @@ HttpResponse ServerState::handle_transcription_live(const HttpRequest & request)
audio_contract.sample_rate = sample_rate;
audio_contract.channels = channels;
task_request.audio_input = std::move(audio_contract);
const std::string language = query_param(request.query, "language");
const std::string language = decoded_query_param(request.query, "language");
// Recognition-context biasing (hotwords), same meaning as the multipart
// route's `prompt` field; URL-encoded because it rides in the query.
const std::string prompt = decoded_query_param(request.query, "prompt");
if (!language.empty()) {
task_request.options["language"] = language;
task_request.text_input = engine::runtime::Transcript{std::string(), language};
}
if (!language.empty() || !prompt.empty()) {
task_request.text_input = engine::runtime::Transcript{prompt, language};
}
task_request = apply_default_request_options(model, std::move(task_request));
} catch (const std::runtime_error & ex) {
Expand Down
2 changes: 2 additions & 0 deletions docs/community_models/r2t2.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -247,6 +247,8 @@ ffmpeg -f avfoundation -i ":0" -ar 16000 -ac 1 -f s16le - \
'http://127.0.0.1:8488/v1/audio/transcriptions/live?model=r2t2-asr-stream&sample_rate=16000&channels=1&sample_format=s16le'
```

Add `&prompt=<url-encoded hotwords>` to bias the live stream the same way `--text` does on the CLI.

## GGUF checkpoints

GGUF is supported for Q8_0 and higher precision. `f16`, `q8_0`, and the native
Expand Down