Skip to content

Stream Temporal engine turn progress over /invoke instead of going silent until completion - #208

Merged
imaustink merged 1 commit into
mainfrom
feat/temporal-engine-streaming-progress
Aug 15, 2026
Merged

Stream Temporal engine turn progress over /invoke instead of going silent until completion#208
imaustink merged 1 commit into
mainfrom
feat/temporal-engine-streaming-progress

Conversation

@imaustink

Copy link
Copy Markdown
Owner

Summary

Reported live: a chat turn routed through the Temporal engine showed no text streaming and no SSE progress events — nothing until the whole turn completed.

TemporalEngine.stream() was the cause, and it was a documented, deliberate simplification, not a regression: it just awaited invoke() (a full accept/poll loop to completion) and yielded exactly one final chunk. Its own comment recorded the tradeoff — "a streaming client on this engine therefore sees the reply rather than the running commentary."

The engine's own Go gateway already solves this problem for its native /v1/chat/completions SSE endpoint: streamTurn polls workflows.TurnProgressQuery (a Temporal query exposing the workflow's own in-flight narration buffer) every 700ms and streams new lines as status events. agent-orchestrator never talks to that endpoint, though — Open WebUI's openaiBaseApiUrl points at agent-orchestrator, which always routes through the generic accept/poll /invoke contract via TemporalEngine.ts, and that contract had no equivalent capability.

Fix

Surfaces the same TurnProgressQuery data over /invoke too, rather than building a second mechanism:

  • GET /invoke/:id's existing "pending" response now includes a progress field — the same narration lines the native endpoint already streams from. Best-effort: a query failure or an inactive turn just means an empty list, never an error, since the pending status itself is still accurate.
  • TemporalEngine.poll() tracks how many lines it has already relayed and calls the turn's progressListener for each new one as it arrives — the exact callback agent-orchestrator's own SSE writer (handleChatCompletionsStreaming) already listens to for the in-process LangGraph engine, so no changes were needed on the consuming side at all.

Test plan

  • New unit tests: relays only new lines across repeated polls (no duplicates), and confirms nothing is called when the caller has no live channel (no progressListener set)
  • go build ./..., go vet ./..., gofmt -l ., go test ./... clean in engines/temporal
  • npx tsc --noEmit, full agent-orchestrator vitest suite clean (2 pre-existing, unrelated failures in nats-agent-channel*.test.ts — a different subsystem, confirmed unaffected by this change)
  • e2e (bridged-agent-workflow, happy-path, caller-tools) still passes against minikube after rebuilding both the gateway and orchestrator images

…ead of going silent until completion

TemporalEngine.stream() previously just awaited invoke() (a full
accept/poll loop to completion) and yielded exactly one final chunk --
its own doc comment recorded this as a deliberate simplification, not a
bug. In practice that meant a streaming chat turn on this engine showed
nothing at all until the whole turn finished: no SSE status events, no
incremental content, just silence followed by the answer.

The engine's own gateway already solves this for its native
/v1/chat/completions SSE endpoint: streamTurn polls
workflows.TurnProgressQuery (a Temporal query exposing the workflow's
own in-flight narration buffer) every 700ms and streams new lines as
`status` events. agent-orchestrator never talks to that endpoint
though -- it always goes through the generic accept/poll /invoke
contract via TemporalEngine.ts, which had no equivalent.

Fix: surface the same TurnProgressQuery data over /invoke too.
GET /invoke/:id's existing "pending" response now includes a
`progress` field (the same narration lines, best-effort -- a query
failure or an inactive turn just means an empty list, never an error,
since the pending status itself is still accurate). TemporalEngine.poll()
tracks how many lines it has already relayed and calls the turn's
progressListener for each new one as it arrives, exactly mirroring what
the engine's own native endpoint already does -- so a streaming chat
caller sees the same live status updates regardless of which engine is
driving the turn.

Verified: new unit tests cover relaying new-lines-only (no repeats) and
confirm nothing is called when the caller has no live channel (no
progressListener set). Full engines/temporal and agent-orchestrator
suites clean; e2e (bridged-agent-workflow, happy-path, caller-tools)
still passes.
@imaustink
imaustink merged commit 9d33dd1 into main Aug 15, 2026
6 checks passed
@imaustink
imaustink deleted the feat/temporal-engine-streaming-progress branch August 15, 2026 15:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant