-
Notifications
You must be signed in to change notification settings - Fork 0
feat(chat): hand run() a streamText with the managed options already applied #1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: qa/agent-triggerdotdev-trigger-dev/pr-01-4884/base
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,19 @@ | ||
| --- | ||
| "@trigger.dev/sdk": minor | ||
| --- | ||
|
|
||
| Actions can now become turns. `onAction` edits history with `chat.history`; to answer after the edit, return `chat.turn()` and a turn runs on the edited history with everything a turn has: the agent's system prompt and tools, steering, compaction, injected instructions, `onTurnStart` and `onTurnComplete`, and persistence. A regenerate is `chat.history.slice(0, -1); return chat.turn();`. | ||
|
|
||
| ```ts | ||
| onAction: async ({ action }) => { | ||
| if (action.type === "regenerate") { | ||
| chat.history.slice(0, -1); | ||
| return chat.turn(); | ||
| } | ||
| if (action.type === "undo") chat.history.slice(0, -2); // edit only | ||
| }, | ||
| ``` | ||
|
|
||
| Returning a `StreamTextResult`, `string` or `UIMessage` from `onAction` is no longer supported and now fails with an error pointing to `chat.turn()`. A response produced that way skipped every turn guarantee, and its delivery to the browser was unreliable: the frontend never read the stream `transport.sendAction` returned, so a regenerate that appeared to work on the server did not render. | ||
|
|
||
| History edits made by an action are still persisted as before: platform-managed snapshots are written after the edit, and apps with their own store mirror the edit themselves. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| --- | ||
| "@trigger.dev/sdk": patch | ||
| --- | ||
|
|
||
| Steering messages are now kept in the conversation when you drive turns yourself with `chat.createSession()` or `chat.MessageAccumulator`. Previously a message that arrived mid-answer shaped that answer and then existed nowhere: it was missing from `turn.uiMessages`, so an app persisting from there never stored it, missing from `turn.messages`, so every later turn answered as though it had never been sent, and it was not queued as its own turn either. It now lands in both, the same way it does on `chat.agent`. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| --- | ||
| "@trigger.dev/sdk": patch | ||
| --- | ||
|
|
||
| Injected system context is merged into a single instruction block, so it works on every supported AI SDK version. Note that a cached system prompt gives up its cache entry for as long as an injection is live, since the cached prefix has changed. | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| --- | ||
| "@trigger.dev/sdk": patch | ||
| --- | ||
|
|
||
| `chat.inject()` with `role: "system"` now works. It previously put the system message into the conversation, which AI SDK 7 rejects for every provider: the next turn died with a generic "An error occurred." and persisted an empty assistant message, so the agent looked like it had stopped answering. System-role context is now appended to the model's instructions, which is also the only way to inject context the agent treats as trusted. | ||
|
|
||
| Two things to know. Instructions are delivered by `chat.toStreamTextOptions()`, so a `run()` that calls `streamText` without spreading it does not receive a system-role injection. The conversational lane has no such requirement. And an injection applies to the next turn only, rather than repeating on every turn that follows it. Every inference call in that turn sees it, so a `run()` that builds options more than once gets the same instructions each time. An instruction injected after an action has run, and before the next message, reaches that next turn rather than the one after it. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,18 @@ | ||
| --- | ||
| "@trigger.dev/sdk": minor | ||
| --- | ||
|
|
||
| `run()` now receives a `streamText` with your agent's managed options already applied, so they cannot be lost by leaving out the spread: | ||
|
|
||
| ```ts | ||
| run: async ({ messages, signal, streamText }) => | ||
| streamText({ model, messages, abortSignal: signal }); | ||
| ``` | ||
|
|
||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Shipwright · HIGH The managed 'streamText' throws when 'system' is set in two places, but the diff only documents this in changesets and docs. Impact: The managed 'streamText' throws when 'system' is set in two places, but the diff only documents this in changesets and docs. There is no test in the visible diff that exercises the throw path, so a regression where the throw is skipped or the wrong value wins would ship silently. Suggested fix: Review the cited evidence, fix the risk if confirmed, and rerun Shipwright. |
||
| Spreading `chat.toStreamTextOptions()` still works and is equivalent. The difference is what happens when your options collide with the managed ones. Passing `tools` after the spread replaces the skill tools, and passing your own `prepareStep` replaces the managed one, which silently switches off steering, compaction and injected context. The managed `streamText` merges tools and composes `prepareStep` instead, so neither can be turned off by accident. | ||
|
|
||
| `system` can be set at the call site, on `chat.agent({ system })`, or through `chat.prompt.set()`, but only in one of them: setting it in two places throws, because no single shape merges two system values across every supported AI SDK version, and dropping one silently is the failure this seam exists to prevent. Injected instructions append to whichever one is in play. | ||
|
|
||
| `chat.agent()` also takes `registry`, `cacheControl` and `systemProviderOptions` now, so a managed prompt's model and its cache breakpoint no longer have to be passed at the call site. `chat.toStreamTextOptions()` applies them as well, so spreading it into the `streamText` imported from `ai` stays equivalent to the one `run()` receives. | ||
|
|
||
| `chat.headStart` and `chat.startHeadStart` hand their `run` the same thing, carrying the options the handover protocol depends on. There it matters more: re-setting `messages`, `prompt`, `stopWhen` or `abortSignal` after a spread breaks the handover rather than degrading a feature, and nothing caught it. On the managed one those four keys are a type error; `tools` is yours to pass. | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| --- | ||
| "@trigger.dev/sdk": patch | ||
| --- | ||
|
|
||
| Undo, edit and regenerate now survive a run ending. History rolled back from `onAction` was only kept in the running worker's memory, so the rollback held while that worker stayed warm and then reverted on the next continuation. The undone messages came back, minutes later, with no error. This also holds when the turn before the action failed: the rollback used to be written against the cursor from before that turn, so a continuation could replay output the failed turn had already superseded. | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Shipwright · HIGH The changeset for 'persist-action-history-mutations' claims rollback persistence now survives a run ending, but the diff contains no implementation code for this fix — only the cha Impact: The changeset for 'persist-action-history-mutations' claims rollback persistence now survives a run ending, but the diff contains no implementation code for this fix — only the changeset and tests. If the runtime change is in a separate package not shown in this diff, the release is incomplete; if it is supposed to be here, the fix is missing entirely. Suggested fix: Review the cited evidence, fix the risk if confirmed, and rerun Shipwright. |
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,9 @@ | ||
| --- | ||
| "@trigger.dev/sdk": patch | ||
| --- | ||
|
|
||
| Steering messages injected mid-answer are now part of the conversation, both for your hooks and for the model on later turns. Previously they reached the model for the answer they steered and reached the browser, but nothing else: `onTurnComplete` never saw them, so an app storing its own transcript lost the instruction the answer was shaped by, and it vanished from the conversation on reload. The model also forgot the instruction from the next turn onwards, answering as though the message had never been sent, while the chat UI still showed it. This holds when the steered turn fails part-way, and when `pendingMessages.prepare` reshapes the message: later turns now see the same form the steered turn did, not the original message. | ||
|
|
||
| Approving a tool call no longer undoes compaction. A tool-approval continuation used to rebuild the model's context from the full conversation, so a chat that had been summarised to fit the context window was sent the whole transcript again on the next call, and could go over the limit it had just been compacted to avoid. | ||
|
|
||
| If you worked around this by saving steering messages as they arrive, in `pendingMessages.onReceived` for example, that write now duplicates the one you get from `newUIMessages`. Drop it, or skip messages you have already stored. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,13 @@ | ||
| --- | ||
| "@trigger.dev/sdk": minor | ||
| --- | ||
|
|
||
| Actions are sent through `useChat` so a turn that follows one renders like any turn. `TriggerChatTransport` recognises `body.action` on a `useChat` request and sends it as an action, so `sendMessage(undefined, { body: { action } })` or `regenerate({ body: { action } })` sends the action and `useChat` owns the response: it streams into the message list, `status` and `error` behave as for a message, and `stop` works. `useChatActions({ sendMessage })` in `@trigger.dev/sdk/chat/react` is a two-line convenience over that. | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Shipwright · HIGH The 'useChatActions' convenience sends arbitrary 'action' objects through 'useChat' request bodies. Impact: The 'useChatActions' convenience sends arbitrary 'action' objects through 'useChat' request bodies. The changeset says the backend validates against 'actionSchema', but the diff does not show the validation path for the new 'body.action' transport route. If validation is bypassed or the schema is permissive, a client could inject action types the server did not intend to expose. Suggested fix: Review the cited evidence, fix the risk if confirmed, and rerun Shipwright. |
||
|
|
||
| ```tsx | ||
| const { sendMessage } = useChat({ id: chatId, transport }); | ||
| const { sendAction } = useChatActions({ sendMessage }); | ||
| sendAction({ type: "regenerate" }); | ||
| ``` | ||
|
|
||
| Previously the frontend docs said `useChat` consumed the stream `transport.sendAction` returns; it never did, so an action's answer was never rendered by an app following them. `transport.sendAction` still returns a stream that callers outside `useChat` must read, and now accepts `{ abortSignal, metadata }`, with per-action metadata merged over the transport's `clientData`. | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,7 +1,7 @@ | ||
| --- | ||
| title: "Actions" | ||
| sidebarTitle: "Actions" | ||
| description: "Custom commands sent from the frontend that mutate chat state without consuming a turn — undo, rollback, edit, regenerate." | ||
| description: "Custom commands sent from the frontend that mutate chat state without consuming a turn: undo, rollback, edit, regenerate." | ||
| --- | ||
|
|
||
| ## Overview | ||
|
|
@@ -44,55 +44,87 @@ export const myChat = chat.agent({ | |
| // returning void → side-effect-only, no model call | ||
| }, | ||
|
|
||
| run: async ({ messages, signal }) => { | ||
| run: async ({ messages, signal, streamText }) => { | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Shipwright · HIGH The docs repeatedly show 'run: async ({ messages, signal, streamText }) => streamText({...})' without explaining that 'streamText' here shadows the 'ai' import. Impact: The docs repeatedly show 'run: async ({ messages, signal, streamText }) => streamText({...})' without explaining that 'streamText' here shadows the 'ai' import. A reader who imports 'streamText' from 'ai' and also destructures it will be confused about which one is managed; the note exists in backend.mdx but is easy to miss in the other examples. Suggested fix: Review the cited evidence, fix the risk if confirmed, and rerun Shipwright. |
||
| return streamText({ model: anthropic("claude-sonnet-4-5"), messages, abortSignal: signal }); | ||
| }, | ||
| }); | ||
| ``` | ||
|
|
||
| **Lifecycle flow:** Wake → parse action against `actionSchema` → `hydrateMessages` (if set) → **`onAction`** → apply `chat.history` mutations → emit `trigger:turn-complete` → wait for next message. | ||
|
|
||
| ## Returning a model response from an action | ||
| When `onAction` returns `chat.turn()`, the flow continues instead of emitting `trigger:turn-complete`: the edit is snapshotted, then a turn runs on the edited history with `trigger: "action-turn"`, so `onTurnStart`, `run()`, `onBeforeTurnComplete` and `onTurnComplete` all fire and the answer is persisted like any turn's. See [Answering after an action](#answering-after-an-action). | ||
|
|
||
| `onAction` can return a `StreamTextResult`, `string`, or `UIMessage` to produce a response. The returned stream is auto-piped to the frontend just like a normal turn, but the rest of the turn machinery (`onTurnStart`, `onTurnComplete`, etc.) still does not fire. | ||
| ## Answering after an action | ||
|
|
||
| An action is a state edit. To answer after the edit, return `chat.turn()`: the edit is applied and snapshotted, then a turn runs on the edited history exactly as a message turn does. `onTurnStart`, `run()`, `onBeforeTurnComplete` and `onTurnComplete` fire, the turn counter advances, and the answer gets everything a turn has: the agent's system prompt and tools, steering, compaction, injected instructions and persistence. | ||
|
|
||
| ```ts | ||
| onAction: async ({ action, messages }) => { | ||
| if (action.type === "regenerate") { | ||
| chat.history.slice(0, -1); // drop the last assistant | ||
| return streamText({ | ||
| model: anthropic("claude-sonnet-4-5"), | ||
| messages, | ||
| stopWhen: stepCountIs(15), | ||
| }); | ||
| onAction: async ({ action }) => { | ||
| switch (action.type) { | ||
| case "undo": | ||
| chat.history.slice(0, -2); | ||
| return; // edit only, no turn | ||
|
|
||
| case "regenerate": | ||
| chat.history.slice(0, -1); | ||
| return chat.turn(); // answer the edited history | ||
|
|
||
| case "retry-formal": | ||
| chat.history.slice(0, -1); | ||
| chat.inject([{ role: "system", content: "Answer formally this time." }]); | ||
| return chat.turn(); // with a one-shot instruction | ||
| } | ||
| // other actions return void → side-effect only | ||
| } | ||
| ``` | ||
|
|
||
| This is useful for actions that both mutate state and want a fresh model response (regenerate-from-here, retry-with-different-style). Persistence is your responsibility inside `onAction` itself; you have access to the streamed response object. | ||
| `run()` receives the edited history with no incoming user message, the same shape as a `regenerate-message` turn, and its `trigger` is `"action-turn"`, so a `run()` that returns early on `"action"` (the pre-May behaviour, when actions invoked `run()` directly) still answers. Returning anything other than `chat.turn()` or nothing is an error; a response can no longer be returned from `onAction` directly. | ||
|
|
||
| ### Actions and persistence | ||
|
|
||
| An action that returns nothing does not fire `onTurnComplete`, and that is where an app that owns its own transcript normally writes. What that means depends on which persistence model you use. | ||
|
|
||
| **Platform-managed** (no `hydrateMessages`): nothing to do. After an action that changed the conversation, the runtime writes the snapshot, so the edit survives the run ending. An action that returns `chat.turn()` is followed by a turn, which persists its answer the way every turn does. | ||
|
|
||
| **Your own store** (`hydrateMessages` registered): the runtime deliberately does not write, because your store is the source of truth. A history edit lives only in the running worker until you persist it, and a continuation rehydrates from your store, not from what the worker had in memory. Mirror each edit in your store, not only additions: a regenerate is a delete *and* an insert. The answer that follows `chat.turn()` reaches your store through `onTurnComplete`, like any turn's answer. | ||
|
|
||
| ```ts | ||
| onAction: async ({ action, chatId }) => { | ||
| if (action.type === "undo") { | ||
| chat.history.slice(0, -2); | ||
| await db.deleteLastExchange(chatId); // the rollback is yours to persist | ||
| } | ||
| if (action.type === "regenerate") { | ||
| chat.history.slice(0, -1); | ||
| await db.deleteLastAssistant(chatId); // the delete half | ||
| return chat.turn(); // the insert half arrives in onTurnComplete | ||
| } | ||
| }, | ||
| onTurnComplete: async ({ chatId, newUIMessages }) => { | ||
| await db.saveMessages(chatId, newUIMessages); | ||
| }, | ||
| ``` | ||
|
|
||
| ## Gating actions on HITL state | ||
|
|
||
| If you have a [human-in-the-loop](/ai-chat/patterns/human-in-the-loop) tool waiting on `addToolOutput`, you usually want to refuse competing actions like `regenerate` until the answer arrives. [`chat.history.getPendingToolCalls()`](/ai-chat/backend#chat-history) gives you exactly that signal: | ||
|
|
||
| ```ts | ||
| onAction: async ({ action, messages, signal }) => { | ||
| onAction: async ({ action }) => { | ||
| if (action.type === "regenerate") { | ||
| if (chat.history.getPendingToolCalls().length > 0) return; // gated | ||
| chat.history.slice(0, -1); | ||
| return streamText({ model: anthropic("claude-sonnet-4-5"), messages, abortSignal: signal }); | ||
| return chat.turn(); | ||
| } | ||
| }, | ||
| ``` | ||
|
|
||
| ## Sending actions from the frontend | ||
|
|
||
| ```ts | ||
| // Browser — TriggerChatTransport | ||
| // Browser: TriggerChatTransport | ||
| const stream = await transport.sendAction(chatId, { type: "undo" }); | ||
|
|
||
| // Server — AgentChat | ||
| // Server: AgentChat | ||
| const stream = await agentChat.sendAction({ type: "rollback", targetMessageId: "msg-3" }); | ||
| ``` | ||
|
|
||
|
|
@@ -104,8 +136,8 @@ The action payload is validated against `actionSchema` on the backend; invalid a | |
|
|
||
| ## See also | ||
|
|
||
| - [`chat.history`](/ai-chat/backend#chat-history) — the imperative API actions use to mutate state | ||
| - [Sending actions from the frontend](/ai-chat/frontend#sending-actions) — `transport.sendAction` ergonomics | ||
| - [`hydrateMessages`](/ai-chat/lifecycle-hooks#hydratemessages) — fires before `onAction` when set | ||
| - [Branching conversations](/ai-chat/patterns/branching-conversations) — pairs action handlers with backend-controlled history | ||
| - [Human-in-the-loop](/ai-chat/patterns/human-in-the-loop) — gating fresh actions while a tool is waiting | ||
| - [`chat.history`](/ai-chat/backend#chat-history): the imperative API actions use to mutate state | ||
| - [Sending actions from the frontend](/ai-chat/frontend#sending-actions): sending actions through `useChat` so a turn that follows one renders like any turn | ||
| - [`hydrateMessages`](/ai-chat/lifecycle-hooks#hydratemessages): fires before `onAction` when set | ||
| - [Branching conversations](/ai-chat/patterns/branching-conversations): pairs action handlers with backend-controlled history | ||
| - [Human-in-the-loop](/ai-chat/patterns/human-in-the-loop): gating fresh actions while a tool is waiting | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Shipwright · LOW
The changeset for 'inject-instructions-shape' notes that a cached system prompt gives up its cache entry while an injection is live, but does not quantify the cost or suggest a mit
Impact: The changeset for 'inject-instructions-shape' notes that a cached system prompt gives up its cache entry while an injection is live, but does not quantify the cost or suggest a mitigation. A maintainer tuning prompt caching later will not know whether this is a minor or major performance regression.
Suggested fix: Review the cited evidence, fix the risk if confirmed, and rerun Shipwright.