Skip to content

fix(runtime): stop estimating context fit; the provider decides, the Maka window is a user target - #4574

Open
Joob1n wants to merge 6 commits into
apache:mainfrom
Joob1n:fix/context-provider-decides
Open

fix(runtime): stop estimating context fit; the provider decides, the Maka window is a user target#4574
Joob1n wants to merge 6 commits into
apache:mainfrom
Joob1n:fix/context-provider-decides

Conversation

@Joob1n

@Joob1n Joob1n commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Summary

The runtime no longer estimates whether a request fits a context window. Every "does it fit" question is answered by a provider; every trigger is a real number. This is the remaining step after #4486, which removed the fabricated capacity and the two terminal gates but kept the chars/4 delta estimate, the 32,000-token fallback history budget, and a per-step JSON.stringify of the whole request as a payload ruler. Design and discussion: #4559 (this PR), #4458 (history).

What decides now

  • The conversation provider's own context-length rejection is recovered by one compact-and-retry; a second rejection surfaces as the provider's error (context_overflow), and the user's message stays editable.
  • The summarizer's provider decides whether compaction input fits: input_too_large retreats the fold by half, and the loop exits through no_safe_completed_span when even the smallest legal span is refused.
  • Proactive compaction has one trigger: the previous accepted request's real inputTokens + outputTokens, as the provider counted them, compared with the context window the user declared for the model (a model-facts pin or a relay profile). Reasoning tokens are counted whole on purpose — high is the safe direction for a trigger that can only ask for a compaction.
  • A reply the provider cut at its output limit (finishReason: length) folds once before the next request.

What is gone: estimateNextRequestTokens, exceedsHighWater, midTurnRequestPayloadChars, the final-request rescue re-entry, maxHistoryEstimatedTokens and its five consumers, the quarter-window / 16,384 reserve, the 32,000 fallback, the replacement_not_smaller and prefix_over_budget replay gates, size-based replay selection, the estimated input fit before summarizer calls, and the chars/4 summary-size floor.

Notes for the user (two new system_note kinds): context_provider_dropping when an append-only step's usage did not grow (a provider evicting or rewriting context), and context_window_suggestion when a rejection lands at a proven-fit total, carrying the number the user can declare.

Closes #4559
Refs #4458, #4486

Follow-up PRs (stacked on this one)

The user-facing surfaces are deliberately split out so this PR stays the atomic runtime change and the UI pieces can be reviewed in parallel. Both are open as drafts on top of this branch and will be rebased and undrafted once this merges:

Not planned in this series (out of scope, tracked on #4559 / #4283): a "compact and retry" action for provider rejections the error classifier does not recognise; the active-turn prune's blindness to image Tool Results.

Behaviour changes to review

  1. No declared window means no proactive compaction. The first time such a session fills up costs one provider round trip and possibly one cut reply; the context_window_suggestion note then offers the last accepted total as the value to declare. This is the "Maka does not decide for the user" principle; the trade-off is stated in Context budget: stop estimating fit; the provider decides, the Maka window is a user target #4559.
  2. A /models report or models.dev metadata is no longer a threshold on its own. resolveDeclaredContextWindow (runtime) and declaredContextWindow (core, single owner of the rule) count only a facts pin or a relay profile. resolveSelectedModelContextWindow keeps its old meaning for display and contextRemaining.
  3. token_usage.lastRequestAnchor is now { inputTokens, outputTokens? }. The retired payloadChars key still decodes, so sessions written by 0.2.0 keep loading; sessions written by this build do not open in 0.2.0 (the closed-allowlist cost already noted in CHANGELOG.md).
  4. Summaries are capped at 8,000 output tokens (DEFAULT_HISTORY_COMPACT_MAX_OUTPUT_TOKENS; the summarizer's maxOutputTokens option was declared and never passed). A length finish gets one shorter retry, then fails open. The too-small-for-fold floor reads the summarizer call's real usage; a producer that reports none is not floored.
  5. Checkpoint replay is structural: a source-matched checkpoint replays as [block, everything after the boundary]. No local size gate stands between a materializable fold and dispatch.
  6. ContextBudgetDiagnostic.maxHistoryEstimatedTokens is retired (decode-only). HistoryCompactionPolicy.midTurn loses reserveTokens. buildDefaultContextBudgetPolicy drops its connection parameter.
  7. Every OpenAI-compatible Chat request now asks for stream usage (stream_options.include_usage), unless the registry opts a provider out. Found live: only two providers opted in, so a local Ollama and every plain OpenAI-compatible relay — the connection type behind Context budget: an undeclared window fabricates a capacity that kills live turns #4458 — returned no usage, and the baseline, the eviction check and the indicator all stayed dark for them.
  8. Summary requests close with a user instruction. Found live: a chat-template model handed a conversation ending on its own turn emits end-of-sequence and nothing else (Ollama qwen2.5, both sizes: finish stop, one output token, empty text), so history compaction never succeeded against a local model. Existing message-shape tests gained the trailing user:text.
  9. The trigger reserves the model's declared output limit and fires on >= (review). baseline + outputLimit >= declaredWindow, where the limit is connection.models[].maxOutputTokens or generated metadata — a provider fact — and 0 when none is declared. Without it a window declared at the provider's real size could never be crossed by an accepted request. The window itself remains the user's declaration (see the reply on the review for why the metadata window stays a hint).
  10. A context-length rejection from the summarizer's provider is input_too_large again (review), so the planner's retreat-by-half has a producer; the same classification is applied to the Codex compaction call.
  11. The window suggestion note is written only when the turn surfaces the rejection (review): after the one fold is spent or when there is no seam, naming the last accepted total, which a fold does not clear.
  12. The size floor judges an initial fold only (review): on a roll-forward the summarizer's input is the increment, not the covered span, so the floor stands down there and the fold context carries usage only. The eviction check compares input against input, since reasoning is not resent on every wire.

On dropping the signed delta

#4486 kept a signed chars/4 delta so step 0 could fold before a large new message or image went out. I dropped it: the delta prices an image at a constant and under-reads CJK by about half, so the round trip it saves is the one it is most likely to misjudge; large text results are already capped by the byte-based tool-result archive before they go out; and removing it takes the whole payload ruler and the per-step JSON.stringify(messages) with it. Happy to be argued back.

Test coverage removed

Thirty-seven tests encoded the estimate, the reserve, the payload ruler, the fit gates, or the pre-turn estimate gate; each is either re-pointed at a real-usage observable or removed with the mechanism. Re-pointed: the usage baseline is the last request's INPUT tokensinput plus output; compacts over the high watercompacts after provider usage crosses the declared window; keeps user_stop when stopping an oversized pre-turn summarykeeps user_stop when stopping while a usage-triggered fold is summarizing; persists the LAST request as the anchorpersists the last request input and output as the anchor; replays a durable pre_turn checkpoint below the current high waterwithout a local size gate; rejects a paragraph-sized summary for a large folded spanwhen usage says it is too small. Removed outright (the mechanism is gone): the three manual compactHistory … exceeds current limits / makes the full replay larger cases, the three reserve-derivation cases, anchors on real provider usage plus a tail char/4 delta, credits a SIGNED negative payload delta, falls back to whole-projection char/4 on cold start, high-water crosses at contextWindow minus reserve, a fold that cannot shrink the real payload is refused, a runaway summary is rejected as replacement_not_smaller, bounds the oldest oversized tool result before dispatch, fails with input_too_large before dispatch, charges the summarization instructions against the input budget, a summary at exactly the floor is accepted under ceil-based token estimates, the capacity verdict measures the steering payload, the anchor a finalization step writes excludes the tool schemas it cleared, dispatches a pre-turn Compaction recipe without projecting history and keeps a transiently unavailable automatic Compaction checkpoint recoverable (both exercised the deleted pre-turn gate; memory extraction from a hook-triggered fold stays covered), applies max-history overrides to checkpoint replay validation, accepts a complete checkpoint above legacy block limits when the full replay fits, compacts one oversized prior turn before an unknown-model request, compacts an oversized latest turn when an older turn is also retained, dispatches an oversized prior turn when its summary fails, after trying once, never runs a pointless summarizer on a small-window model, a usage object without usable input tokens falls back to cold start, a last-request anchor is only valid as a complete positive pair, a half-written anchor fails the whole token_usage message decode (replaced by the new-shape and retired-key cases).

New coverage: the declared-window rule for pin / relay / reported / metadata sources; baseline = input + output, > not >=, cleared after a fold, seeded from the previous turn, discarded across a model switch; a /models window is not a declaration; pendingLengthFold; the summarizer output cap, shorten retry, and output_length after two cuts; usage-based size floor; retreat by half and its give-up exit; structural replay; both notes, including the non-firing cases (pruned step, step 0, declaration already crossed); the retired payloadChars key decodes and an unknown key still fails.

Live verification against a real provider (Ollama)

Driven through the real AiSdkBackend with getAIModel against a local Ollama (qwen2.5:0.5b / qwen2.5:1.5b, default num_ctx 4096), five turns of ~740 tokens each, the previous turn's ledger and run header fed back as prior context.

Scenario Observed
Declared window 1,500, qwen2.5:1.5b Turns 1–3: anchors {745,10}, {1481,10}, {2217,10} persisted; baseline 1,491 at turn 3 is < 1,500, no fold. Turn 4: baseline 2,227 > 1,500 → pre_turn fold; summary malformed twice → failedOpen[malformed_summary_missing_section], request dispatched, end_turn. Turn 5: fold again, repair produces a 1,737-char checkpoint → replaced; the request's provider-counted input drops 2,954 → 1,222.
Same, before the two fixes above No token_usage event at all (Ollama returned no usage because it was never asked); once usage flowed, the summarizer returned empty text every time. Both fixed in this PR.
No declared window, 8 turns Never folds, never terminates; every turn end_turn. Usage grows 745 → 3,700 and then plateaus at 3,716 for turns 6–8 while ~740 tokens are appended per turn: Ollama truncates silently at num_ctx with no error.

Known limitation, stated plainly: the silent-truncation plateau above happens between turns, and context_provider_dropping only fires within a turn (step ≥ 1), because at step 0 a user edit can shrink the input legitimately. A provider that truncates instead of rejecting therefore gives an undeclared-window user no signal today; declaring the window is the protection. Detecting the cross-turn plateau without an estimate needs a ledger-shape check on the anchor (did anything before the anchored request change?) — a follow-up, tracked on #4559.

Verification

Rebased on main@8ea3c4f05 (after #4578 landed, so this PR no longer carries the dependency bump); the Runtime Host compatibility epoch moves 98 → 99 for the two new note kinds and the reshaped anchor. Locally: npm --workspace @maka/{core,storage,runtime,runtime-host,mcp,ui} run build, npm run typecheck (all workspaces), npm run lint, npm run format:check, npm run check:asf-headers, node scripts/protocol-epoch-check.mjs --base origin/main — all clean. Targeted suites: 425/425 across usage-record-last-request-anchor, context-budget-mid-turn-policy, context-budget-model-facts, history-compaction, history-compact-checkpoint, history-compact-summarizer, mid-turn-capacity-backend, overflow-reactive-recovery, ai-sdk-backend; core suite 760/760; provider-conformance 24/24, history-compact-summarizer 53/53, execution-model-composition 28/28 and the CLI pi-transcript suites after the review fixes. Not run locally: the full npm test matrix (the Electron/OpenSSL and owned-Host timing cases fail on this machine independently of the branch) — relying on CI for those.

Self-review

  • The core rule declaredContextWindow was implemented twice (runtime and chat-model-choice) in an earlier draft; consolidated into @maka/core/model-thinking with runtime delegating. One owner for the one rule the design turns on.
  • The user-stop-during-summary obligation had lost its test when the pre-turn gate went; re-pointed at the hook-triggered fold. The test also records that one request is still attempted after the stop and is rejected by the transport on its already-aborted signal — pre-existing behaviour, now pinned.
  • An earlier draft regenerated the Astryx theme with a stale local @astryxdesign/core@0.5.0; that artifact is not in this PR.
  • memory-extraction.ts and session-recap.ts keep their own bounded-request budgets; they are auxiliary calls, not history compaction, and are out of scope here.
  • Diagnostics (estimatedTokensBefore/After on compaction decisions) still use the chars/4 estimator. They are recorded numbers, not decisions, and are kept so existing telemetry consumers do not break.

AI use

Select exactly one:

  • No generative tool made a substantive contribution
  • Generative tooling made a substantive contribution

Tool(s) and scope: Claude Code — design discussion, implementation, tests, and review; verified by the author.

Checklist

  • Tests cover the change and fail without it
  • Lint, format, typecheck and the affected suites pass locally

Does this PR entail a change in behavior?

  • Yes — described under Summary above
  • No

https://claude.ai/code/session_014ajaRxC4jydavY9nYUFj5J

@Astro-Han Astro-Han left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The direction is the one from #4458 and I still hold it: fit is the provider's answer, the trigger is a real number, and the only irreversible outcome is the provider's rejection. The reduction is real, every symbol in "What is gone" is gone from docs and desktop too, and all six workspace suites, typecheck, lint and format pass locally on 372c1de9, and the three commits since (148326eb) touch none of the files cited below. What stops me approving is not the code, it is three places where the design as implemented does not do what the design says. Line-level fixes are inline.

The proactive trigger cannot fire against a correctly declared window. The trigger is baseline > declaredWindow, and the baseline is a request the provider accepted. If the user declares the provider's real window, no accepted request can exceed it, the fold never runs, and every session goes through rejection first. It fires only when the user declares something smaller, which is the retired reserve reintroduced as a number the user has to guess without being told. This is open point 2 of #4559, and I would settle it the other way: resolve the window as declared > relay profile > models.dev > /models, none only when all are absent; trigger on baseline > window − reserve; make the reserve a real number, the model's output limit plus the tool-result archive cap times pending tool calls, with a fraction of the window as the floor. Those are provider facts and Maka's own content policy, so the "real numbers only" principle holds. A default is not Maka deciding for the user, it is a default the user can override, and the alternative is that out of the box every fill costs a rejection and possibly a cut reply, which CLI users get without even the hint, because pi-transcript.ts drops the two new note kinds. The per-step check already gives mid-turn triggering; only the window and the threshold change.

The summarizer's provider does not get to decide. The design names four deciders, and one of them, "the summarizer's provider decides whether compaction input fits, input_too_large retreats the fold by half", has no producer. Only fitHistoryCompactMessages throws input_too_large, this PR removes that call from history-compact-summarizer.ts and openai-codex-history-compactor.ts, and both catches wrap every provider error as provider_error. So a session whose fold prefix is too large for the summarizer fails open on the first rejection, /compact included, and has no in-app way back under the window. That is worse than main, which at least estimated. The fix is small, classify ContextLength in both catches, but the regression must use a fake provider that rejects above N tokens; all five current input_too_large tests throw the error from a fixture, which is why this was invisible.

The compatibility contract. Two new system_note kinds and the new lastRequestAnchor shape land in packages/core closed allowlists, the same class of change #4486 bumped the epoch for, and the epoch is unchanged. A merge-base client handshakes fine (host-kernel.ts admits on strict equality) and fails decodeStoredMessage on the first transcript carrying a new note. main is on 96 now: rebase, take 97, and put back the three CHANGELOG sentences from #4486 that are still true (see inline).

One consequence of the first point that lives outside the diff: settings-provider-copy.ts still says "when empty, built-in metadata decides", and /context and the inspector still divide by resolveSelectedModelContextWindow, so an undeclared user sees 70% of 200k and never gets a fold. Whichever way the window question settles, the copy and the denominator have to follow the same rule.

On the test claim: five re-pointed tests pass unchanged on main's built sources (the no-size-gate replay, the two size-floor cases, the no-usage case, fails open after repeated input-too-large retreat), and two rules lost their only coverage with the deletions while still existing, the memory gate's unavailable outcome and the rule that a synthetic /compact usage row must not shadow the real anchor. The remaining persisted-anchor test uses an anchor below the window, so it passes regardless of the run-header check.

Evidence boundary: static read against main 61224f51; all suites run locally; new tests also run against main's built sources in a scratch copy; no live provider.

AI-assisted review: drafted with Maka; I verified the trigger arithmetic, the input_too_large producers, the epoch state and the size-floor inputs myself.

简体中文

方向就是 #4458 上讨论定下的,我仍然支持:放不放得下由 provider 回答,压缩触发用真实数字,唯一不可逆的结果是 provider 的拒绝。删得也干净,「What is gone」里的每个符号在 docs 和 desktop 里都没有残留,本地六个 workspace 的测试、typecheck、lint、format 全绿;之后的三个提交(148326eb)没有碰到下面引用的任何文件。让我暂时不 approve 的不是代码本身,而是三处实现和设计说的不是一回事。能局部修的都放在行内评论里。

主动触发对着正确声明的窗口永远打不响。 触发条件是「上次请求的 input+output 大于声明窗口」,而上次请求是 provider 已经接受的。如果用户把窗口声明成 provider 的真实大小,任何被接受的请求都不可能超过它,主动压缩一次也不会触发,每个会话都要先撞一次拒绝。只有用户把窗口声明得比真实值小它才触发,这等于把删掉的 reserve 变成了一个要用户自己猜、又不告诉他的数字。这是 #4559 的 open point 2,我的意见相反:窗口按「用户声明 > relay profile > models.dev 元数据 > /models 报告」解析,全都没有才算无;触发条件改成「上次用量 > 窗口 − reserve」;reserve 用真实数字:模型的 output limit(下一步回复最多这么大)加上工具结果归档上限乘以待处理的工具调用数,再用窗口的固定比例兜底。这些都是 provider 的事实和 Maka 自己的内容策略,「只用真实数字」的原则不受影响。给一个默认值不是替用户决定,用户随时可以覆盖;不给的后果是开箱用户每次填满都要付一次拒绝、可能还拿到被截断的回复,CLI 用户连提示都看不到,因为 pi-transcript.ts 把两个新 note kind 丢掉了。turn 内每一步都检查的逻辑已经有了,只需要改窗口和阈值。

summarizer 的 provider 并没有拿到决定权。 设计列了四个决定者,其中「summarizer 的 provider 决定压缩输入放不放得下,input_too_large 就把范围退半」这一条没有产生者。全仓只有 fitHistoryCompactMessages 会抛 input_too_large,本 PR 把它从 history-compact-summarizer.tsopenai-codex-history-compactor.ts 里删掉了,两处的 catch 又把所有 provider 错误一律包成 provider_error。于是一个长会话的折叠前缀如果对 summarizer 也太大,第一次拒绝就整体放弃,/compact 走同一条路也一样,会话在 app 内没有回到窗口内的办法,比 main 还差,main 至少还估算一下。修法很小,两个 catch 里判断 ContextLength 再抛,但回归测试必须用「输入超过 N tokens 就拒绝」的假 provider;现在五个相关测试全是 fixture 手抛这个错误,所以这个洞看不见。

兼容契约。 两个新的 system_note kind 和新的 lastRequestAnchor 形状都进了 packages/core 的闭合 allowlist,和 #4486 推 epoch 的那次是同一类改动,但 epoch 没动。merge-base 上的旧 client 握手能通过(host-kernel.ts 按严格相等准入),然后在第一条带新 note 的 transcript 上解码失败。main 现在已经是 96:rebase 取 97,并把 #4486 写的、仍然成立的三句 CHANGELOG 加回来(见行内)。

第一点在 diff 之外的一个后果:settings-provider-copy.ts 的帮助文案「留空跟随内置元数据」现在不成立,/context 和 inspector 的分母也还是 resolveSelectedModelContextWindow,未声明的用户会看到「已用 70% / 200k」却永远等不到压缩。窗口问题无论怎么定,文案和分母都要跟同一条规则。

关于测试的说法:五条 re-pointed 测试在 main 的构建产物上原样通过(无 size gate 的 replay、两条体量下限、无 usage、退半后 fail open);两条规则随删测试失去了唯一覆盖但仍然存在:memory gate 返回 unavailable 的处理,以及手动 /compact 写的合成 usage 行不得遮蔽真实 anchor。剩下那条持久 anchor 测试的 anchor 低于窗口,不管 run header 合不合法都过。

// window, or the provider cut the previous reply at its output limit.
const lengthFold = state.pendingLengthFold;
state.pendingLengthFold = false;
const overWindow =

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

baselineTokens is a request the provider accepted, so with the window declared at the provider's real size this is never true; the fold only runs when the user declares something smaller. That is the reserve in disguise. baseline > window − reserve, with the reserve from output limit plus the tool-result cap, is what makes this fire before the rejection. First point in the main comment.

@@ -335,7 +281,12 @@ export async function planHistoryCompaction(
} catch (error) {
if (error instanceof HistoryCompactSummarizerError) {
if (error.reason === 'input_too_large') {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dead branch: nothing throws input_too_large any more. fitHistoryCompactMessages was its only producer and both summarizer catches now wrap provider errors as provider_error. Classify ContextLength with classifyError in history-compact-summarizer.ts and openai-codex-history-compactor.ts so this retreat has a producer, and test it with a fake provider that rejects above N tokens.

Comment thread packages/runtime/src/ai-sdk-backend.ts Outdated
stepUsage !== undefined &&
Number.isFinite(stepUsage.inputTokens) &&
stepUsage.inputTokens > 0 &&
stepUsage.inputTokens < midTurnState.baselineTokens

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Baseline includes last step's reasoning; on wires that do not resend it (gpt-5.x), next input ≈ previous input + tool call + result, which is below input + output whenever reasoning exceeded the appended content, the normal case. Compare against last step's input alone; keep input + output for the trigger.

Comment thread packages/runtime/src/ai-sdk-backend.ts Outdated
id: this.newId(),
turnId,
ts: this.now(),
kind: 'context_window_suggestion',

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Written before recoverFromOverflowError runs, so a send that overflows once, folds and completes still tells the user the provider refused. #4559 places this under "already folded". Move it after the recovery verdict; overflow-reactive-recovery.test.ts:1834 (tool, overflow, done) currently pins the wrong order. And suggestedContextWindow: baselineTokens sits on the strict > boundary, so a user who applies it still needs one more rejection.

) {
return 'malformed_summary_too_small_for_fold';
}
const usage = foldContext?.summarizerUsage;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On a roll-forward the summarizer is sent previous summary + new events, so inputTokens is the increment while the checkpoint replaces the whole covered span; in steady state this floor never trips and a fifty-token summary after a small increment is persisted. The new test passes on main because the stub's usage is unrelated to the messages. Feed it a span-sized real number (previous checkpoint's compactCallInputTokens plus this increment) or drop the floor openly.

@@ -225,7 +225,6 @@ export function buildHistoryCompactCheckpoint(
if (!providerState && input.summaryFormat !== 'legacy_freeform') {
const defect = findCheckpointSummaryDefect(summary!, {
coveredRuntimeEvents: input.coveredRuntimeEvents,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comment above says this re-runs the size floor, but no summarizerUsage is passed so the floor is skipped, and coveredRuntimeEvents is now required and unread by the validator. Same for the external-producer branch in history-compaction.ts. Either pass a real number or delete the field, the three constructions and the comment.

Comment thread CHANGELOG.md Outdated
and SessionEvent-to-RuntimeEvent conversion remains a pure mapper.
- Retired the Task Ledger domain: SessionTodo is now the sole authority for in-session work items, and the operational-state schema drops the `workflow_task_ledger_events` table on first open. **Unfinished Tasks are not migrated and are permanently deleted.** This affects workspaces last opened by `v0.1.0` through `v0.1.11`, `cli-v0.1.0-beta.1`, `v0.2.0-incubating-rc1`, or a `v0.2.0-dev` build; those releases wrote Tasks to a table that no shipped build ever bridged into SessionTodo. Before opening such a workspace with this build, finish or export the Tasks you still need, or copy the workspace's `runtime.sqlite` aside — the migration removes the only live copy, so afterwards recovery requires a backup made in advance.
- Let the provider decide whether a request fits, and anchored the estimate that decides when to compact on the last request the provider actually counted. `token_usage` records now persist that anchor under a new `lastRequestAnchor` key. **Sessions this build writes do not open in earlier releases:** those decode `token_usage` against a closed allowlist, so the unknown key fails the record and, with it, the Session that contains it. Downgrading therefore needs a copy of the workspace's `runtime.sqlite` taken before the upgrade. Retired with the local verdict: nothing produces the `context_budget_exhausted` stop reason any more — a request that really is too large is compacted and retried once, then reported as a `context_overflow` provider error — though sessions that already recorded it still decode and present. The Runtime Host compatibility epoch moves to 94.
- Let the provider decide whether a request fits. Proactive compaction now uses only a user-declared Maka window and the previous accepted request's provider-reported `inputTokens + outputTokens`; no declaration means no proactive capacity threshold. `/models` and generated model metadata are display hints, not limits. `token_usage` records persist the last-request anchor under `lastRequestAnchor`; its new `{ inputTokens, outputTokens }` shape still decodes the retired `payloadChars` key from older sessions. Requests that are too large are compacted and retried once after a real provider rejection, then reported as a `context_overflow` provider error. New provider-dropping and context-window suggestion system notes explain provider-side context changes.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Three sentences from #4486 were dropped and are still true: sessions from this build do not open in earlier releases and the runtime.sqlite copy must be taken before upgrading; context_budget_exhausted is no longer produced but still decodes; the epoch line (97 after rebase). The body says the allowlist cost is "already noted in CHANGELOG.md", and it no longer is.

@Joob1n
Joob1n force-pushed the fix/context-provider-decides branch from 148326e to a650762 Compare September 2, 2026 17:15
Joob1n added a commit to Joob1n/maka-agent that referenced this pull request Sep 2, 2026
Review findings from apache#4574, each with a test that fails on the previous
commit:

- The proactive trigger now reserves the model's declared output limit
  (`connection.models[].maxOutputTokens` or generated metadata, a provider
  fact; 0 when none is declared) and fires on `>=`: with the window
  declared at the provider's real size, an accepted request alone can never
  exceed it, and the reply the next request must leave room for is what
  tips it. The window itself stays the user's declaration.
- The summarizer's provider is a decider again: a context-length rejection
  from the summary call (or the Codex compaction call) surfaces as
  `input_too_large`, the reason the planner retreats on. Removing the
  estimated pre-fit had left that path without a producer.
- The eviction check compares input against input: the previous reply's
  reasoning is not resent on every wire, so input + output is not a floor.
- The window suggestion is written after recovery, only when the turn
  surfaces the rejection, and names the last accepted total, which a fold
  does not clear. It is no longer a number on the `>` boundary.
- The size floor judges an initial fold only; on a roll-forward the
  summarizer's input is the increment, not the covered span.
- `CheckpointSummaryFoldContext` carries usage only; the builder and copy
  seams validate structure and truncation, which is all they can.
- The CLI transcript renders the two new note kinds.
- Restored coverage: a synthetic /compact usage row does not shadow the
  real anchor; the run-header check is exercised with an anchor above the
  window.
- CHANGELOG carries the downgrade and epoch sentences again.

The Host's reported `contextWindow` stays the model's window: it is a
protocol fact clients already display, and the declared window has its own
surface in the composer indicator.

Refs apache#4559

Generated-by: Claude Code
Claude-Session: https://claude.ai/code/session_014ajaRxC4jydavY9nYUFj5J
@Joob1n

Joob1n commented Sep 2, 2026

Copy link
Copy Markdown
Contributor Author

Thanks for the read — every finding was real. Fixes are in 38b2d837 (one commit, each item with a test that fails on the previous commit), plus two live-provider findings in the commits before it.

Trigger arithmetic. You are right that baseline > declaredWindow could never fire against a window declared at the provider's real size. Fixed the way you suggested for the reserve, with one narrowing: the reserve is the model's declared output limit only (connection.models[].maxOutputTokens or generated metadata; 0 when none is declared), and the trigger is baseline + limit >= window. I did not add the tool-result cap × pending calls term: the active prune already bounds each result before it goes out, and a second constant on the trigger side is the kind of number the design is trying not to own. Live on Ollama (qwen2.5:1.5b, declared 1,500) the fold now fires at 2,227 with the previous usage as the only input, folds to a 1,737-char checkpoint after one repair, and the next request's provider-counted input drops 2,954 → 1,222.

Window source — kept as declared-only, deliberately. This is the one point where I did not take the suggestion, and it is the issue author's call rather than mine: the Maka window is a target the user chooses to stay under, not a fact about the model, so the model's reported window is shown beside the setting with one click to apply (#4575) but never becomes the threshold on its own. The cost you name is real and stated in the description: with nothing declared, the first fill costs one rejection and possibly one cut reply. The context_window_suggestion note then offers the last accepted total; it is now written only when the turn actually surfaces the error, and the CLI transcript renders both new note kinds.

input_too_large producer. Correct — I removed the only producer. Both summarizer catches now classify ContextLength as input_too_large, with a fake provider that rejects above N characters as the regression.

Epoch and CHANGELOG. Bumped to 97 on rebase and the three sentences are back.

Inline items: eviction check compares input with input; the size floor applies to the initial fold only (the roll-forward input is the increment, as you say) and CheckpointSummaryFoldContext carries usage only, so the builder/copy seams validate structure and truncation; the suggestion note moves after recovery and names the fold-proof last accepted total; the run-header test's anchor now sits above the window so the header check decides; the synthetic /compact row test is restored; the roll-forward floor test now asserts both directions.

Two things I left as they were, with reasons. The Host's runComposition.contextWindow stays the model's reported window: it is a protocol fact clients already display, and the declared window gets its own surface in the composer indicator (#4576) rather than a changed meaning for an existing field. And the memory-gate unavailable outcome is still exercised by the mid-turn fixture's extraction path (remember returns unavailable there, asserted at the fold that dispatches extraction); if you meant a different obligation, tell me which and I will pin it.

The runtime no longer estimates whether a request fits a context window.
Every "does it fit" question is answered by a provider: the conversation
model's own context-length rejection is recovered by one compact-and-retry,
and the summarizer's provider answers for compaction input (input_too_large
retreats the fold by half). The chars/4 payload ruler, the signed delta
estimate, the 32,000-token fallback history budget, the quarter-window
reserve, the replacement-not-smaller and prefix-over-budget replay gates,
and the final-request rescue re-entry are removed.

Proactive compaction keeps one trigger: the previous accepted request's
real input plus output tokens, as the provider counted them, compared with
the context window the user declared for the model (a model-facts pin or a
relay profile). A provider's /models report and generated metadata are no
longer a threshold on their own. With no declaration there is no proactive
fold; the provider decides. A reply the provider cut at its output limit
(finishReason length) folds once before the next request.

The persisted last-request anchor becomes { inputTokens, outputTokens };
the retired payloadChars key still decodes so 0.2.0 sessions keep loading.
Summaries are capped at 8,000 output tokens with one shorter retry, and the
too-small-for-fold floor reads the summarizer call's real usage instead of
a chars/4 estimate. Two user-visible notes explain provider-side context
changes: context_provider_dropping (an append-only step whose usage did not
grow) and context_window_suggestion (a rejection at a proven-fit total,
with the number the user can declare).

Closes apache#4559
Refs apache#4458, apache#4486

Generated-by: Claude Code
Claude-Session: https://claude.ai/code/session_014ajaRxC4jydavY9nYUFj5J
Usage is the only signal the runtime's context handling now reads: the
proactive compaction baseline, the eviction check, the window suggestion
and the usage indicator all start from the provider's own token counts.
A Chat Completions server returns none unless the request carries
`stream_options.include_usage`, and the factory only sent that for the two
providers whose registry entries opted in. A local Ollama and every plain
OpenAI-compatible relay — the connection type behind apache#4458 — therefore
produced no usage at all, and everything downstream stayed dark.

Live probe against Ollama (qwen2.5:0.5b): before, the finish chunk carried
an empty usage object and no token_usage event was emitted; after, every
turn reports its input and output and persists the anchor.

The default is now on for the OpenAI-compatible Chat adapter; a registry
entry can still opt out with `includeUsage: false`.

Refs apache#4559

Generated-by: Claude Code
Claude-Session: https://claude.ai/code/session_014ajaRxC4jydavY9nYUFj5J
… can answer

The folded span handed to the summarizer usually ends on an assistant
message. A chat-template model given a conversation that already ends with
its own turn emits an end-of-sequence token and nothing else — observed
live on Ollama with qwen2.5:0.5b and qwen2.5:1.5b: finish `stop`, one
output token, empty text, on every compaction attempt. The write gate then
fails the fold open, so history compaction never succeeded against a local
model. The request now closes with a user instruction to write the
structured summary; on the first attempt and on the repair.

Live against Ollama (qwen2.5:1.5b, declared 1,500-token window): the fold
that previously produced an empty summary now writes a 1,737-character
checkpoint after one repair, and the next request's provider-counted input
drops from 2,954 to 1,222 tokens.

Refs apache#4559

Generated-by: Claude Code
Claude-Session: https://claude.ai/code/session_014ajaRxC4jydavY9nYUFj5J
The two user-visible compaction notes keyed on the `priorReplay` stage only.
Since apache#4486 every new fold happens in the request-projection hook
(`activeStep`), so the turn that was actually compacted showed nothing and
the note arrived one turn later, when the checkpoint was replayed; a fold
that failed open in the hook was never surfaced at all. Live against Ollama
a fold succeeded and a fold failed in consecutive turns with no note either
time. Both predicates now accept a history-compaction decision from either
stage; the once-per-send flags in the backend are unchanged.

Refs apache#4559

Generated-by: Claude Code
Claude-Session: https://claude.ai/code/session_014ajaRxC4jydavY9nYUFj5J
Review findings from apache#4574, each with a test that fails on the previous
commit:

- The proactive trigger now reserves the model's declared output limit
  (`connection.models[].maxOutputTokens` or generated metadata, a provider
  fact; 0 when none is declared) and fires on `>=`: with the window
  declared at the provider's real size, an accepted request alone can never
  exceed it, and the reply the next request must leave room for is what
  tips it. The window itself stays the user's declaration.
- The summarizer's provider is a decider again: a context-length rejection
  from the summary call (or the Codex compaction call) surfaces as
  `input_too_large`, the reason the planner retreats on. Removing the
  estimated pre-fit had left that path without a producer.
- The eviction check compares input against input: the previous reply's
  reasoning is not resent on every wire, so input + output is not a floor.
- The window suggestion is written after recovery, only when the turn
  surfaces the rejection, and names the last accepted total, which a fold
  does not clear. It is no longer a number on the `>` boundary.
- The size floor judges an initial fold only; on a roll-forward the
  summarizer's input is the increment, not the covered span.
- `CheckpointSummaryFoldContext` carries usage only; the builder and copy
  seams validate structure and truncation, which is all they can.
- The CLI transcript renders the two new note kinds.
- Restored coverage: a synthetic /compact usage row does not shadow the
  real anchor; the run-header check is exercised with an anchor above the
  window.
- CHANGELOG carries the downgrade and epoch sentences again.

The Host's reported `contextWindow` stays the model's window: it is a
protocol fact clients already display, and the declared window has its own
surface in the composer indicator.

Refs apache#4559

Generated-by: Claude Code
Claude-Session: https://claude.ai/code/session_014ajaRxC4jydavY9nYUFj5J
Two new system_note kinds and the reshaped lastRequestAnchor land in
@maka/core's closed allowlists. A merge-base client handshakes on strict
epoch equality and would then fail to decode the first transcript carrying
them; the epoch moves so the pair refuses each other at the handshake (main is at 98 after apache#4561/apache#4568, so this lands as 99).

Refs apache#4559

Generated-by: Claude Code
Claude-Session: https://claude.ai/code/session_014ajaRxC4jydavY9nYUFj5J
@Joob1n
Joob1n force-pushed the fix/context-provider-decides branch from a650762 to a67327b Compare September 3, 2026 02:22
@Joob1n

Joob1n commented Sep 3, 2026

Copy link
Copy Markdown
Contributor Author

Rebased on main@8ea3c4f05 now that #4578 has landed, so the dependency bump is no longer carried here (6 commits). main moved the compatibility epoch to 98 in the meantime (#4561 / #4568), so this PR's bump is now 98 → 99; the CHANGELOG line and the epoch comment follow. No other changes since a65076258.

@Joob1n
Joob1n requested review from Astro-Han September 3, 2026 02:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

effort/XL Over 1000 readable lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Context budget: stop estimating fit; the provider decides, the Maka window is a user target

2 participants