feat(desktop): show the model's reported context window beside the declared setting - #4575
Draft
Joob1n wants to merge 13 commits into
Draft
feat(desktop): show the model's reported context window beside the declared setting#4575Joob1n wants to merge 13 commits into
Joob1n wants to merge 13 commits into
Conversation
6 tasks
Joob1n
force-pushed
the
feat/context-window-setting-hint
branch
11 times, most recently
from
September 3, 2026 08:50
a47bfd5 to
7f865d3
Compare
The runtime no longer estimates whether a request fits a context window.
Every "does it fit" question is answered by a provider: the conversation
model's own context-length rejection is recovered by one compact-and-retry,
and the summarizer's provider answers for compaction input (input_too_large
retreats the fold by half). The chars/4 payload ruler, the signed delta
estimate, the 32,000-token fallback history budget, the quarter-window
reserve, the replacement-not-smaller and prefix-over-budget replay gates,
and the final-request rescue re-entry are removed.
Proactive compaction keeps one trigger: the previous accepted request's
real input plus output tokens, as the provider counted them, compared with
the context window the user declared for the model (a model-facts pin or a
relay profile). A provider's /models report and generated metadata are no
longer a threshold on their own. With no declaration there is no proactive
fold; the provider decides. A reply the provider cut at its output limit
(finishReason length) folds once before the next request.
The persisted last-request anchor becomes { inputTokens, outputTokens };
the retired payloadChars key still decodes so 0.2.0 sessions keep loading.
Summaries are capped at 8,000 output tokens with one shorter retry, and the
too-small-for-fold floor reads the summarizer call's real usage instead of
a chars/4 estimate. Two user-visible notes explain provider-side context
changes: context_provider_dropping (an append-only step whose usage did not
grow) and context_window_suggestion (a rejection at a proven-fit total,
with the number the user can declare).
Refs apache#4559
Refs apache#4458, apache#4486
Generated-by: Claude Code
Claude-Session: https://claude.ai/code/session_014ajaRxC4jydavY9nYUFj5J
Usage is the only signal the runtime's context handling now reads: the proactive compaction baseline, the eviction check, the window suggestion and the usage indicator all start from the provider's own token counts. A Chat Completions server returns none unless the request carries `stream_options.include_usage`, and the factory only sent that for the two providers whose registry entries opted in. A local Ollama and every plain OpenAI-compatible relay — the connection type behind apache#4458 — therefore produced no usage at all, and everything downstream stayed dark. Live probe against Ollama (qwen2.5:0.5b): before, the finish chunk carried an empty usage object and no token_usage event was emitted; after, every turn reports its input and output and persists the anchor. The default is now on for the OpenAI-compatible Chat adapter; a registry entry can still opt out with `includeUsage: false`. Refs apache#4559 Generated-by: Claude Code Claude-Session: https://claude.ai/code/session_014ajaRxC4jydavY9nYUFj5J
… can answer The folded span handed to the summarizer usually ends on an assistant message. A chat-template model given a conversation that already ends with its own turn emits an end-of-sequence token and nothing else — observed live on Ollama with qwen2.5:0.5b and qwen2.5:1.5b: finish `stop`, one output token, empty text, on every compaction attempt. The write gate then fails the fold open, so history compaction never succeeded against a local model. The request now closes with a user instruction to write the structured summary; on the first attempt and on the repair. Live against Ollama (qwen2.5:1.5b, declared 1,500-token window): the fold that previously produced an empty summary now writes a 1,737-character checkpoint after one repair, and the next request's provider-counted input drops from 2,954 to 1,222 tokens. Refs apache#4559 Generated-by: Claude Code Claude-Session: https://claude.ai/code/session_014ajaRxC4jydavY9nYUFj5J
The two user-visible compaction notes keyed on the `priorReplay` stage only. Since apache#4486 every new fold happens in the request-projection hook (`activeStep`), so the turn that was actually compacted showed nothing and the note arrived one turn later, when the checkpoint was replayed; a fold that failed open in the hook was never surfaced at all. Live against Ollama a fold succeeded and a fold failed in consecutive turns with no note either time. Both predicates now accept a history-compaction decision from either stage; the once-per-send flags in the backend are unchanged. Refs apache#4559 Generated-by: Claude Code Claude-Session: https://claude.ai/code/session_014ajaRxC4jydavY9nYUFj5J
Review findings from apache#4574, each with a test that fails on the previous commit: - The proactive trigger now reserves the model's declared output limit (`connection.models[].maxOutputTokens` or generated metadata, a provider fact; 0 when none is declared) and fires on `>=`: with the window declared at the provider's real size, an accepted request alone can never exceed it, and the reply the next request must leave room for is what tips it. The window itself stays the user's declaration. - The summarizer's provider is a decider again: a context-length rejection from the summary call (or the Codex compaction call) surfaces as `input_too_large`, the reason the planner retreats on. Removing the estimated pre-fit had left that path without a producer. - The eviction check compares input against input: the previous reply's reasoning is not resent on every wire, so input + output is not a floor. - The window suggestion is written after recovery, only when the turn surfaces the rejection, and names the last accepted total, which a fold does not clear. It is no longer a number on the `>` boundary. - The size floor judges an initial fold only; on a roll-forward the summarizer's input is the increment, not the covered span. - `CheckpointSummaryFoldContext` carries usage only; the builder and copy seams validate structure and truncation, which is all they can. - The CLI transcript renders the two new note kinds. - Restored coverage: a synthetic /compact usage row does not shadow the real anchor; the run-header check is exercised with an anchor above the window. - CHANGELOG carries the downgrade and epoch sentences again. The Host's reported `contextWindow` stays the model's window: it is a protocol fact clients already display, and the declared window has its own surface in the composer indicator. Refs apache#4559 Generated-by: Claude Code Claude-Session: https://claude.ai/code/session_014ajaRxC4jydavY9nYUFj5J
Two new system_note kinds and the reshaped lastRequestAnchor land in @maka/core's closed allowlists. A merge-base client handshakes on strict epoch equality and would then fail to decode the first transcript carrying them; the epoch moves so the pair refuses each other at the handshake (main is at 101 after apache#4608, so this lands as 102). Refs apache#4559 Generated-by: Claude Code Claude-Session: https://claude.ai/code/session_014ajaRxC4jydavY9nYUFj5J
…epair The repair request carries a longer system prompt than the first one. When the first summary fit but the repair was rejected for context length, the repair catch filed the rejection under the initial malformed defect, so the planner never saw `input_too_large` and never retreated. Classify it before wrapping. Regression test covers "initial malformed, repair overflow". Also align both architecture documents with the implemented active-turn trigger: previous accepted usage plus the reply reserve (the model's declared `maxOutputTokens`, zero when unknown) reaching the declared window, not bare usage exceeding it. Refs apache#4559 Generated-by: Claude Code Claude-Session: https://claude.ai/code/session_014ajaRxC4jydavY9nYUFj5J
Every OpenAI-compatible chat request asks for `stream_options.include_usage`. A relay that rejects the field answers 400; the runtime surfaces that as the provider's own error and does not resend without the field. A connection that cannot report usage has no context baseline, and the user should learn that from the error, not from a compaction that never happens. Refs apache#4559 Generated-by: Claude Code Claude-Session: https://claude.ai/code/session_014ajaRxC4jydavY9nYUFj5J
The check required a strict decrease, but the copy, the PR summary and apache#4623 all define the signal as "content was appended but usage did not grow", and the live Ollama evidence is a plateau at 3,716 input tokens, not a drop: a provider truncating to its own window reports the same total on every later request. Compare with `<=` so the promised contract is the implemented one. An equal count after an append-only step is already impossible without provider-side eviction or rewriting. Also sync two passages in the Chinese architecture document that still carried the retired size policy: the materialization inputs and the checkpoint's replay question now match the English text. Refs apache#4559 Generated-by: Claude Code Claude-Session: https://claude.ai/code/session_014ajaRxC4jydavY9nYUFj5J
…note Three defects found by review against production kimi-coding-plan/k3-256k data (apache#4634), each on a path where the provider does not speak clearly. The per-send failure latch only recorded malformed-summary reasons, but the baseline that fired the trigger survives a fail-open, so after a provider error or an output-length failure every later step re-evaluated the same condition and dispatched the same doomed call: 15 consecutive failed summarizer calls over ~47 minutes before one main request. Latch every fail-open reason for the rest of the send. A `finishReason: length` was folded unconditionally, but a reply that stopped at the `maxOutputTokens` Maka itself sends was cut by Maka's own budget, and the next request carries the same budget. Folding there shrinks history every step without touching the constraint. Only a cut below that budget is the provider running out of room. The provider-dropping note excluded folds, prunes and image omissions but not Maka's own tool-schema changes: a finalization step resolves an empty tool set and legitimately drops several thousand schema tokens, which read as the provider evicting context. Refs apache#4559, apache#4634 Generated-by: Claude Code Claude-Session: https://claude.ai/code/session_014ajaRxC4jydavY9nYUFj5J
The reply reserve that arms the proactive threshold was the model's declared maximum output. On a model whose limit is a large fraction of its window (k3-256k reports 131,072 against 262,144) that folds at half the declared window, long before the user's declaration is anywhere near crossed. Measure the room from the reply the model actually wrote instead: twice the last one, bounded at 8,000 tokens. Both numbers are provider-reported, the margin tracks the session's own behaviour, and no previous reply means no reserve. A reserve measured from a smaller previous reply can still be too small, and that is now visible rather than silent: when an accepted request's own input plus its reply exceed the declared window, the transcript carries one `context_window_overrun` note per send naming both numbers. The next request folds anyway, because the baseline now exceeds the window; the note explains why, and tells a user who wants whole replies to raise the declaration. The mid-turn fixtures are retuned around the new reserve: their steps report 100/20, 150/30 and 120/10, so a default window of 190 keeps the first request inside it and crosses on the second, which is the journey they describe. Refs apache#4559, apache#4634 Generated-by: Claude Code Claude-Session: https://claude.ai/code/session_014ajaRxC4jydavY9nYUFj5J
…ndow Some providers accept a request larger than the window their own catalog entry reports, without rejecting it and without truncating: review of this PR observed a kimi-coding-plan session accepted at 322K input tokens against a reported 262,144, with usage still growing (apache#4634). With nothing declared, every signal this design reads stays dark there — no rejection to recover from, no plateau to read, and no declaration to arm the proactive threshold — so the session degrades quietly and indefinitely. One note per send now states the two real numbers: what the exchange used and what the model reports. It changes nothing on its own. A reported window is still a hint rather than a declaration, and Maka still declares nothing on the user's behalf; the note is what makes the silent case visible enough for them to decide. With a declaration in place the overrun note already owns this ground, so the two never fire for the same fact. Refs apache#4559, apache#4634 Generated-by: Claude Code Claude-Session: https://claude.ai/code/session_014ajaRxC4jydavY9nYUFj5J
…clared setting The context-window setting is the Maka window: the compaction target the user declares. When nothing is declared and the provider's /models report carries a window for the model, show that number beside the empty field with a one-click "Use it" that fills the declaration. The reported value never takes effect on its own (apache#4559); the help text now says what the setting does and what leaving it empty means. Refs apache#4559 Generated-by: Claude Code Claude-Session: https://claude.ai/code/session_014ajaRxC4jydavY9nYUFj5J
Joob1n
force-pushed
the
feat/context-window-setting-hint
branch
from
September 3, 2026 08:58
7f865d3 to
2b2aac6
Compare
6 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Stacked on #4574 (the first commit here is that PR; review only the last one).
After #4574 the context-window setting is the Maka window: the compaction target the user declares, and the only proactive threshold. A provider's
/modelsreport no longer takes effect on its own. This PR makes that visible where the user sets it: when nothing is declared and the connection's model catalog reports a window, the setting shows "This model declares a N-token window" with a one-click Use it that fills the declaration. The reported value never applies by itself. The help text now says what the setting does ("Maka compacts once the previous request's real usage exceeds it") and what leaving it empty means ("never compact proactively; the provider decides").Refs #4559
Verification
npm --workspace @maka/desktop run typecheck,npm run check:renderer-architecture -- --base origin/main(no debt increase inprovider-connection-detail.tsx),npm run astryx:surface-inventory,npm run lint,npm run format:check— clean. Not run locally: Desktop e2e — relying on CI.Self-review
connection.models[].contextWindow(the Host's catalog entry) rather than models.dev directly, so it stays inside the dependency debt the ratchet allows for this legacy file.setDraftContextWindowpath as typing the value, so the declaration is persisted and validated exactly as a manual entry would be.AI use
Select exactly one:
Tool(s) and scope: Claude Code — implementation; reviewed and verified by the author.
Checklist
Does this PR entail a change in behavior?
https://claude.ai/code/session_014ajaRxC4jydavY9nYUFj5J