Skip to content

Feature/tokens command and calculation - #393

Open
AlanAAG wants to merge 2 commits into
session-native-redesignfrom
feature/tokens_command_and_calculation
Open

Feature/tokens command and calculation#393
AlanAAG wants to merge 2 commits into
session-native-redesignfrom
feature/tokens_command_and_calculation

Conversation

@AlanAAG

@AlanAAG AlanAAG commented Jul 30, 2026

Copy link
Copy Markdown
Collaborator

What

Two related changes to how token usage is reported.

1. Fix token display math. input_tokens from every provider is the full prompt size, cache reads included. cached is a subset of it, never a sibling. The dashboard was rendering both as if they were independent, which read as double-counting.

  • Input now shows input − cached (genuinely new tokens)
  • Total now shows input + output with cached excluded, derived client-side instead of using the API's token.total (which is rawInput + output)
  • Cached % now measured against the raw prompt, not the new Input value. It's a cache hit rate, and output tokens are never cacheable so they don't belong in the denominator

2. Add /tokens. Per-session token breakdown, printed into the chat it was typed in. Deterministic: no LLM call, no agent dispatch, same shape as /clear.

Session token usage
Input: 79,022
Cached: 312,455
Output: 5,770
Total: 84,792

Why the new Session fields

Session.input_tokens/output_tokens/cache_tokens already existed but are per-run. reset_run_counters() is called from
session_manager.start_run() on every fresh run, so tt turn. The usage_events table has a session_idcolumn but it's never populated (collector.py:1236 passes task_id=None).

So per-chat cumulative totals didn't exist anywhere. This adds total_input_tokens / total_output_tokens / total_cache_tokens, incremented in attribute_usage_to_current_task alongdeliberately excluded from reset_run_counters().

No schema migration: session_storage persists the wh, and from_dict defaults to 0, so existing sessionsload clean and start counting.

Explicitly not changed

The storage layer is provider truth and other consumers depend on current semantics:

  • interface.py:2588 Anthropic's base_input + cache_creation + cache_read reassembly, and every other provider's extraction
  • storage.py:227 SUM(input_tokens + output_tokens)
  • token.py:40 billable_tokens, already uncached-only and documented as intentionally separate from display (it's the budget limiter)
  • WebSocket payload, SQLite schema

This is presentation-only plus the new cumulative coun

Testing

  • tests/test_tokens_command.py, 6 checks: totals sur, round-trip through to_dict/from_dict`,pre-existing sessions load as zero, the subtraction, the clamp when a provider over-reports cache reads, and unsaved sessions reading as
    zero
  • tests/test_token_attribution.py, 6 passed, no regression
  • tsc --noEmit clean, npm run build green
  • Verified live: /tokens against a real session returned 79,022 / 312,455 / 5,770 / 84,792, matching sessions.db
    (total_input=391,477, total_cache=312,455)

Reviewer notes

  • Existing sessions start at zero. No backfill, be were being wiped and usage_events never recordedsession_id. Current chats read 0 until they run again.
  • A brand-new chat reports zeros, not an error. Seon first message, so an unsent chat sends sessionId:"new" and session_manager.get() returns None. Handled via getattr(session, ..., 0).
  • **First LLM call of a session always shows Cached: e cache rather than reading it (interface.py:1406`records a miss). Correct, but looks like a bug if you test with a single message.
  • The dashboard's Total and /tokens' Total now agreet both differ from the raw token.total the API stillreturns.

Follow-up (not in this PR)

Populating session_id in usage_events would let per-session totals survive a DB rebuild and feed the dashboard, and would remove the
need for counters on the Session object at all.

@AlanAAG
AlanAAG requested a review from zfoong July 30, 2026 21:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant