Skip to content

Add opt-in SSH costs for Claude and Codex - #3545

Draft
wicolian wants to merge 8 commits into
steipete:mainfrom
wicolian:codex/ssh-remote-costs
Draft

Add opt-in SSH costs for Claude and Codex#3545
wicolian wants to merge 8 commits into
steipete:mainfrom
wicolian:codex/ssh-remote-costs

Conversation

@wicolian

@wicolian wicolian commented Sep 10, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Adds a default-off Fetch costs from SSH devices toggle with up to eight SSH config aliases.
  • Shows each SSH box and provider separately, then lets users explicitly select same-account boxes for combined totals with this Mac.
  • Stacks selected SSH devices into the existing daily cost chart in a separately configurable color; SSH-only days are visible and chart totals include both sources.
  • Keeps Claude and Codex totals separate and marks unavailable or incomplete combined inputs as partial.
  • Generalizes cost --remote and --summary-only to Claude, Codex, and both.
  • Does not use or require Agent Sessions.

Safety

  • Uses system SSH with BatchMode, a five-second connect timeout, existing host-key trust, and no interactive prompts.
  • Transfers bounded numeric summaries and daily buckets only; no transcripts, paths, model names, session IDs, account identifiers, or credentials.
  • Limits remote daily history to the requested window with a hard 365-day ceiling and a 64 KiB output cap.
  • Account sameness is a local user choice rather than inferred or transmitted identity.
  • Throttles normal refreshes to once per 15 minutes, preserves that throttle when opening the menu, and clears in-memory reports when disabled.
  • Warns that copied or resumed sessions present on multiple machines can make additive totals double-count.

Live UI proof

Exact-branch Claude cost history with this Mac in orange and the selected SSH device in the configured cyan color:

Live combined local and SSH cost chart

Exact-branch settings with sandbox selected for SSH costs and same-account combination while Agent Sessions remains independent:

Live SSH cost settings

Verification

  • Exact head 3abd4ca6: built, packaged, and launched the app, then captured both screenshots from that running binary.
  • Full test harness: 1,070 selections in 90/90 groups, with zero failures, retries, or timeouts.
  • Focused SSH/chart/controller coverage: 78 tests in 3 suites pass.
  • Provider architecture gatekeeper: 41/41 tests pass.
  • SwiftFormat, SwiftLint strict, and git diff --check pass for the touched code; all repository-owned make check gates pass. The final TypeScript wrapper still encounters an unrelated parent /Users/kelashik/node_modules/@types/node installation missing undici-types; the project passes with an isolated empty type root.
  • Live SSH proof on the configured Ubuntu sandbox: fresh 90-day JSON returned both providers and dated daily buckets, including 249,005,023 Codex tokens / $353.61 and 6,956,682,068 Claude tokens / $4,164.95. The screenshots show those SSH buckets reaching the receiving chart path; host identity is omitted from published proof.
  • Regression coverage now includes disabled/unsupported SSH selections, partial qualifiers for Claude and cost mode, and SSH-only daily chart availability.
  • Current-head GitHub checks are pending after this update.

This remains a draft pending maintainer scope approval for additive same-account estimates.

@clawsweeper

clawsweeper Bot commented Sep 10, 2026

Copy link
Copy Markdown

🦞👀
ClawSweeper picked this up.

Pull request received. I will update this pull request when review starts.

ClawSweeper review complete

ClawSweeper finished reviewing this revision. The review result is being finalized.

View the workflow run.

Ennio Cuteri and others added 3 commits September 11, 2026 02:49
Mac cost reports cannot currently show native Codex history from a remote
Linux or macOS host. Add separate per-host estimates to the native menu and
Usage & Spend, with explicit SSH aliases and a CLI --remote mode.

Reuse the existing cost scanner and send only compact summary JSON. Preserve
host calendars, pricing provenance, unknown costs, and incomplete coverage.
Keep local totals independent, fail per host, and cancel or discard reports
when the configured hosts or history window change. Do not sum overlapping
histories or copy raw transcripts, session identifiers, or account metadata.

Document the opt-in protocol and updated-CLI requirement. Extend existing
tests for validation, scoped failures, cancellation, menu integration, and
native controls; relocate unchanged architecture-catalog source anchors.

Validation: make check passed with zero violations; make test passed all
89 groups without retries or timeouts. Linux ARM64 CLI build, real Mac-to-DGX
SSH reports, and native Apply/error/disconnect interactions also passed.

Requires updated CodexBar on the remote host. Reports remain API-equivalent
estimates and deliberately do not combine overlapping host histories.

Refs: steipete#3036
Remote cost configuration reused whole-target lowercase deduplication, so
Alice@server and alice@server collapsed into one account. Normalize only
the hostname in the shared SSH helper, preserving distinct usernames for
both remote cost reports and live sessions.

Extend the existing target checks to retain both accounts while still
deduplicating hostname case variants. Exercise empty and legacy stored
preferences to confirm remote costs require separate opt-in and preserve
existing agent-session settings.

Validation: the two identity regressions failed before the fix and pass
after it; 39 focused tests, make check, Linux ARM64 CLI build, real Mac-to-DGX
SSH success/failure checks, and native Apply persistence all pass.
Full make test (group size 48): all 1,060 selections / 23 groups passed
on the first attempt, with no retries or timeouts.

The PR remains a draft pending maintainer approval of the CLI/native
polling scope. No production app or remote CLI was permanently replaced.
@clawsweeper clawsweeper Bot added P2 Normal priority bug or improvement with limited blast radius. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. labels Sep 10, 2026
@clawsweeper

clawsweeper Bot commented Sep 10, 2026

Copy link
Copy Markdown

Codex review: blocked before merge. Reviewed September 11, 2026, 1:50 PM ET / 17:50 UTC (Revision 10).

ClawSweeper review

What this changes

Adds optional SSH cost reports for Claude and Codex, saved host controls, separate CLI reports, and selectable combined totals with stacked charts.

Merge readiness

Blocked before merge - 5 items remain

This remains useful work absent from main. The updated proof is sufficient, but the remote-only chart repair is incomplete and its new fallback misstates coverage.

Priority: P2
Reviewed head: 3abd4ca6e1fed0347570034c5b8de707045329c2
Owner decision: Required. See Decision needed.

Review scores

Measure Result What it means
Overall readiness 🦐 gold shrimp (3/6) Useful, demonstrated functionality remains limited by two concrete chart integration defects.
Proof confidence 🐚 platinum hermit (4/6) ✨ media proof bonus Sufficient (screenshot): The captured exact-head Ubuntu SSH results and inspected native screenshots connect remote summaries to enabled combined-host controls and the receiving stacked Claude chart. They resolve the earlier live-demonstration gap; partial-provider tests and the related PR's real failure trace provide supplemental coverage.
Patch quality 🦐 gold shrimp (3/6) 2 actionable review findings remain.

Verification

Check Result Evidence
Real behavior Verified Sufficient (screenshot): The captured exact-head Ubuntu SSH results and inspected native screenshots connect remote summaries to enabled combined-host controls and the receiving stacked Claude chart. They resolve the earlier live-demonstration gap; partial-provider tests and the related PR's real failure trace provide supplemental coverage.
Evidence reviewed 9 items Repository policy and scope: Read the full root AGENTS.md and VISION.md. No nested AGENTS.md or maintainer-notes directory was found. VISION.md requires sign-off for new features; the supplied repository profile also prohibits automatic cleanup closure.
Main still lacks the proposed cost mode: The fetched main cost command proceeds directly into existing local/provider cost handling without remote or summary-only dispatch. Its source tree contains no remote cost implementation.
Latest release boundary: The cost help at the supplied v0.59.0 release commit lists existing cost options without --remote or --summary-only.
Findings 2 actionable findings [P2] Include remote history in the outer menu availability check
[P2] Mark the missing-local fallback as incomplete
Security None None.

How this fits together

CodexBar reads local provider histories to estimate token usage and cost. This change runs the remote CodexBar CLI over SSH, validates numeric summaries, and feeds them into native reports and charts alongside local estimates.

flowchart LR
  A[Saved SSH hosts] --> B[Opt-in refresh scheduler]
  B --> C[System SSH and remote CLI]
  C --> D[Validated provider summaries]
  D --> E[Separate host reports]
  D --> F[Selected combined estimates]
  G[Local cost history] --> F
  F --> H[Native cost charts]
Loading

Decision needed

Question Recommendation
Should the first SSH-cost release include native polling and additive Claude/Codex charts, or start with separate per-host reporting? Start with separate reports: Coordinate one landing path with #3538 and defer additive charts until their accounting contract is approved.

Why: VISION.md requires feature sign-off, the contributor explicitly awaits it, and the overlapping proposal does not settle the double-counting tradeoff.

Before merge

  • Include remote history in the outer menu availability check (P2) - The previous remote-only finding remains partly unresolved. With selected SSH daily buckets and an empty local daily array, this check still sets hasCostHistory to false. Both the ordinary history-menu insertion and the sectioned cost-card submenu consult that flag before calling the newly repaired makeCostHistorySubmenu, so users still cannot reach the SSH-only chart through those paths. Reuse the remote-aware availability predicate here and cover the menu context, rather than only the helper.
  • Mark the missing-local fallback as incomplete (P2) - When the local snapshot is unavailable but SSH history exists, this new fallback inherits CostUsageTokenSnapshot's default historyCoverageIsEstablished: true. If the remote sources are complete, chart construction therefore suppresses the partial indicator and presents their subtotal as a complete combined estimate, despite the missing Mac contribution. Set the fallback coverage explicitly to false and cover this missing-local case.
  • Resolve merge risk (P1) - Selected-host totals deliberately add potentially overlapping histories without deduplication; account sameness is a user assertion, so accepting these estimates requires an explicit accounting decision.
  • Complete next step (P2) - Repair both chart findings, obtain scope approval for native polling and additive estimates, and coordinate one landing path with Add opt-in SSH Codex cost reports #3538.
  • Resolve maintainer decision - Resolve the maintainer decision shown above before merge.

Findings

  • [P2] Include remote history in the outer menu availability check — Sources/CodexBar/StatusItemController+Menu.swift:507-508
  • [P2] Mark the missing-local fallback as incomplete — Sources/CodexBar/StatusItemController+HostedSubmenus.swift:347-354
Agent review details

Security

None.

Review metrics

Metric Value Why it matters
Production and test growth Production +1,299 net lines; tests +724 net lines The added transport, controls, and chart integration explain the growth, while its scope still requires sign-off.

Root-cause cluster

Relationship: fixed_by_candidate
Canonical: #3036
Summary: This is an expanded implementation candidate for the SSH-cost request; the separate Codex-only proposal overlaps without superseding it.

Members:

Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.

Merge-risk options

Maintainer options:

  1. Keep the initial reports separate (recommended)
    Defer combined totals while retaining useful per-host SSH reporting and its source provenance.
  2. Accept additive accounting
    Explicitly approve possible double-counting as an opt-in estimate rather than unique account spend.

Technical review

Best possible solution:

Keep per-host estimates independently usable and approve an explicit accounting contract before offering combined charts, with remote-only access and missing-source coverage handled consistently.

Do we have a high-confidence way to reproduce the issue?

Yes, by source inspection: selected SSH history with empty local daily history fails the outer menu check, and the missing-local fallback defaults coverage to complete. No runtime tests were executed during this read-only review.

Is this the best way to solve the issue?

Unclear overall: reusing the remote CLI and retaining bounded per-host summaries is sound, but additive accounting needs approval and the chart integration still has two concrete defects.

Full review comments:

  • [P2] Include remote history in the outer menu availability check — Sources/CodexBar/StatusItemController+Menu.swift:507-508
    The previous remote-only finding remains partly unresolved. With selected SSH daily buckets and an empty local daily array, this check still sets hasCostHistory to false. Both the ordinary history-menu insertion and the sectioned cost-card submenu consult that flag before calling the newly repaired makeCostHistorySubmenu, so users still cannot reach the SSH-only chart through those paths. Reuse the remote-aware availability predicate here and cover the menu context, rather than only the helper.
    Confidence: 0.99
  • [P2] Mark the missing-local fallback as incomplete — Sources/CodexBar/StatusItemController+HostedSubmenus.swift:347-354
    When the local snapshot is unavailable but SSH history exists, this new fallback inherits CostUsageTokenSnapshot's default historyCoverageIsEstablished: true. If the remote sources are complete, chart construction therefore suppresses the partial indicator and presents their subtotal as a complete combined estimate, despite the missing Mac contribution. Set the fallback coverage explicitly to false and cover this missing-local case.
    Confidence: 0.97

Overall correctness: patch is incorrect
Overall confidence: 0.96

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning medium; reviewed against 1c4650d1b421.

Labels

Label changes:

  • add proof: sufficient: Contributor real behavior proof is sufficient. The captured exact-head Ubuntu SSH results and inspected native screenshots connect remote summaries to enabled combined-host controls and the receiving stacked Claude chart. They resolve the earlier live-demonstration gap; partial-provider tests and the related PR's real failure trace provide supplemental coverage.
  • add status: ⏳ waiting on author: ClawSweeper has contributor-facing work open and is waiting for author action. Sufficient (screenshot): The captured exact-head Ubuntu SSH results and inspected native screenshots connect remote summaries to enabled combined-host controls and the receiving stacked Claude chart. They resolve the earlier live-demonstration gap; partial-provider tests and the related PR's real failure trace provide supplemental coverage.
  • remove status: 📣 needs proof: Current PR status label is status: ⏳ waiting on author.

Label justifications:

  • P2: This is an optional reporting improvement with bounded chart defects and no demonstrated disruption to existing local reporting.
  • merge-risk: 🚨 other: Additive cross-host estimates can double-count copied or resumed histories, an accounting tradeoff that green CI cannot resolve.
  • rating: 🦐 gold shrimp: Overall readiness is 🦐 gold shrimp; proof is 🐚 platinum hermit and patch quality is 🦐 gold shrimp.
  • status: ⏳ waiting on author: ClawSweeper has contributor-facing work open and is waiting for author action. Sufficient (screenshot): The captured exact-head Ubuntu SSH results and inspected native screenshots connect remote summaries to enabled combined-host controls and the receiving stacked Claude chart. They resolve the earlier live-demonstration gap; partial-provider tests and the related PR's real failure trace provide supplemental coverage.
  • proof: sufficient: Contributor real behavior proof is sufficient. The captured exact-head Ubuntu SSH results and inspected native screenshots connect remote summaries to enabled combined-host controls and the receiving stacked Claude chart. They resolve the earlier live-demonstration gap; partial-provider tests and the related PR's real failure trace provide supplemental coverage.
  • proof: 📸 screenshot: Contributor real behavior proof includes screenshot evidence. The captured exact-head Ubuntu SSH results and inspected native screenshots connect remote summaries to enabled combined-host controls and the receiving stacked Claude chart. They resolve the earlier live-demonstration gap; partial-provider tests and the related PR's real failure trace provide supplemental coverage.

Evidence

What I checked:

  • Repository policy and scope: Read the full root AGENTS.md and VISION.md. No nested AGENTS.md or maintainer-notes directory was found. VISION.md requires sign-off for new features; the supplied repository profile also prohibits automatic cleanup closure. (VISION.md:13, 3abd4ca6e1fe)
  • Main still lacks the proposed cost mode: The fetched main cost command proceeds directly into existing local/provider cost handling without remote or summary-only dispatch. Its source tree contains no remote cost implementation. (Sources/CodexBarCLI/CLICostCommand.swift:30, 1c4650d1b421)
  • Latest release boundary: The cost help at the supplied v0.59.0 release commit lists existing cost options without --remote or --summary-only. (Sources/CodexBarCLI/CLIHelp.swift:115, dca9c5f4b7a2)
  • Re-review continuity: The exact head commit adds disabled/unsupported selection guards, visible partial qualifiers, and remote-aware chart construction. The outer menu-context check still requires local daily entries, so the previous remote-only availability finding remains partly unresolved. (Sources/CodexBar/StatusItemController+Menu.swift:507, 3abd4ca6e1fe)
  • Missing local coverage becomes complete: The new remote-only fallback omits historyCoverageIsEstablished. CostUsageTokenSnapshot defaults that field to true, and chart construction combines it with remote coverage, allowing missing local history to appear complete. (Sources/CodexBar/StatusItemController+HostedSubmenus.swift:347, 3abd4ca6e1fe)
  • Current real behavior proof: Inspected both prepared screenshots from the captured body at sourceRevision 4d14c9f7d868ba8d2bd26415ceacf12f15ea6ff06fdbd23ae08aa54636a1a3d3. They show enabled SSH and combined-host controls, a selected cyan color, and native Claude chart bars containing local and SSH values. The body identifies the exact head and reports real Ubuntu SSH results for both providers; the chart total agrees with the reported local and remote Claude totals. (docs/screenshots/remote-codex-costs.png, 3abd4ca6e1fe)

Likely related people:

  • unknown: The claimed source-line change could not be verified from bounded local history. (role: source history unknown; confidence: low)
  • unknown: The claimed source-line change could not be verified from bounded local history. (role: source history unknown; confidence: low)

Rank-up moves

Optional improvements that raise the rating; they are not merge blockers.

  • Cover the actual menu-context path with SSH history and empty local daily buckets.
  • Keep missing-local fallback coverage partial and add a focused regression check.

Rating scale

Score Internal tier Crab rank Meaning
6/6 S 🦀 challenger crab Exceptional readiness
5/6 A 🦞 diamond lobster Very strong readiness
4/6 B 🐚 platinum hermit Good normal PR; ordinary maintainer review
3/6 C 🦐 gold shrimp Useful, but confidence is limited
2/6 D 🦪 silver shellfish Proof or implementation needs work
1/6 F 🧂 unranked krab Not merge-ready
N/A NA 🌊 off-meta tidepool Rating does not apply

Overall follows the weaker of proof and patch quality.
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

Workflow

  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

History

Review history (9 earlier review cycles; latest 8 shown)
  • reviewed 2026-09-10T21:38:11.291Z sha d5431e3 :: needs real behavior proof before merge. :: [P2] Emit an error report for every missing remote provider
  • reviewed 2026-09-10T21:52:23.742Z sha 5d3164a :: needs real behavior proof before merge. :: [P2] Update the CLI architecture anchors after adding the helper
  • reviewed 2026-09-10T22:02:12.777Z sha 9c9dcf1 :: needs real behavior proof before merge. :: none
  • reviewed 2026-09-10T22:13:12.084Z sha 9c9dcf1 :: needs real behavior proof before merge. :: [P2] Preserve the SSH throttle for automatic menu refreshes
  • reviewed 2026-09-11T11:03:00.087Z sha 2523ec7 :: needs real behavior proof before merge. :: [P2] Preserve the SSH throttle for automatic menu refreshes | [P2] Preserve incomplete-history warnings in combined totals
  • reviewed 2026-09-11T11:29:35.258Z sha 2523ec7 :: needs real behavior proof before merge. :: [P2] Preserve the SSH throttle for automatic menu refreshes | [P2] Preserve incomplete-history warnings in combined totals
  • reviewed 2026-09-11T12:16:39.878Z sha fe4cf5f :: needs real behavior proof before merge. :: [P2] Ignore saved combined hosts when SSH costs are disabled | [P2] Mark partial combined totals in the cost chart
  • reviewed 2026-09-11T12:23:12.171Z sha fe4cf5f :: needs real behavior proof before merge. :: [P2] Ignore saved combined hosts when SSH costs are disabled | [P2] Mark partial combined totals in the cost chart | [P2] Allow remote-only history through the chart availability guards

@wicolian
wicolian force-pushed the codex/ssh-remote-costs branch from 6098762 to d5431e3 Compare September 10, 2026 21:33
@wicolian

Copy link
Copy Markdown
Contributor Author

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented Sep 10, 2026

Copy link
Copy Markdown

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event exact_review_queue).
Result: when the review finishes, ClawSweeper will create the durable review comment if needed or update the existing comment in place.

Re-review progress:

@clawsweeper clawsweeper Bot added the merge-risk: 🚨 other 🚨 Merging this PR has meaningful risk outside the owned taxonomy. label Sep 11, 2026
@wicolian

Copy link
Copy Markdown
Contributor Author

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented Sep 11, 2026

Copy link
Copy Markdown

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event exact_review_queue).
Result: when the review finishes, ClawSweeper will create the durable review comment if needed or update the existing comment in place.

@clawsweeper clawsweeper Bot added the proof: 📸 screenshot Contributor real behavior proof includes screenshot evidence. label Sep 11, 2026
@wicolian

Copy link
Copy Markdown
Contributor Author

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented Sep 11, 2026

Copy link
Copy Markdown

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event exact_review_queue).
Result: when the review finishes, ClawSweeper will create the durable review comment if needed or update the existing comment in place.

Re-review progress:

@wicolian

Copy link
Copy Markdown
Contributor Author

Addressed all three actionable findings from the latest review on 3abd4ca6:

  • saved combined hosts are ignored while SSH costs are disabled and for unsupported providers;
  • incomplete Claude/token/cost charts now show a visible Partial state and qualify the estimated-total footer;
  • selected SSH daily history can create and render the chart even when this Mac has no local daily buckets.

I added focused regression coverage, reran the full 1,070-test harness cleanly in 90/90 groups, and rebuilt/launched the exact branch. The PR now uses live screenshots from that binary for both the stacked local + SSH Claude chart and the SSH cost settings. I also removed the PR-body port/builds-on framing and the auto-closing issue language while preserving the original commit authorship.

The PR remains draft for the maintainer decision on additive same-account estimates and the documented double-counting tradeoff.

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented Sep 11, 2026

Copy link
Copy Markdown

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event exact_review_queue).
Result: when the review finishes, ClawSweeper will create the durable review comment if needed or update the existing comment in place.

@clawsweeper clawsweeper Bot added proof: sufficient Contributor real behavior proof is sufficient. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. and removed status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. labels Sep 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge-risk: 🚨 other 🚨 Merging this PR has meaningful risk outside the owned taxonomy. P2 Normal priority bug or improvement with limited blast radius. proof: 📸 screenshot Contributor real behavior proof includes screenshot evidence. proof: sufficient Contributor real behavior proof is sufficient. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant