Add opt-in SSH Codex cost reports - #3538
Open
ECuteri wants to merge 3 commits into
Open
Conversation
Mac cost reports cannot currently show native Codex history from a remote Linux or macOS host. Add separate per-host estimates to the native menu and Usage & Spend, with explicit SSH aliases and a CLI --remote mode. Reuse the existing cost scanner and send only compact summary JSON. Preserve host calendars, pricing provenance, unknown costs, and incomplete coverage. Keep local totals independent, fail per host, and cancel or discard reports when the configured hosts or history window change. Do not sum overlapping histories or copy raw transcripts, session identifiers, or account metadata. Document the opt-in protocol and updated-CLI requirement. Extend existing tests for validation, scoped failures, cancellation, menu integration, and native controls; relocate unchanged architecture-catalog source anchors. Validation: make check passed with zero violations; make test passed all 89 groups without retries or timeouts. Linux ARM64 CLI build, real Mac-to-DGX SSH reports, and native Apply/error/disconnect interactions also passed. Requires updated CodexBar on the remote host. Reports remain API-equivalent estimates and deliberately do not combine overlapping host histories. Refs: steipete#3036
|
🦞👀 Pull request received. I will update this pull request when review starts. ClawSweeper review completeClawSweeper finished reviewing this revision. The review result is being finalized. |
clawsweeper
Bot
added
P2
|
Codex review: blocked before merge. Reviewed September 11, 2026, 6:08 AM ET / 10:08 UTC (Revision 3). ClawSweeper reviewWhat this changesAdds opt-in SSH reports for remote Codex cost estimates in the CLI, menu, and Usage & Spend, keeping each host separate from local totals. Merge readiness⛔ Blocked before merge - 4 items remain Keep open: this remains a distinct, useful contribution with sufficient SSH runtime proof. One refresh defect remains, and the native polling scope still requires product approval. Priority: P2 Review scores
Verification
How this fits togetherCodexBar estimates token costs from locally recorded usage history. This change asks an explicitly configured SSH host to run its own scanner and returns compact summaries for display alongside local usage. flowchart LR
A[Explicit SSH hosts] --> B[CLI request or native refresh]
B --> C[System SSH authentication]
C --> D[Remote Codex history scanner]
D --> E[Validate compact summary]
E --> F[Separate host estimates or errors]
G[Local usage history] --> H[Unchanged local totals]
Decision needed
Why: VISION.md requires feature sign-off, and the owner's related discussion specifically deferred persistent native behavior; runtime proof cannot resolve that product choice. Before merge
Findings
Agent review detailsSecurityNone. Review metrics
Root-cause clusterRelationship: Members:
Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything. Merge-risk optionsMaintainer options:
Technical reviewBest possible solution: Prefer an approved CLI-first summary contract with separate host estimates; include native polling only with explicit scope approval and reliable forced-refresh handling. Do we have a high-confidence way to reproduce the issue? Yes, by source inspection of the PR: start an automatic poll, then request force=true before it finishes; the active-task guard discards the forced request. No execution was performed. Is this the best way to solve the issue? Partly: compact summaries reuse the scanner and avoid raw-log retention and double-counting, but the scheduler must preserve explicit refresh requests and the native scope needs approval. Full review comments:
Overall correctness: patch is incorrect AGENTS.md: found and applied where relevant. Codex review notes: model internal, reasoning medium; reviewed against 4cfb60692bdd. LabelsLabel changes:
Label justifications:
EvidenceWhat I checked:
Likely related people:
Rank-up movesOptional improvements that raise the rating; they are not merge blockers.
Rating scale
Overall follows the weaker of proof and patch quality. Workflow
History |
Remote cost configuration reused whole-target lowercase deduplication, so Alice@server and alice@server collapsed into one account. Normalize only the hostname in the shared SSH helper, preserving distinct usernames for both remote cost reports and live sessions. Extend the existing target checks to retain both accounts while still deduplicating hostname case variants. Exercise empty and legacy stored preferences to confirm remote costs require separate opt-in and preserve existing agent-session settings. Validation: the two identity regressions failed before the fix and pass after it; 39 focused tests, make check, Linux ARM64 CLI build, real Mac-to-DGX SSH success/failure checks, and native Apply persistence all pass. Full make test (group size 48): all 1,060 selections / 23 groups passed on the first attempt, with no retries or timeouts. The PR remains a draft pending maintainer approval of the CLI/native polling scope. No production app or remote CLI was permanently replaced.
clawsweeper
Bot
added
proof: sufficient
Resolve the architecture catalog conflict by retaining upstream's current constructs and relocating unchanged source anchors for the remote-cost additions. Preserve every existing expectation and suppression rule. Move the pending feature's changelog entry to the new unreleased version. Group observation reads using the existing tuple pattern so the combined UsageStore changes stay within the repository's file-length limit. Validation: make check passes with zero violations. The full serial make test run passes all 1070 selections across 23 groups on the first attempt, without retries or timeouts. The remote transport and opt-in contract remain unchanged. Maintainer feature-scope approval is still required before merging upstream.
ECuteri
marked this pull request as ready for review
September 11, 2026 09:57
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 145eb1ed53
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| self.hosts = hosts | ||
| self.historyDays = historyDays | ||
| } | ||
| guard !hosts.isEmpty, self.task == nil else { return } |
There was a problem hiding this comment.
When the menu-open or dashboard-appearance poll is still in progress, clicking Usage & Spend’s Refresh button calls this method with force: true, but this guard returns before examining force because self.task is non-nil. The in-flight loader was started with force: false, so it does not pass --refresh to the remote CLI and may return cached scan data despite the explicit refresh; queue a forced follow-up or cancel and restart the current poll.
Useful? React with 👍 / 👎.
clawsweeper
Bot
added
rating: 🦐 gold shrimp
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Codex cost reports currently cover the machine running CodexBar. A Mac therefore cannot show token estimates from a Linux/SSH host. This adds explicit per-host reports to the Codex menu and Usage & Spend, plus
codexbar cost --provider codex --remote my-server.Related to #3036. Ready for maintainer review. Feature-scope sign-off under
VISION.mdis still required before merge: include opt-in native polling in the first version, or land the one-shot CLI contract first. The current implementation includes native polling so maintainers can review the complete proposed behavior.--summary-only --format jsonmode uses the existing cost scanner and sends numeric summaries, dates, time zone, provenance, and coverage. Native Codex history only; no transcripts, paths, session IDs, or account metadata are transferred.Both ends need a CLI with summary support. An older/unavailable remote produces its own error while preserving successful reports. The existing cost JSON schema remains unchanged unless one of the new modes is explicitly selected.
Validation:
4cfb60692bdd8cfbba7a491e94bb121b59a835a5; resolved the exact-line architecture catalog conflict without relaxing its checks and moved the changelog entry to 0.59.1 Unreleased.make check(zero violations) andCODEXBAR_TEST_GROUP_SIZE=48 make test(1,070 selections / 23 groups; zero failures, retries, or timeouts).722cf1ca7; the real SSH and native-control evidence below remains applicable.make check(zero violations).CODEXBAR_TEST_GROUP_SIZE=48 make test— all 1,060 selections / 23 groups passed on the first attempt; zero retries or timeouts.swift build --product CodexBarCLIin the official Swift 6.2.3 / Ubuntu Noble container.Real SSH transport output after the fix (2026-09-10)
Verified source:
722cf1ca7d77dd96caa1cca8714b4f61a2ba347d. The Mac CLI calls the productionRemoteCodexCostFetcherand system SSH. The DGX runs the same updated CLI source on Linux ARM64, built with Swift 6.2.3 / Ubuntu Noble. Both scanners read actual Codex history; there is no injected SSH runner or loader in this check.Command (
my-serversubstitutes for the private SSH alias):The first invocation used the development CLI remotely and exited 0. The second used the restored remote v0.56.8 CLI and exited 1. Assertions verified two separate successful reports, positive numeric token/cost values, no aggregate, and successful local output retained when the remote failed.
Host labels are substituted and the four token/cost values per summary are redacted below. Other stdout fields are unchanged; the redaction strings are not the transport's actual numeric types.
{ "success": { "exitCode": 0, "stdout": [ { "summary": { "provenance": "listPriceEstimate", "provider": "codex", "historyCoverageIsEstablished": false, "historyDays": 1, "sessionTokens": "<redacted number>", "coverage": { "unmetered": 0, "priced": 1, "unpriced": 0, "estimated": 0 }, "sessionCostUSD": "<redacted number>", "currencyCode": "USD", "last30DaysTokens": "<redacted number>", "updatedAt": "2026-09-10T11:39:01Z", "bucketTimeZone": "Europe/Rome", "last30DaysCostUSD": "<redacted number>" }, "host": "local-mac", "source": "local" }, { "summary": { "provenance": "listPriceEstimate", "provider": "codex", "historyDays": 1, "historyCoverageIsEstablished": true, "sessionTokens": "<redacted number>", "coverage": { "unpriced": 0, "unmetered": 0, "estimated": 0, "priced": 1 }, "sessionCostUSD": "<redacted number>", "currencyCode": "USD", "last30DaysTokens": "<redacted number>", "bucketTimeZone": "Etc/UTC", "updatedAt": "2026-09-10T11:39:41Z", "last30DaysCostUSD": "<redacted number>" }, "host": "dgx-spark", "source": "ssh" } ] }, "unsupportedRemote": { "exitCode": 1, "stdout": [ { "source": "local", "summary": { "sessionCostUSD": "<redacted number>", "sessionTokens": "<redacted number>", "last30DaysCostUSD": "<redacted number>", "historyCoverageIsEstablished": false, "coverage": { "unpriced": 0, "unmetered": 0, "priced": 1, "estimated": 0 }, "provider": "codex", "last30DaysTokens": "<redacted number>", "bucketTimeZone": "Europe/Rome", "historyDays": 1, "currencyCode": "USD", "updatedAt": "2026-09-10T11:39:42Z", "provenance": "listPriceEstimate" }, "host": "local-mac" }, { "source": "ssh", "error": "Could not read remote costs. Check SSH and that the remote CodexBar CLI supports --summary-only.", "host": "dgx-spark" } ] } }Captured verification output:
Fresh and upgraded preference initialization stays opt-in
The existing native settings test now initializes the production
SettingsStorefrom dictionary-backed defaults for both cases: empty preferences, and legacy preferences with agent sessions enabled and an existing SSH host. This tests preference loading in isolation; it is not a claim of an OS-level installation or upgrade test.In both cases the new cost-host setting remains empty, refreshing it (including force refresh) starts no task and leaves no reports, and the legacy session setting is preserved. The existing native UI screenshot above uses a synthetic loader and demonstrates presentation and Apply/disconnect interactions only; transport proof is the real SSH output above.
Runnable focused checks:
Native Apply regression check on the freshly rebuilt isolated proof app (production editor and store, synthetic cost loader): entering
Alice@server, alice@server, Alice@SERVERand pressing Return leaves both distinct accounts visible. The test app's settings/report receipt was:{"hosts":"Alice@server, alice@server","reports":"2","errors":"0"}