Skip to content

fix(antigravity): read sidecar-less WAL conversations with an immutable fallback - #3532

Merged
steipete merged 6 commits into
steipete:mainfrom
urda:urda/agy-immutable-fallback
Sep 11, 2026
Merged

fix(antigravity): read sidecar-less WAL conversations with an immutable fallback#3532
steipete merged 6 commits into
steipete:mainfrom
urda:urda/agy-immutable-fallback

Conversation

@urda

@urda urda commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

Summary

One cleanly closed Antigravity conversation makes the whole Antigravity spend history unavailable on macOS. The Usage & Spend pane then lists no Antigravity source and shows "Refresh failures: 1", and codexbar cost --provider antigravity returns an empty report with historyCoverageIsEstablished: false.

The cause is a SQLite platform behavior. When SQLite closes the last connection to a WAL database cleanly, it deletes the -wal and -shm sidecars. Apple's system SQLite (3.51.0 on macOS 26) then declines a read-only open of that file with SQLITE_CANTOPEN, because a read-only connection may not create the sidecars. The reader marks that database incomplete, the report becomes partial, and the fetcher withholds every row by design.

This PR adds a narrow fallback in AntigravityLocalReader.readDatabase:

  • When the ordinary read-only attempt fails with SQLITE_CANTOPEN before any row is read, and no -wal sidecar exists next to the database, the reader retries that one database with file:<path>?immutable=1 and SQLITE_OPEN_READONLY | SQLITE_OPEN_URI.
  • An absent -wal means no WAL connection holds the database, so the main file alone carries the checkpointed state. A database with a -wal sidecar present stays unavailable, as before.
  • The immutable open never creates or touches sidecars. The path is percent-encoded so %, ?, and # cannot change the URI.
  • An immutable connection neither locks nor detects changes. The reader therefore records the file's size, modification time, file system number, and 100-byte header before the retry, and accepts the result only when they and the sidecar state are unchanged afterwards. A writer that appears and checkpoints during the retry leaves that database incomplete, which keeps the report partial and withheld, as before.
  • Statistics.immutableFallbacks counts the retries.

The #3212 WAL test carried the comment "Some SQLite builds decline read-only WAL access without sidecars; that must stay unavailable." This PR argues against that ruling with field data: every new conversation lacks sidecars until some later bulk event recreates them, so the current policy blanks Antigravity spend for most macOS users most of the time.

Evidence

On the reporting machine, 214 conversation databases exist. 213 have sidecars and open read-only. One has a WAL header (writer version 2, read version 2) and no sidecars, and it fails:

$ sqlite3 -readonly 31125604-ffff-41db-b409-11d11a858825.db 'select 1'
Error: in prepare, unable to open database file (14)

A one-off Swift probe that links the system SQLite3 library reproduces rc=14 at prepare after a successful open and BEGIN DEFERRED. Python's bundled SQLite 3.53.4 opens the same file and creates the sidecars. Sidecar birth times on the 213 databases cluster on five moments (184 of them on one day), not on each conversation's creation, so new conversations regularly lack sidecars.

With this change, the same CLI command on the same machine returns 38 daily rows from 2026-06-20 through 2026-09-06 with historyCoverageIsEstablished: true, and the file still has no sidecars afterwards.

Trade-off

An immutable connection skips locking and change detection, so the read alone cannot prove one snapshot. The stability bracket above turns that into a fail-closed check: any change to the main file or the appearance of a -wal sidecar during the retry marks the database incomplete. The next scan reads the stable file. A temp copy would carry the same window and cost I/O on every refresh.

Tests

All three cases keep the existing raw-SQLite control pattern: a separate control fixture reports which outcome the platform's SQLite produced, and the expectations follow that outcome. On Apple's SQLite the control returns SQLITE_CANTOPEN and the fallback path runs; on a SQLite that creates the sidecars itself, the ordinary path runs and no fallback is counted.

  • AntigravityLocalWALTests: the existing sidecar-less case now expects .complete coverage and the fixture row on every platform, immutableFallbacks equal to 1 only when the control declined, and no sidecars afterwards in that case.
  • New: a sidecar-less database whose file name contains %, ?, and # reads completely through the escaped URI.
  • New: a database with an unreadable empty -wal sidecar never takes the fallback. Apple's SQLite declines that open, so the guard is exercised on macOS and the report stays partial. A root CI user can still read the file, and the control accepts that outcome.
  • New: a writer that reopens the database, commits a row, checkpoints, and closes while the reader is inside the immutable retry. On Apple's SQLite the fallback runs and the result is incomplete; the next scan is complete with both rows. On a SQLite that takes the ordinary path, the read-only snapshot excludes the later write as in the existing coordinated-writer test.
  • New: an unchanged two-row sidecar-less database reads completely through the bracket.

Commands run

swift test --filter AntigravityLocalWALTests
make check
make test
.build/debug/CodexBarCLI cost --provider antigravity --format json --days 365

Docs

docs/antigravity.md describes the fallback and its guard under "Local token history". No CHANGELOG entry, per repository convention.

@clawsweeper

clawsweeper Bot commented Sep 9, 2026

Copy link
Copy Markdown

🦞👀
ClawSweeper picked this up.

Pull request received. I will update this pull request when review starts.

ClawSweeper review complete

ClawSweeper finished reviewing this revision. The review result is being finalized.

View the workflow run.

@urda
urda marked this pull request as ready for review September 9, 2026 23:53
@clawsweeper clawsweeper Bot added P2 Normal priority bug or improvement with limited blast radius. merge-risk: 🚨 other 🚨 Merging this PR has meaningful risk outside the owned taxonomy. proof: sufficient Contributor real behavior proof is sufficient. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. labels Sep 9, 2026
@clawsweeper

clawsweeper Bot commented Sep 9, 2026

Copy link
Copy Markdown

Codex review: needs maintainer review before merge. Reviewed September 11, 2026, 5:23 PM ET / 21:23 UTC (Revision 7).

ClawSweeper review

What this changes

Adds a guarded immutable SQLite fallback for Antigravity conversations without WAL sidecars, regression coverage and documentation, plus a billing-test timing adjustment.

Merge readiness

Ready for maintainer review

The macOS history failure remains on current main and in v0.59.0. The earlier findings are resolved, the supplied real-run evidence supports recovery, and no actionable introduced defect was found.

Priority: P2
Reviewed head: 923d5ae792afbe4838367996f04a043c506529a4

Review scores

Measure Result What it means
Overall readiness 🐚 platinum hermit (4/6) A focused repair with relevant real-run evidence, regression coverage and resolved prior findings.
Proof confidence 🐚 platinum hermit (4/6) Sufficient (terminal): The supplied macOS run exercises the production cost CLI and Antigravity reader against 214 real conversation databases, reporting recovery to 38 daily rows with established coverage and no new sidecars; native SQLite regression tests supplement that evidence.
Patch quality 🐚 platinum hermit (4/6) No actionable review findings were identified.

Verification

Check Result Evidence
Real behavior Verified Sufficient (terminal): The supplied macOS run exercises the production cost CLI and Antigravity reader against 214 real conversation databases, reporting recovery to 38 daily rows with established coverage and no new sidecars; native SQLite regression tests supplement that evidence.
Evidence reviewed 9 items Verified head and effective scope: The raw head records current main ad8b174 as its first parent and the previously reviewed branch as its second parent. The 17-file supplied comparison includes main work carried into this merge.
Five-file change relative to current main: Only the Antigravity reader, scan statistics, WAL tests, Antigravity documentation and one OpenCodeGo test differ from current main; the account, menu and Grok changes are already present there.
Current-main failure path: Current main uses ordinary SQLITE_OPEN_READONLY and returns incomplete on failed schema access, without the proposed retry.
Findings None None.
Security None None.

How this fits together

CodexBar reads Antigravity’s local conversation databases to build daily token history for the CLI and Usage & Spend views. Incomplete database scans prevent publication of the entire report.

flowchart TD
  A[Local conversation databases] --> B[Normal read-only SQLite scan]
  B --> C{Cannot open and no WAL sidecar?}
  C -->|Yes| D[Immutable read with stability checks]
  C -->|No| E[Existing scan result]
  D --> F{Complete and stable?}
  E --> F
  F -->|Yes| G[Publish daily token history]
  F -->|No| H[Withhold incomplete report]
Loading

Before merge

None.

Agent review details

Security

None.

Review metrics

Metric Value Why it matters
Production and test growth Production +117/-13 (net +104); tests +134/-2 (net +132) Relative to current main, production growth implements guarded retry and cleanup, with focused regression coverage.

Technical review

Best possible solution:

Recover stable, checkpointed conversation history while preserving ordinary WAL coordination and withholding any incomplete or changing read.

Do we have a high-confidence way to reproduce the issue?

Yes: the reported macOS SQLite error maps directly to current main’s incomplete-report path for a cleanly closed WAL database. This read-only review inspected source and supplied evidence without executing tests or live probes.

Is this the best way to solve the issue?

Yes: the retry is narrowly limited to pre-row open failures, retains existing validation and budgets, and rejects results when the database changes; the earlier snapshot and documentation concerns are addressed.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning medium; reviewed against ad8b1745257e.

Labels

Label justifications:

  • P2: Repairs unavailable Antigravity token history on affected macOS SQLite builds without changing other providers’ production behavior.
  • rating: 🐚 platinum hermit: Overall readiness is 🐚 platinum hermit; proof is 🐚 platinum hermit and patch quality is 🐚 platinum hermit.
  • status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR. Sufficient (terminal): The supplied macOS run exercises the production cost CLI and Antigravity reader against 214 real conversation databases, reporting recovery to 38 daily rows with established coverage and no new sidecars; native SQLite regression tests supplement that evidence.
  • proof: sufficient: Contributor real behavior proof is sufficient. The supplied macOS run exercises the production cost CLI and Antigravity reader against 214 real conversation databases, reporting recovery to 38 daily rows with established coverage and no new sidecars; native SQLite regression tests supplement that evidence.

Evidence

What I checked:

Likely related people:

  • unknown: The claimed source-line change could not be verified from bounded local history. (role: source history unknown; confidence: low)
  • Yuxin-Qiao: Suggested for follow-up; no historical authorship or introduction is verified. (role: unverified routing candidate; confidence: low)

Rating scale

Score Internal tier Crab rank Meaning
6/6 S 🦀 challenger crab Exceptional readiness
5/6 A 🦞 diamond lobster Very strong readiness
4/6 B 🐚 platinum hermit Good normal PR; ordinary maintainer review
3/6 C 🦐 gold shrimp Useful, but confidence is limited
2/6 D 🦪 silver shellfish Proof or implementation needs work
1/6 F 🧂 unranked krab Not merge-ready
N/A NA 🌊 off-meta tidepool Rating does not apply

Overall follows the weaker of proof and patch quality.
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

Workflow

  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

History

Review history (6 earlier review cycles)
  • reviewed 2026-09-09T23:56:05.028Z sha 895a9ba :: blocked before merge. :: [P2] Preserve a stable snapshot throughout the fallback read
  • reviewed 2026-09-10T00:44:49.212Z sha 476c332 :: needs maintainer review before merge. :: none
  • reviewed 2026-09-10T00:48:36.666Z sha 476c332 :: needs maintainer review before merge. :: none
  • reviewed 2026-09-10T14:16:22.619Z sha 3ea3844 :: needs maintainer review before merge. :: none
  • reviewed 2026-09-10T15:18:30.794Z sha a95e2b8 :: needs maintainer review before merge. :: none
  • reviewed 2026-09-11T19:30:58.887Z sha 68eb982 :: needs maintainer review before merge. :: none

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 895a9ba145

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread Sources/CodexBarCore/Providers/Antigravity/AntigravityLocalSQLite.swift Outdated
Comment thread docs/antigravity.md Outdated
@urda

urda commented Sep 10, 2026

Copy link
Copy Markdown
Contributor Author

@codex review

@urda

urda commented Sep 10, 2026

Copy link
Copy Markdown
Contributor Author

@clawsweeper re-review

@clawsweeper

clawsweeper Bot commented Sep 10, 2026

Copy link
Copy Markdown

🦞🧹
ClawSweeper re-review requested.

I asked ClawSweeper to review this item again.
Action: item re-review queued (workflow sweep.yml, event exact_review_queue).
Result: when the review finishes, ClawSweeper will create the durable review comment if needed or update the existing comment in place.

@clawsweeper clawsweeper Bot added rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. and removed status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. merge-risk: 🚨 other 🚨 Merging this PR has meaningful risk outside the owned taxonomy. labels Sep 10, 2026
@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. 🚀

Reviewed commit: 476c3322d4

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

@urda
urda force-pushed the urda/agy-immutable-fallback branch from 476c332 to 3ea3844 Compare September 10, 2026 14:12
@urda
urda force-pushed the urda/agy-immutable-fallback branch from a95e2b8 to 68eb982 Compare September 11, 2026 19:25
Route System rows through the existing system Codex login even when they retain a managed storage ID. Refresh the projection and reject stale source, slot or workspace selections before login. Share action exclusion and show live reauthentication progress.

Fixes steipete#3558. Thanks @Nek-12 for the credential-source comparison. No credential copying, automatic renewal, profile-home changes or projection preference changes. Production code is net five lines smaller.
Normalize primary-first Cocoa screen frames into Quartz window coordinates before containment and blocked-proxy detection. Use the CoreGraphics rectangle decoder after validating numeric fields.

Covers displays above, below and offset from the primary plus actual startup recovery guards. Removes eleven production lines. Related to steipete#3377 and steipete#3355 without claiming their broader symptoms are resolved.

Includes the upper-display primary-origin proxy regression from review while preserving interior/top-edge containment safeguards.
…#3562)

Check existing CLI billing acceptance before scanning local history, probing the version or fetching settings. Consolidate successful and eligible Team identity-only results while preserving errors, credentials, public APIs and legacy history semantics.

Extracts the lazy-scan observation from steipete#3236 without its new history format or accounting scope. Reduces production code by four lines.

Co-authored-by: Yuxin Qiao <104957188+Yuxin-Qiao@users.noreply.github.com>

Preserves the original post-scan/version eligibility checkpoint and identity admitted before settings enrichment.
@steipete

Copy link
Copy Markdown
Owner

Verified the main-synchronized head 923d5ae792afbe4838367996f04a043c506529a4.

The real built CLI reproduces the repair with a synthetic, sidecar-less WAL database whose filename also contains %, ?, and #: installed v0.59.0 returns no rows and historyCoverageIsEstablished: false; this head returns the expected 198 tokens and established coverage. The database hash is unchanged, and neither WAL nor SHM sidecars are created. No provider credentials or real account history were used.

A separate Developer ID–signed debug proof bundle from this head launched successfully. Peekaboo verified the synthetic status item on-screen, opened its menu, selected Cached, and observed the expected template-image renderer state before quitting it. This is native launch/interaction smoke; the CLI/database comparison above is the bug-specific proof.

make check passed with zero SwiftLint violations. Independent branch review is clean at P0–P2. Exact-head CI passed, including both macOS test shards and the Linux jobs.

Local full-suite limitation: the first attempt stopped on the unrelated Gemini helper's two-second success-case timeout; its isolated 20-test suite then passed. A fresh-private-TMPDIR full retry reached a 180-second CostUsageBoundedProgressTests fixture-seeding timeout; sampling showed file creation/open/write work. The full-suite pass is the exact-head CI run above. No tests or assertions were removed or relaxed.

Contributor credit remains intact. The changelog entry is centralized in #3564, which should merge after this PR.

@steipete
steipete merged commit e2f0b0b into steipete:main Sep 11, 2026
9 checks passed
steipete added a commit that referenced this pull request Sep 11, 2026
Update SwiftLog to 1.15.1, SwiftLint to 0.65.1, Oxlint to 1.82.0, and Oxfmt to 0.67.0 with matching lockfile and verified platform digests. Preserve the existing formatting and lint rules.

Collect the Antigravity fix note and contributor credit with the existing 0.59.1 maintenance notes, ordered by user impact. Merge after #3532; released sections and release artifacts are unchanged.
@urda
urda deleted the urda/agy-immutable-fallback branch September 12, 2026 00:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

P2 Normal priority bug or improvement with limited blast radius. proof: sufficient Contributor real behavior proof is sufficient. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants