feat(mcp): add CBM_IN_PROCESS to serve stdio MCP without the daemon

[pmaxvsbobo]

What does this PR do?

Adds an opt-in CBM_IN_PROCESS environment variable that serves an MCP session
in-process over stdio, without starting or connecting to the coordination daemon.

Why it's needed: on hosts whose sandbox denies socket syscalls, the daemon
handshake can never complete. A macOS seatbelt profile that permits filesystem
access but applies (deny network*) makes both bind() and connect() fail
with EPERM — AF_UNIX included — so there is no reachable rendezvous and no
alternative transport. Today that surfaces as a 30s hang and a misleading
"CBM daemon is active or starting" error;

The change is small because the server already existed: cbm_mcp_server_run()
is a complete socket-free stdio loop (zero socket/IPC references in
src/mcp/mcp.c) whose only caller was tests/test_mcp.c. This routes MCP
clients to it behind a gate, so default behaviour is unchanged.

Measured under an active sandbox: 31.1s failure → 4.6s initialize result.

Trade-off (documented in README.md and docs/CONFIGURATION.md): no daemon
means no cross-session coordination — no shared watchers, no shared indexing
jobs, no UI, no exact-build admission barrier.

Checklist

• Every commit is signed off (git commit -s) — required, CI rejects
  unsigned commits (DCO, see CONTRIBUTING.md)
• Tests pass locally (make -f Makefile.cbm test)
• Lint passes (make -f Makefile.cbm lint-ci)
• New behavior is covered by a test (reproduce-first for bug fixes)

[github-actions]

Thanks for opening this — it has been seen, and it is queued.

This note is automated, but it is not a brush-off: it exists so you know where your PR stands instead of having to guess from silence.

Current review status: working through a backlog. 0.9.1-rc.1 is out, so the release freeze that held reviews is over — but it left a large queue of open pull requests behind it, and we are reading through them oldest-first. The background is in discussion #1144.

What that means for this PR, concretely:

• It will not be closed for inactivity. No stale bot touches pull requests here.
• It may still sit a while before a human reads it. That is on us, not on you.
• Older PRs are read first, so a recent one is not being skipped — it is behind a queue.

Things that will genuinely speed it up whenever review does happen:

• Keep it rebased on main — the tree is moving quickly right now, and a conflicting branch cannot be reviewed as the diff you intended.
• Get CI green, or say which failures you believe are pre-existing.
• Keep the change to one claim. Bundled features and refactors get split before they get merged, which costs you a round trip.
• Every commit needs a sign-off (git commit -s) — CI enforces DCO.

If this fixes a bug, a reproduction we can run is worth more than a description of the symptom.

Thanks for contributing, and sorry in advance for the wait.

[DeusData]

Thanks for this, and for measuring it (31 s to 4.6 s is a real pain point, and the analysis of why the failure is misreported is right: the connect result carries no errno, so an EPERM connect is classified as a reserved endpoint and retried blindly). Before we decide on the direction I want to understand the use case better, because a daemon opt-out is trickier than the diff suggests: the daemon is where the cross-session mutation lease, the index executor, the project-mutation guard and the exact-build admission live, and the in-process server as wired here gets none of them while background tasks stay on, so an in-process index_repository writes the shared cache without the lease that protects a concurrent daemon session on the same machine. Could you tell us more about:

1. The environment. Which host, sandbox or agent product applies (deny network*) to the MCP server process, and is that its default profile or one you wrote? A concrete reproduction (profile snippet or product name plus version) would help us judge how common this is.
2. Why AF_UNIX cannot be allowed. Most seatbelt profiles that deny network still permit unix-domain sockets to a specific path ((allow network* (local unix-socket)) style rules, or a path-scoped allow). Is that ruled out where you are, and if so by whom?
3. What you need inside that sandbox. Is it read-only querying of an index built elsewhere, or does the session also have to index and re-index? If reads are enough, a read-only in-process mode sidesteps the lease problem entirely; if writes are needed, how would you propose to coordinate with a daemon session on the same cache?
4. Whether the real defect alone would do for now. If the connect failure surfaced immediately as "the sandbox denies unix sockets: …" instead of a 30 s "daemon is active or starting", would that unblock you while the design question is settled? That fix (errno propagation in the connect result) is small, clearly right, and we would take it as its own PR straight away.

No decision yet on the mode itself; the maintainer wants to think about it with your answers in hand. Thank you again for the careful write-up.

[pmaxvsbobo]

Thanks — you're right about the lease, and it was worse than "no lease is acquired". Fixed inc72cb8b; the mode is now read-only.
mcp_project_mutation_begin() is return !srv->mutation_begin || srv->mutation_begin(...), so an unset guard fails open and the mutation proceeds. I'd wired only set_tool_profile and set_session_context, left background_tasks at its default true, and installed no mutation guard, no try guard and no index executor. Every other construction site in the tree already gets this right — daemon/application.c, ui/http_server.c, and main.c's own local-CLI path in the same function I edited. I should have mirrored them.
I verified it rather than taking it on description: with the three calls removed, index_repository on an in-process session doesn't merely skip the guard, it returns {"status":"indexed"} with isError:false and writes the cache. So this was live, not theoretical. Thank you for catching it.

What c72cb8b does — mirroring ui/http_server.c:

• background_tasks(false), so maybe_auto_index() can't index on the initialize path
• a mutation guard that refuses, with the try guard deliberately left NULL: with mutation_begin set and mutation_try_begin NULL, mcp_project_mutation_try_begin() also returns false, so opportunistic writes during a read are refused too
• an index executor that rejects with a message naming the reason, rather than letting the guard report "blocked by an active index" when nothing is blocking
  The test now asserts both that the call is refused and that no .db appears under CBM_CACHE_DIR — an error message alone could be emitted after a partial write, and only the second check distinguishes a refusing guard from an absent one. I confirmed that assertion fails against a build with the three calls removed.

[pmaxvsbobo]

Answers to your questions:
1. The environment. An enterprise-managed build of Claude Code, deployed by my employer's
security org — not a profile I wrote, and not one I can change. It applies a seatbelt profile
to every subprocess it spawns, MCP servers included: (allow default) for the filesystem,
(deny network*), then four narrow re-allows. The profile file is regenerated from a compiled-in
constant at every launch and sha256-verified, so local edits are reverted; changing the policy
requires review by the team that owns it. I'd rather not paste my employer's profile wholesale
in a public issue, but I can share the exact deny/allow lines if that helps. The shape is:
filesystem unrestricted, all networking denied, a handful of vendor-owned control sockets
re-allowed by exact filename.

I can't tell you how common this is beyond my own org. But the pattern — sandbox the agent's
subprocesses, deny networking, leave the filesystem alone — isn't exotic, and CBM is unusually
exposed to it because coordination is mandatory and the only transport is AF_UNIX. #1533/#2023
(Windows app-container ACLs) and #1830 (Docker userns-remap) look like the same class arriving
through different mechanisms.

2. Why AF_UNIX can't be allowed. It's ruled out, by the team that owns the profile. Two specifics that matter:

• network* covers network-bind as well as network-outbound, so bind() and connect() are both EPERM for AF_UNIX. There's no connect-only loophole.
• The only unix-socket re-allows are (allow network-outbound (remote unix-socket (regex ...))) matching an exact filename pattern for the vendor's own control sockets. They're not directory-scoped, so no value of CBM_RUNTIME_DIR can produce a matching path — CBM hardcodes both the cbm-daemon-<uid> subdirectory and the cbm-<hash>.sock filename.
  Measured inside one sandboxed process: connect() to a live socket under $HOME → EPERM;
  connect() to the CBM socket under /private/tmp → EPERM; bind() in $HOME, /private/tmp
  and $TMPDIR → EPERM, 8/8. The loopback UI port is EPERM too.

There is an escape hatch — the vendor lets a user exempt a named binary from the sandbox entirely — but I don't think that's the better answer. It trades a scoped capability for removing containment altogether from a process that indexes source code, and it's a per-machine manual step the project can't rely on.

3. What I need inside the sandbox — reads only. This is the useful answer, and it's why your suggestion is simply better than what I sent. Querying an index built elsewhere is enough. Indexing already happens outside the sandbox, from an ordinary terminal where the daemon works normally and owns the watcher and the lease. The sandboxed session only needs search_graph, search_code, get_code_snippet, trace_path and friends against the cache the daemon built. So the read-only mode in c72cb8b is not a concession — it's what I actually needed, and it removes the coordination question rather than answering it. If you'd prefer this expressed through the existing --tool-profile machinery rather than a new env var, say so and I'll rework it; a read-only profile is arguably where this belongs.

4. Would the errno fix alone unblock me? Not on its own — an accurate instant error still leaves the tools unusable, so I'd still be reaching for a terminal. But it's worth far more than its size, and I'd rather it landed first regardless. It cost me most of a day: the 30 s wait lands just past the 30 s MCP handshake budget my host imposes, so all I saw was "connection timed out after 30000ms" — the actual error never arrived. And when it does arrive it actively misleads: the daemon was healthy throughout, and a running daemon is what makes the wait last the full 30 s, since holding the cohort lock is what makes presence read COORDINATED and the EPERM classify as RESERVED/waitable. daemon start made it worse, which is the opposite of what the message implies.

I'll send that as its own PR: an os_error field on cbm_daemon_runtime_connect_result_t, and a fast-fail for non-retryable values (EPERM, EACCES) in cbm_daemon_bootstrap_classify_failed_connect() with a message naming the cause. Independent of whatever you decide here, and it should help #1955 too — that report hits the same message, and it currently can't distinguish "denied" from "contended" from "still starting".

On CI: DCO is fixed and both commits are signed off. The clang-format failures in cypher.c, runtime.c, version_cohort.c, frontend.c and cli.c are pre-existing and not from this branch — this PR touches four files, none of them those. I only hit them locally because Homebrew ships clang-format 23 while you pin 20. The one violation that was mine (an over-wrapped fprintf in src/main.c) is fixed.

[pmaxvsbobo]

@DeusData I think this is needed for making codebase-memory-mcp more popular on enterprise env, can u pls prioritize reviewing it?

[DeusData]

Thank you for clarifying the read-only and sandbox use cases. Those answers address the use-case questions already raised. We need more time to decide the in-process/read-only execution contract before making an integration decision; there is no additional questionnaire or speculative rebase requested from you at this stage.

[DeusData]

Thank you, @pmaxvsbobo. We've made the decision: CBM_IN_PROCESS is accepted as a documented, opt-in escape hatch for read-only mounts and sandboxes where the shared daemon can't run. The daemon stays the default.

Two things before we review it for merge:

1. A test that the in-process mode never writes shared cache state. That means no daemon coordination files, no cohort or lifetime locks, and no writes to another process's index, so an opted-in client can't disturb a daemon running alongside it.
2. A short doc entry (README or the configuration docs) saying what the mode is for, that it opts out of the shared daemon (no cross-client sharing, one index per process), and when not to use it.

Thanks again for spelling out the use cases. They made the decision easy.

[pmaxvsbobo]

Written with Claude Code; I've reviewed it and I'm accountable for it.

Thanks, @DeusData. Both requests are done, and working on the first one turned up a real bug. On the current main, the mode did write shared state.

The bug. After merging main, 21ae021 has handle_index_repository calling load_index_policy() before the index executor gets a chance to refuse. An in-process server has no config, so that path runs cbm_config_open(), which creates ${CBM_CACHE_DIR}/_config.db (or opens the daemon's copy read-write). CI never caught it because no workflow ran the old scripts/test_mcp_in_process.py. That was my mistake.

Commits (on top of main):

1. fix(mcp): turn CBM_IN_PROCESS on only for 1 or true: previously CBM_IN_PROCESS=false turned it on. It now follows the CBM_DIAGNOSTICS convention.
2. test: prove CBM_IN_PROCESS writes no shared state: this test fails at this commit with FAIL: CBM_IN_PROCESS created shared state from a fresh start: ['cache/_config.db'].
3. fix(mcp): serve only the analysis/scout tool set under CBM_IN_PROCESS: this makes it pass. Behaviour change: in-process tools/list now serves the analysis set (or scout, if requested), so index_repository, delete_project, manage_adr and ingest_traces are no longer offered. dispatch_tool() refuses them before any handler runs, so pre-lease work like the policy load can't reach shared state. The refusing guard, rejecting executor and background_tasks=false stay in place as a fail-closed second line.
4. docs: document running CBM_IN_PROCESS without the daemon.

The test is tests/test_in_process_mcp.sh with a Python driver. It runs as Step 5g of scripts/test.sh, on the private runtime from scripts/test-runtime.sh, and has three legs:

• Fresh state, no daemon. Every mutating tool is attempted. Runtime, cache and HOME must still be empty afterwards.
• A live permanent daemon owning two real indexes. An in-process session runs every tool it serves (each must succeed, which proves it read the daemon's index) and attempts every mutation. Every entry must match a quiescent baseline: type, mode, size, mtime, inode and sha256. The live session also may hold no socket and no descriptor under CBM_RUNTIME_DIR. A pre-existing lock file shows no trace of being locked, so only the descriptor check can prove no cohort or lifetime lock was taken.
• The same after a daemon-side ADR write has put the index in WAL mode. Here the only permitted changes are SQLite's reader -shm and an empty -wal, which any reader creates, the daemon's own included. The .db must stay byte-identical.

A positive control run with CBM_IN_PROCESS=false must find the socket and the runtime lock.

Evidence (macOS 27.0 arm64, Python 3.9, git 2.54): bash tests/test_in_process_mcp.sh

build	result
commit 3	PASS (~32 s)
commit 2 (no clamp)	FAIL ['cache/_config.db']
mutant: in-process branch also calls cbm_version_cohort_acquire()	FAIL: leg 1 creates the cohort locks; with leg 1 disabled, the descriptor check finds cbm-version-cohort-lifetime-v1.lock
mutant: in-process branch utimes() each project .db	FAIL in leg 2, and again in leg 3 alone, so the WAL allowance hides no index write

The test hasn't been run on Linux; this CI run is its first. On Windows it's skipped, as test_watcher_disabled.sh is, because the descriptor checks need /proc or lsof.

Branch. I rebased the PR onto main as linear commits to fix committer metadata on the first two. Their trees are unchanged, and the old c72cb8b I referenced earlier is now 4e2124a0.

Read-only mounts aren't covered, and I'd like your call. Every CBM process, this mode included, runs main_build_identity() before the in-process branch. That fchmod(0700)s CBM_CACHE_DIR unconditionally (src/daemon/ipc.c), so on a read-only filesystem it fails with EROFS (cache-private). The docs say so. If you want read-only mounts supported, I'd suggest a verify-only path for in-process sessions: refuse unless the cache is already owner-only, and never chmod it. That could go here or in a separate PR.

The errno PR (fast-failing EPERM/EACCES from connect() instead of retrying for 30 s) is still coming as its own PR.

[DeusData]

Thank you, @pmaxvsbobo. This is a far stronger answer than we asked for. Writing the test first and catching the _config.db creation on current main is exactly why we asked for a test rather than an argument. Clamping the tool set so the refusal happens before any handler runs is the right fix, and the positive control plus the mutants you ran show the test actually binds. The docs section is clear about what the mode gives up.

Two things before merge:

1. Read-only mounts: please do it here. You're right that main_build_identity() chmods CBM_CACHE_DIR before the in-process branch, which fails with EROFS. We'd like your verify-only path in this PR: in in-process mode, refuse unless the cache directory is already owner-only, and never chmod it. Please add a test for a read-only cache directory as well, since read-only mounts and sandboxes are the reason this mode exists.
2. The settle loop in the test. quiescent_snapshot() waits for two identical snapshots 0.5 s apart. That's a settle window, so the verdict still depends on timing: a daemon write landing just after the window would fail the leg for reasons unrelated to in-process mode. We ask tests to wait for a state they can observe instead. Could the baseline be taken once the daemon is idle by construction, for example with the watcher off via config (as tests/test_watcher_disabled.sh does) and the indexes already published when cli index_repository returns? If you saw a specific late daemon write that made the settle loop necessary, tell us what it was and we'll wait on that exact signal instead. (Small nit: if daemon stop already blocks until the daemon exits, the stop poll at the end can go.)

Thanks again. Your testing turned up a bug we would otherwise have shipped.