refactor(icp): move identity loading and manifest parsing into icp-cli

[raymondk]

Stacked PR. Base is fm/icp-split-inc1-events (#709), not main. Review this one on top of that.

Intent

Move identity loading and manifest parsing out of the icp library crate and into icp-cli.

The architecture this serves: icp-cli is the frontend/UX for the CLI; icp is the library, and it
works with an agent. Loading an identity is a CLI concern, parsing a manifest is a CLI concern,
and the library should touch the filesystem only for building and deploying. The library does not
stop using the filesystem by pretending — it uses the port/abstraction pattern already established
in icp::context::Context, which is entirely Arc<dyn …> today.

Four required pieces:

1. Move crates/icp/src/identity (~3,347 LOC) into icp-cli. Outside the module and the Context
   wiring there were exactly two genuine couplings — telemetry_data.rs's use of
   identity::manifest::IdentitySpec and directories.rs's use of
   identity::{IdentityDirectories, IdentityPaths} — and a home had to be chosen for those two
   types and justified. Three other apparent references were false positives to verify, not touch:
   network/managed/run.rs and network/access.rs use ic_agent's own AnonymousIdentity, and
   network/custom_domains.rs merely names the Internet Identity canister ID. The library should
   end up receiving a constructed Agent.
2. Move manifest parsing to icp-cli. Project and the ProjectLoad trait stay in icp — with
   deploy in the library, a consumer must still be able to receive a project; a library that can
   deploy but cannot represent a project is the failure mode to avoid. What moves is the native,
   filesystem-touching loading implementation behind the existing port.
3. Delete Context.password_func entirely (26 of its 37 references were inside
   crates/icp/src/identity). Explicitly no Prompt port: that was considered and ruled out,
   because with identity in the CLI it is unnecessary abstraction.
4. Delete Context.debug — a presentation flag has no place in a library.

Out of scope by instruction: moving operations/, starting the larger crate split, and adding any
wasm target, cfg gate or wasm CI job.

What Changed

Identity → icp-cli (crates/icp-cli/src/identity/, moved with git mv; the module bodies are
unchanged apart from crate:: → icp:: path rewrites). With it went the two Context fields that
only served it, and the selection-resolving helpers built on them:

• password_func is deleted from the library. The CLI's context owns the password reader now
  (commands/identity/reauth.rs is its one non-loader consumer).
• identity: Arc<dyn identity::Load> is deleted, along with get_identity, get_agent, and
  get_agent_for_{env,network,url}. Those take an IdentitySelection — a CLI concept — and must
  interleave identity loading with root-key resolution, because a delegated identity is validated
  against the network's root key. They now live on the CLI's context. What stays in the library is
  Context::create_agent: identity + resolved network → Agent. The agent::Create port has to
  stay regardless, since network::Accessor builds an anonymous agent to fetch root keys.
• Context.debug is deleted; --debug is presentation (tracing layer, progress-bar hiding).

New icp_cli::context::Context wraps the library context and Derefs to it, carrying the
frontend-only state (debug, password_func, the identity loader). Because it derefs, every library
port (dirs, ids, project, network, …) is still reached straight through ctx, ctx.debug
and ctx.get_agent(...) call sites are unchanged, and operations/ keeps taking
&icp::context::Context via deref coercion — which also kept this out of the way of the concurrent
operations/ work. The identity loader is a single shared instance, so its per-selection cache still
unlocks an encrypted identity (and asks for its password) at most once per invocation.

The two couplings, decided:

• IdentityStorageType stays in icp::telemetry_data — telemetry is a library-side data bag
  hanging off Context, written by subsystems. Its From<&IdentitySpec> conversion travels with
  IdentitySpec into icp-cli (orphan rule is satisfied: &IdentitySpec is a local, fundamental
  type).
• IdentityPaths/IdentityDirectories travel with identity, and
  directories::Access::identity() becomes identity_dir() -> PathBuf. The generic
  directory-lock machinery (icp::fs::lock) stays put — it is filesystem infrastructure, not
  identity, and settings and the package cache use it the same way. The CLI re-locks the directory
  through Context::identity_dirs().

Manifest parsing → icp-cli: ProjectRootLocateImpl and the YAML loader
(crates/icp-cli/src/manifest.rs), consolidation plus ProjectLoadImpl and Lazy
(crates/icp-cli/src/project.rs). Project, ProjectLoad, ProjectRootLocate and the manifest
shape types stay in icp: the shapes are plain serde with no filesystem in them, BuildSteps and
RootKeySpec are part of the Project model the builder consumes, and schema-gen generates the
published JSON schemas from them. Their modules widened from pub(crate) to pub so the loader can
reach the same types it always did (not a semver concern — the crate is publish = false).

ProjectLoadError's two loader-specific variants collapse into a single boxed
#[snafu(transparent)] Load variant, so the port does not name any one loader's error type; the CLI
loader keeps variants with the same messages. Locate stays a real variant because callers match on
it to detect "no project here". Error output is unchanged — see Testing.

icp::context::initialize now receives the directories, the root locator and the project loader
instead of constructing them (the $PWD-vs-getcwd() resolution that feeds the locator moved to the
CLI with it), and since it no longer locks an identity directory it can no longer fail: it returns a
Context and ContextInitError left the library. DEFAULT_LOCAL_NETWORK_{BIND,PORT} moved to
icp::network, where the default network they describe is defined.

Tests split along the same line: of the 31 Context tests, 14 stayed in icp and 17 moved to
crates/icp-cli/src/context/tests.rs. So the CLI can build a mocked library context, icp's port
mocks are now compiled under #[cfg(any(test, feature = "mocks"))] and icp-cli enables that
feature as a dev-dependency (resolver 3 keeps it out of cargo build --bin icp).

Four items that were already dead surfaced once the module landed in a binary crate, where pub no
longer suppresses dead-code analysis, and are removed: load_identity_in_context,
IdentityPaths::ensure_identity_{defaults,list}_path, WriteIdentityError::CreateDirectoryError,
MockIdentityLoader::with_default, plus the never-constructed ConsolidateManifestError::Locate.

Agent-facing docs record the boundary: a new "Crate Boundary" section in .claude/architecture.md,
plus updates to .claude/CLAUDE.md and .claude/testing.md.

Measured payoff

	before	after
filesystem call sites in crates/icp/src	241	137
icp dependencies	74	58

grep -rn 'fs::\|std::fs\|File::\|read_to_string\|create_dir_all\|\.exists()\|remove_file\|remove_dir' \
  crates/icp/src --include=*.rs | wc -l

Dependencies that left icp entirely: bip32, crypto-bigint, elliptic-curve, hmac,
ic-ed25519, ic-identity-hsm, k256, keyring, p256, pem, pkcs8, rand, scrypt, sec1,
tiny-bip39, zeroize. pkcs8, pem, sec1, bip32 and hmac had zero users outside identity as
expected. keyring had one: context/init.rs's ICP_CLI_KEYRING_MOCK_DIR hook, which moved to the
CLI's initialize alongside the loader it configures — so keyring leaves too.

Risk Assessment

✅ Low. This is a mechanical relocation: the moved module bodies and the moved consolidation logic are
unchanged apart from import paths, and no Project, manifest or identity semantics were touched. The
two behavioral surfaces that could have drifted silently were checked directly: the password prompt
(the loader's caching is what keeps it to one prompt per invocation) and the manifest error chain
(where a new boxed transparent variant sits in the path). Both are covered below, the latter now by a
new test. The one identity form not exercised locally is HSM linking — SoftHSM2 is not installed on
the dev machine — and CI provisions SoftHSM2 on all three platforms, so the five unconditional
identity_link_hsm* tests cover it here.

Testing

cargo build, cargo fmt --all -- --check, cargo clippy --tests --benches -- -D warnings and
cargo clippy --all-targets --all-features all clean. Both doc-drift generators
(./scripts/generate-cli-docs.sh, ./scripts/generate-config-schemas.sh) leave the worktree
byte-identical, so the cli-ref and icp-yaml-schema CI jobs have nothing to report.

Automated: cargo test -p icp --lib 173 pass; cargo test -p icp-cli --bin icp 148 pass, including
the 17 relocated identity/agent context tests; whole-workspace cargo test --no-fail-fast passes
every target. Three initially-failing tests were environmental and each was chased down:
canister_install_large_wasm_chunked needed wasm-tools (installed → passes), and two
network_tests port-conflict tests hit an unauthenticated GitHub API 403 resolving the launcher
version (pass with ICP_CLI_GITHUB_TOKEN, as CI sets). The remaining local failures are the five
identity_link_hsm* tests (no SoftHSM2) and two canister_snapshot_*_resume tests (no mitmproxy) —
both provisioned in CI.

Identity behavior specifically: identity_tests 31 pass, including identity_storage_forms,
pem_session_delegation_avoids_second_password_prompt, pem_explicit_login_creates_session and
pem_session_migrated_on_rename. Beyond that, the validation pipeline drove the interactive
prompt over a real pty (correct password prints the PEM; a wrong one gives failed to decrypt PEM file) and confirmed session reuse by passing an empty password file on a second call, plus a full
icp identity new/list/default/principal/account-id/rename/export transcript against an isolated
ICP_HOME.

Manifest errors: compared by hand and now locked in by a new test,
malformed_manifest_reports_the_whole_load_chain in crates/icp-cli/tests/project_tests.rs, which
runs a real command against a malformed icp.yaml and asserts stderr carries both levels the new
indirection could have swallowed:

Error: failed to load project
Caused by:
    0: failed to load project manifest
    1: failed to parse manifest at '<path>/icp.yaml'
    2: did not find expected ',' or ']' at line 3 column 1, ...

and, for the Locate path (already covered by network_ping_tests/network_status_tests):

Error: failed to load project
Caused by:
    0: failed to locate project directory
    1: project manifest not found in <dir>

The error chain is preserved by construction, not by luck: snafu-derive 0.9.1's transparent
delegates Display to the source and returns source.source() from Error::source(), so the boxed
ProjectLoadError::Load(NativeProjectLoadError::ProjectManifest) renders exactly the two levels the
old ProjectManifest variant did.

Pipeline

Validated through no-mistakes (review → test → document → lint → push). Review closed with no
findings after one fix round; that round added the manifest-error-chain test above, which the
original change was missing. Two informational lint items are recorded as out of scope: 12
pre-existing rustdoc diagnostics in files this change does not touch (CI does not run rustdoc), and
taplo fmt --check not being runnable locally — the four changed TOML files were hand-checked
against taplo.toml, and the toml-fmt CI job is the real gate.

🤖 Generated with Claude Code

[adamspofford-dfinity]

I'm not sure I agree with this refactor, which shifts the icp-cli/icp split from 'anything that interacts with the terminal or user input' to 'anything that interacts with the filesystem'. The former split is what you would expect from a CLI wrapper <-> tool core; the current icp-cli contains everything that you'd have to replace if you wanted to make, for example, a GUI version. The latter split is what you would expect from a pure library published to crates.io, where icp-cli would contain everything that an arbitrary caller might not want. We don't plan to publish the full icp core, and almost anything of value outside of #713 to import if we were publishing things ends up on the wrong side of the split. We also don't currently plan to make a GUI version, though if I had to pick which of the two was more likely to change, it's that one. So the location of the split is a question of what's good for code discipline, instead of what's required for a product we're making. On the matter of code discipline, I believe that requiring factoring out all I/O from core code instead of just doing one's best to situate it at what feels like the right point in the call stack is not worth it, while keeping details of business logic out of icp-cli is.