Skip to content

pack:verify should render an MCP App from the installed tarball, not just assert sandbox_proxy.html exists #2003

Description

@cliffhall

No check covers the intersection that #1859 fell through, and the two that come closest each miss it by one axis:

Check Runs against Drives the App path?
smoke:web:app the repo build tree yes — connect → open app → data-app-status="ready"
pack:verify the installed tarball no — only GET / on the prod web boot

So a packaging regression that keeps the file present-but-unreachable (wrong relative position, a rename with a stale reader, a future .npmignore/allowlist edit) passes both: the repo smoke never installs, and pack:verify only asserts the path exists (scripts/pack-and-verify.mjs:179 tarball, :244 on disk) without ever loading a widget through it. smoke-web-app.mjs's own header already names this as the open seam — "pack:verify proves the file ships, this proves the App path works. Neither subsumes the other."

Verified manually against the v2.2.0 tarball (#1993): packed, installed into a throwaway consumer, ran the installed mcp-inspector --web, deep-linked into mcp_app_demo, and the widget reached data-app-status="ready" with mcp-app-demo initialized in App logs. That manual pass is the thing to automate.

Scope

Add a step to pack:verify, after the existing prod --web boot, that drives the App flow against the installed bin:

  1. Spawn test-servers/build/server-composable.js --config test-servers/configs/mcp-app-http.json and parse the announced URL from stderr (console.error) — the port is resolved by findAvailablePort() and is not necessarily the config's 3130.
  2. Navigate the same deep link smoke-web-app.mjs uses (serverUrl + transport=http + autoConnect=<token> + openApp=mcp_app_demo + appArgs + autoOpen=<token>).
  3. Assert [data-testid="apps-form"][data-app-status="ready"], the contract in clients/web/README.md.

Notes for whoever picks this up

  • Most of the logic already exists in scripts/smoke-web-app.mjs. Prefer extracting its drive-the-App-flow half into scripts/lib/ and calling it from both, over a second copy — the two would drift, and the deep-link shape is the part that silently rots.
  • The test server stays a repo fixture. It is not in the tarball and should not be; only the client under test comes from the install.
  • Playwright resolution: pack:verify runs from the repo root, so use the createRequire based at clients/web/package.json that both browser smokes use — a bare import("playwright") resolves relative to scripts/.
  • Cost: pack:verify is already network-bound (a real npm install) and local/release-only, not part of validate/ci, so a Chromium run is affordable here.
  • Teardown must stop the App server on every failure path — see Three small cleanups flagged in the v2.2.0 merge review #2000 item 3, the same orphan bug in the existing smoke.

Metadata

Metadata

Assignees

No one assigned

    Labels

    choreMaintenance: deps, build tooling, CI, cleanup — no user-facing behavior changev2Issues and PRs for v2

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions