feat(ios): appstate reads the session app's XCUIApplication state through the runner by okwasniewski · Pull Request #2929 · callstack/agent-device

okwasniewski · 2026-09-24T17:49:40Z

Summary

iOS appstate answered from the session record alone, so after home it still named the app as if in front. The session app's own state is a fact a live runner has.

A new appState runner command reads XCUIApplication(bundleIdentifier:).state and names it (runningForeground, runningBackground, runningBackgroundSuspended, notRunning, unknown), declared once in the kernel as APPLE_APPLICATION_STATES. A lifecycle read like status, it skips the activation preflight.

The Apple runtime binds the read (bindAppleAppStateRuntime) and never starts a runner for it: with no live runner session it answers nothing and the handler keeps source: session with no state, so appState stays on the simulator host in the runner-demand table. With a live runner the handler adds state and source: runner. A cancelled request stays cancelled. The MCP output schema carries both source values and the state enum. The CLI's Apple lines read Session app:. Which app took the screen is still not read.

Validation

Tested at the pushed head, rebased onto 6428c54. pnpm check:xctest-selection, pnpm check:packaged-runner-swift, and pnpm check:affected --run --base 6428c5485 passed (916 test files, 7,280 tests). The appState request is pinned in runner-requests.json; its XCTests pass on the booted simulator.

Live on an iPhone 17 Pro simulator, runner warm from open:

appstate            State: runningForeground            Source: runner
home; appstate      State: runningBackgroundSuspended   Source: runner
snapshot; appstate  State: runningForeground            (the capture re-activated it)

Handler tests cover the runner answer, no live runner, and a runner that throws; a binder test proves no interactor is resolved without a live session.

Risk: source on the iOS result gains the value runner.

Copilot

Copilot review overview

🟡 Changes recommended

Unresolved integration, schema, runner-demand/readiness, cancellation, and lifecycle-test issues remain.

Get a fresh assessment by requesting another Copilot review.

Review effort: Lite
Findings: 1 · 3 · 1

Open (5)

Update iOS E2E expectations for runner-backed appstate · New
Update MCP schema for runner-backed appstate responses · New
Use a session-app label for background runner results · New
Propagate appstate request cancellation instead of falling back · New
Document either background state after home · New

What changed in this PR

Adds runner-backed iOS session app-state reporting with session fallback behavior.

Changes:

Adds the Apple appState runner command and lifecycle state mapping.
Extends contracts, daemon responses, CLI output, and runtime/interactor support.
Updates runner protocol, tests, and documentation.

File	Summary
`website/docs/docs/commands.md`	Documents iOS app-state behavior.
`src/daemon/handlers/session-state.ts`	Reads runner state with fallback handling.
`src/daemon/handlers/__tests__/session-appstate-input.test.ts`	Tests runner and fallback responses.
`src/commands/system/output.ts`	Formats state and source fields.
`packages/platform-apple/src/runtime.ts`	Adds Apple app-state runtime support.
`packages/platform-apple/src/runtime.test.ts`	Tests runtime capability facts.
`packages/platform-apple/src/runner/runner-contract.ts`	Adds the runner command.
`packages/platform-apple/src/runner/runner-command-traits.ts`	Defines command traits.
`packages/platform-apple/src/runner/__tests__/runner-command-traits.test.ts`	Tests command classification.
`packages/platform-apple/src/runner/__tests__/runner-client.test.ts`	Tests protocol fixtures.
`packages/platform-apple/src/interactions.ts`	Routes state reads through the runner.
`packages/platform-apple/src/__tests__/interactor-runner-provider.test.ts`	Tests runner transport routing.
`packages/contracts/src/interactor-types.ts`	Extends the interactor contract.
`packages/contracts/src/app-state.ts`	Extends the public app-state contract.
`packages/contracts/src/app-state-runtime.ts`	Defines runtime state types and binding.
`apple/runner/RUNNER_PROTOCOL.md`	Documents the runner command.
`apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/UnitTests/RunnerTests+ApplicationStateRawValueTests.swift`	Tests native state names.
`apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+Models.swift`	Adds runner command models.
`apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+CommandJournal.swift`	Journals the new command.
`apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+CommandExecution.swift`	Handles command execution behavior.
`apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+CommandDispatch.swift`	Dispatches and maps app state.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

okwasniewski · 2026-09-24T19:06:16Z

+        source: state ? 'runner' : 'session',
+        ...(state ? { state } : {}),


[claude-fable-5-1] responding on behalf of Oskar

Fixed in d8c243f: the scenario accepts runner (asserting state is a declared AppleApplicationState) or session, since the runner is warm in that tier but the read is allowed to answer from the record.

okwasniewski · 2026-09-24T19:06:18Z

+      source: 'session' | 'runner';
+      /**
+       * How the session app is running, as the runner reads it; absent with `source: 'session'`.
+       * `runningBackground` after `home` says the app left the foreground, not what took it.
+       */
+      state?: AppleApplicationState;


[claude-fable-5-1] responding on behalf of Oskar

Done in d8c243f: source is an enum of session and runner, state is the APPLE_APPLICATION_STATES enum, and src/mcp/__tests__/command-tools-appstate-schema.test.ts validates both iOS answers and the Android one and rejects an undeclared state.

okwasniewski · 2026-09-24T19:06:20Z

+function formatAppleAppState(
+  data: Extract<AppStateCommandResult, { platform: 'ios' | 'macos' }>,
+): string {
+  const lines = [`Foreground app: ${data.appName ?? data.appBundleId ?? 'unknown'}`];


[claude-fable-5-1] responding on behalf of Oskar

Agreed; the Apple formatter now labels the line Session app: in d8c243f. Android keeps Foreground app:, which is what it reads.

okwasniewski · 2026-09-24T19:06:22Z

+  } catch (error) {
+    emitDiagnostic({
+      level: 'warn',
+      phase: 'apple_appstate_runner_read_failed',
+      data: { code: asAppError(error).code, message: asAppError(error).message },


[claude-fable-5-1] responding on behalf of Oskar

Fixed in d8c243f: the catch rethrows when isRequestCanceledError says so and only a genuine runner-read failure falls back to the session answer.

okwasniewski · 2026-09-24T19:06:23Z


 - Android `appstate` reports live foreground package/activity.
- iOS `appstate` is unavailable: the Apple target answers no sessionless foreground probe, and a session-scoped guess about the foreground is not a fact. The per-command answer arrives as the [`targetActivation` disclosure](#foreground-repairs-on-ios), and the refusal's hint says so.
+- iOS `appstate` needs a session and answers about that session's app: `appName` and `appBundleId` from the session record, and `state` from the runner, which reads the app's `XCUIApplication` state (`runningForeground`, `runningBackground`, `runningBackgroundSuspended`, `notRunning`, `unknown`) with `source: runner`. After `home` the app reports `runningBackground`; which app took the screen is not read, since no Apple target answers a sessionless foreground probe. When the runner cannot answer, the record alone does, with `source: session` and no `state`. A command that had to bring the app back reports that as the [`targetActivation` disclosure](#foreground-repairs-on-ios).


[claude-fable-5-1] responding on behalf of Oskar

Reworded in d8c243f: after home the docs promise a background state, runningBackground or runningBackgroundSuspended, which is what the validation run showed.

thymikee · 2026-09-24T18:39:16Z

This makes iOS appstate go through the runner (bindLocalAppStateInteractor → iosRunnerOverrides.appState → runAppleRunnerCommand → ensureRunnerSession), but APPLE_SIMULATOR_OPERATION_HOSTS still declares appState: 'simulator' (https://github.com/callstack/agent-device/blob/ab43384/packages/platform-apple/src/runner-demand.ts#L37). A session with no live runner (a bridge simulator after open, an idle-stopped runner, any physical device) now cold-starts XCTest to answer a command that used to read instantly from the session record, and a plan like open X; appstate releases the speculative runner before appstate needs it, because the demand classification says 'none'. A runner-demand classification must name the host the operation actually runs through, and a session-state read should never start a runner: either read appState only from a runner session that is already live and fall back to source: session, or reclassify appState as 'runner' here and document the startup cost in commands.md, plus add a handler test for the no-runner case. Is a cold runner start able to move the session app out of the foreground, so the first appstate reports background instead of the state before the start? I didn't run the runner to check.

The appstate outputSchema still requires source: constSchema('session') on the Apple branch, but a runner-answered result now carries source: 'runner', so structuredContent no longer matches the advertised schema (https://github.com/callstack/agent-device/blob/ab43384/src/mcp/command-output-schemas.ts#L698). Every iOS appstate result the runner answers now fails validation for MCP clients that check structuredContent against outputSchema, and that's the normal path after this change. Every field on AppStateCommandResult should be mirrored here: set source to an enum of ['session', 'runner'], add state as an enum built from APPLE_APPLICATION_STATES, and add a schema-parity test in src/mcp/__tests__ that validates a runner-answered result.

Not blocking: the new binder pulls a runtime import into what was a type-only contracts module and trips the eager-closure-budgets check (its only consumer is platform-apple, so it could move there next to iosRunnerOverrides), runner-client.test.ts grew from 1029 to 1030 lines and now trips the test-file-size ratchet, the full-tier lifecycle scenario still asserts source === 'session' when the runner is warm and should accept or assert 'runner' instead, the daemon's appState catch appears to downgrade a cancelled request to a successful session-only answer instead of rethrowing, and the CLI's "Foreground app" label next to runningBackgroundSuspended plus the stale "appstate unavailable" line in sessions.md are worth a look — take or leave these.

Could the whole feature live with less surface: keep contracts type-only, put the binder in platform-apple next to iosRunnerOverrides, and have the handler read state only from an already-live runner session so appstate never starts one and runner-demand can stay 'simulator'? I checked whether INTERACTOR_OPERATIONS could own this instead, but Android's local-operation walk over that catalog would bind over its own adb foreground read, so that's not a fit; what would need to land first is either a non-starting "live runner session" query on the runner-session owner, or an explicit, documented decision that appstate is allowed to start the runner.

I haven't run the no-warm-runner route on a local simulator: stopping the runner (or letting it idle-stop), running open <app> on a bridge simulator, then appstate --json --debug would show whether the runner starts, how long the call takes, and whether the session app is still runningForeground right after the cold start; the same run on a physical device matters too, since appleAppStateFact admits kind 'device'. Both Coverage failures trace to this diff (the eager-closure edge from app-state-runtime.ts and the runner-client.test.ts line-count growth), and Smoke Tests was still running when I looked, with no obvious overlap between this diff and the smoke tier's routes. Before this merges, appstate needs to stop cold-starting the runner (or be reclassified with the cost documented), the outputSchema needs the source and state enums, and the two Coverage gates need to clear.

okwasniewski · 2026-09-24T19:06:15Z

[claude-fable-5-1] responding on behalf of Oskar

Reworked in d8c243f along the lines you sketched, with less surface:

No cold start. The binder moved to packages/platform-apple/src/app-state-runtime.ts beside iosRunnerOverrides and asks host.appleApplications.hasLiveRunnerSession first; without a live runner session it answers nothing and never resolves the interactor, so the handler keeps source: session with no state. appState stays 'simulator' in the runner-demand table, since the read demands no runner of its own; the docs say so. A binder test asserts no interactor is resolved without a live session, and a handler test covers the empty answer.
Contracts type-only again. APPLE_APPLICATION_STATES, the type, and the guard live in kernel/snapshot.ts next to the prior states; app-state-runtime.ts imports types only, so the eager-closure budget holds.
MCP schema. source is ['session', 'runner'], state is the enum from APPLE_APPLICATION_STATES, and command-tools-appstate-schema.test.ts validates a runner answer, a session answer, an Android answer, and rejects an undeclared state word.
Ratchet. runner-client.test.ts is back at 1029 lines.
Not blocking items: the full-tier scenario accepts runner (and checks state is declared) or session; the handler rethrows a cancelled request instead of downgrading it; the CLI Apple lines read Session app:; commands.md says a background state rather than runningBackground; the stale sessions.md line is rewritten.

On your question about a cold start moving the app out of the foreground: moot now, since the read never starts one. Gate at d8c243f: 904 test files, 7,183 tests, both Coverage tripwires green locally.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

okwasniewski · 2026-09-24T19:18:17Z

[claude-fable-5-1] responding on behalf of Oskar

Rebased onto 6428c54 (clean) and pinned the new command in the golden table #2900 added: ios-simulator.interactor-app-state.read in contracts/fixtures/runner-requests.json, driven through interactor.appState() in the producer test. That was the Smoke and Coverage failure on d8c243f (runner commands with no production request). Head addf05f; local gate check:affected --run --base 6428c5485: 916 test files, 7,280 tests. testEveryRunnerCommandTypeHasAProductionRequest, testProductionRunnerRequestsDecodeWithoutDroppingAKey, testEveryRunnerRequestFieldHasAProductionRequest, and testApplicationStateNamesMatchTheAppStateContract pass on the booted iPhone 17 Pro simulator via xcodebuild test-without-building.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

thymikee · 2026-09-24T22:41:32Z

This is ready for human review at addf05f. The runner cold-start and MCP schema issues from the earlier review (ab43384, #2929 (comment)) are both fixed, and all 15 checks pass, including the runner-requests golden-table and the eager-closure/test-size checks that failed before.

I did not reproduce the author's warm-runner transcript on a simulator, and no live run shows the no-runner route (runner idle-stopped or bridge simulator right after open, then appstate --json --debug returning source: session with no runner start). That guarantee rests on the binder unit test with a mocked hasLiveRunnerSession and on reading hasLiveIosRunnerSession. A race is possible but not exercised: if the runner reports ready but has died, or idle-stops between the liveness check and the send, the read-only retry path in runAppleRunnerCommand may recover or restart it, and the runtime-snapshot find path uses the same check-then-send pattern. I also did not run the physical-device route (kind: 'device' admitted by appleAppStateFact) or the Swift XCTests locally.

Not blocking: whether readAppleSessionAppState should rethrow a runtime-contract-invalid error instead of falling back to the session answer (src/daemon/handlers/session-state.ts:112), whether the test-file-size ratchet on packages/platform-apple/src/runner/tests/runner-client.test.ts should stay avoided by moving runnerProtocolCommandFixtures to a sibling module, and whether the new isRequestCanceledError rethrow needs its own handler test (src/daemon/handlers/tests/session-appstate-input.test.ts:431) — take or leave any of these.

…ough the runner iOS appstate answered from the session record alone: after home it still named the app as if in front, and the runtime declared the read unavailable because no Apple target answers a sessionless foreground probe. The session app's own state is a fact the runner has: a new appState runner command reads XCUIApplication(bundleIdentifier:).state and names it (runningForeground, runningBackground, runningBackgroundSuspended, notRunning, unknown). It is a lifecycle read, so the activation preflight is skipped and the state reported is the one the app is in. The Apple runtime admits the read on iOS-family simulators and devices, the handler adds state and source: runner to the session answer when the runner answers, and keeps source: session otherwise. The frontmost app is still not read.

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

… contract failure (#2950) * refactor(apple): put the Apple app states at one source and reuse the contract failure #2929 added `appState` beside the #2682 activation disclosure, and the two state lists ended up pointing the wrong way: the full `XCUIApplication.State` list was spread from the smaller prior-state subset, so the narrower concept owned the names the broader one answers with. The full list is now the single declaration and the prior states derive from it by dropping the one state the runner skips `activate()` in. The Apple binder also restated two things contracts already owns: a resolver type that is the local interactor resolver under another name, and the advertised-without-an-implementation failure the `invalidRuntimeContract` leaf already builds. Both are reused now, and the failure branch finally has a test pinning its typed reason, which is what keeps it outside every closed reason set that licenses answering from the session record. `AppleApplicationState` no longer re-exports through `contracts/app-state-runtime`: the kernel is its declaration site and both consumers read it there. * chore(gates): publish the contracts runtime-contract-error subpath `invalidRuntimeContract` had no export target: it was a leaf kept out of the `platform-runtime` façade so that façade stays exhaustive over its sources and inside its eager-closure budget. Now that a platform package consumes it, the subpath is its declaration site in the manifest, and the independently committed contracts export snapshot carries it with the one-line regeneration.

Copilot AI lite review requested due to automatic review settings September 24, 2026 17:49

Copilot started reviewing on behalf of okwasniewski September 24, 2026 17:52 View session

Copilot AI reviewed Sep 24, 2026

View reviewed changes

Copilot AI review requested due to automatic review settings September 24, 2026 19:06

okwasniewski force-pushed the oskar/ios-appstate-runner branch from ab43384 to d8c243f Compare September 24, 2026 19:06

Copilot AI reviewed Sep 24, 2026

Copilot AI review requested due to automatic review settings September 24, 2026 19:18

okwasniewski force-pushed the oskar/ios-appstate-runner branch from d8c243f to addf05f Compare September 24, 2026 19:18

Copilot AI reviewed Sep 24, 2026

thymikee added the ready-for-human Valid work that needs human implementation, judgment, or maintainer merge label Sep 24, 2026

thymikee force-pushed the oskar/ios-appstate-runner branch from addf05f to 197ace9 Compare September 25, 2026 06:03

Copilot AI review requested due to automatic review settings September 25, 2026 06:03

Copilot AI reviewed Sep 25, 2026

thymikee merged commit f2813ac into callstack:main Sep 25, 2026
15 checks passed

This was referenced Sep 25, 2026

refactor(ios-runner): make off-main access to main-owned capture state fail the runner gates #2911

Merged

refactor(apple): put the Apple app states at one source and reuse the contract failure #2950

Merged

		source: state ? 'runner' : 'session',
		...(state ? { state } : {}),

Uh oh!

Conversation

okwasniewski commented Sep 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Validation

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

🟡 Changes recommended

Uh oh!

okwasniewski Sep 24, 2026

Choose a reason for hiding this comment

Uh oh!

okwasniewski Sep 24, 2026

Choose a reason for hiding this comment

Uh oh!

okwasniewski Sep 24, 2026

Choose a reason for hiding this comment

Uh oh!

okwasniewski Sep 24, 2026

Choose a reason for hiding this comment

Uh oh!

okwasniewski Sep 24, 2026

Choose a reason for hiding this comment

Uh oh!

thymikee commented Sep 24, 2026

Uh oh!

okwasniewski commented Sep 24, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

okwasniewski commented Sep 24, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

thymikee commented Sep 24, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

okwasniewski commented Sep 24, 2026 •

edited

Loading