Skip to content

fix(ios-runner): log the fast app guard and gesture policy only when they change - #2814

Merged
thymikee merged 4 commits into
callstack:mainfrom
okwasniewski:oskar/runner-log-dedupe
Sep 23, 2026
Merged

thymikee merged 4 commits into
callstack:mainfrom
okwasniewski:oskar/runner-log-dedupe

Conversation

@okwasniewski

Copy link
Copy Markdown
Contributor

Summary

Two days of running the Bluesky e2e suite on one simulator left a 374,000-line runner.log. The per-command breakdown:

Marker Lines
FAST_APP_GUARD 35,080
COMMAND_ACCEPTED / COMMAND_COMPLETED 39,415 / 39,408
SYNTHESIZED_GESTURE_POLICY 5,612
PRIVATE_AX_SNAPSHOT_USED 5,639
SNAPSHOT_XCTEST_CHANNEL_DEFERRED 5,467

The accepted/completed pairs are worth every line (the whole investigation was built on them). The guard and policy markers restate, on every command, a fact that had not changed since the previous one: same bundle in the foreground, same gesture policy for the same kind.

Change

  • RepeatedLogSuppressor (in RunnerTests+Lifecycle.swift): logIfChanged(key:fact:line:) writes line only when fact differs from the last one written under key. Locked, reset() forgets every key.
  • canUseFastForegroundAppGuard: keyed fast_app_guard, fact is bundle plus state; the line still carries the command that observed the change.
  • logSynthesizedGesturePolicyDecision: keyed per gesture kind so alternating taps and scrolls do not restate each other; the decision line is the fact.
  • invalidateCachedTarget resets the suppressor, so a rebind states the guard once more.

Snapshot markers are left alone here: they carry per-capture numbers (nodes, depth) and are the evidence for the busy-runner work in #2804 and #2783.

Validation

  • Two unit tests under the existing #if AGENT_DEVICE_RUNNER_UNIT_TESTS guard: the suppressor writes on first sight, change, other key and after reset; invalidating the cached target resets it. Both pass on iPhone 17 Pro Max (iOS 27.0) via xcodebuild test-without-building.
  • End to end, daemon pointed at this runner build, Settings on the same simulator: 8 tap commands wrote 1 FAST_APP_GUARD line and 1 SYNTHESIZED_GESTURE_POLICY line.
  • pnpm check:xctest-selection (277 methods, 0 unreachable) and pnpm check:packaged-runner-swift (50 files parse, line parity kept) clean.

…they change

Two days of one e2e suite left a 374k-line runner.log: 35,080 FAST_APP_GUARD
and 5,612 SYNTHESIZED_GESTURE_POLICY lines, one per command, each restating a
fact that had not changed since the previous command.

Both markers now go through a small RepeatedLogSuppressor keyed by marker (and
gesture kind for the policy): a line is written when the fact differs from the
last one written under that key, and the suppressor is reset with the cached
target so a rebind states the fact once more. COMMAND_ACCEPTED/COMPLETED pairs
are untouched.

Verified on an iOS simulator against Settings: 8 tap commands wrote 1 guard
line and 1 policy line.
Copilot AI lite review requested due to automatic review settings September 23, 2026 12:14

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

One or more issues must be addressed before approval.

Get a fresh assessment by requesting another Copilot review.

Review effort: Lite
Findings: 1 Low severity

Open (1)
What changed in this PR

Reduces repetitive iOS runner logging while preserving meaningful policy and guard transitions.

Changes:

  • Adds keyed, resettable log suppression.
  • Suppresses unchanged foreground-guard and gesture-policy markers.
  • Adds focused unit tests for suppression and reset behavior.
File Description
apple/​runner/​AgentDeviceRunner/​AgentDeviceRunnerUITests/​RunnerTests+SynthesizedGesturePolicy.swift Updated as part of this pull request.
apple/​runner/​AgentDeviceRunner/​AgentDeviceRunnerUITests/​RunnerTests+Lifecycle.swift Updated as part of this pull request.
apple/​runner/​AgentDeviceRunner/​AgentDeviceRunnerUITests/​RunnerTests.swift Updated as part of this pull request.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests.swift Outdated
Copilot AI review requested due to automatic review settings September 23, 2026 12:29
Review follow-ups on the deduped per-command markers:
- the RepeatedLogSuppressor class (string keys, lock, autoclosure) is gone;
  the two markers keep a last-written line beside the other target-bound state,
  the policy one keyed by SynthesizedGesturePolicyKind
- the target-reset chorus copied at three sites (host activate, invalidate,
  activate target) is one resetTargetBoundState(), so a rebind through any of
  them restates both markers, as the comment already claimed
- the guard line drops command=, which a deduped line would misreport; the
  command is on the adjacent COMMAND_ACCEPTED line
- the policy line is built by one pure function with its own unit test

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

Reset suppression on process-change rebinds and verify actual log emission behavior in tests.

Review effort: Lite
Findings: None

Resolved since last review (1)
Previously missed (1)

In code that hasn't changed since last review

Low severity Test does not verify runner.log emission behavior

apple/​runner/​AgentDeviceRunner/​AgentDeviceRunnerUITests/​RunnerTests+Lifecycle.swift:518

This test only checks the Boolean return value; it never observes or evaluates line, so it would still pass if the NSLog at lines 499–500 were removed or moved to the wrong branch. The behavior being changed is the actual runner.log emission count/content, so add a logger seam or a side-effecting autoclosure assertion that verifies first/change/reset calls emit and repeated facts do not.

Copilot AI review requested due to automatic review settings September 23, 2026 12:47
@okwasniewski

Copy link
Copy Markdown
Contributor Author

Restructured after an internal quality review (a2f0a5c):

  • The RepeatedLogSuppressor class is gone. Each marker keeps a last-written line as a plain field beside the other target-bound state; the policy one is keyed by SynthesizedGesturePolicyKind, no string keys, no lock.
  • The target-reset chorus that was copied at three sites (host activate, invalidate, activate target) is one resetTargetBoundState(), so a rebind through any of them restates both markers. The previous version only reset on invalidate, which the comment overstated.
  • The guard line drops command=: a deduped line carrying one command would misreport; the command is on the adjacent COMMAND_ACCEPTED line.
  • The policy line is built by a pure function with its own unit test. Both new tests pass on the simulator; end to end, 8 taps still write 1 guard line and 1 policy line.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

One or more issues must be addressed before approval.

Get a fresh assessment by requesting another Copilot review.

Review effort: Lite
Findings: 3 Medium severity

Open (3)

)
// The command is on the adjacent COMMAND_ACCEPTED line; repeating it here would make a deduped
// marker read as if only that command ever passed the guard.
let line = "AGENT_DEVICE_RUNNER_FAST_APP_GUARD bundle=\(requestedBundleId) state=\(activeApp.state.rawValue)"
@thymikee

Copy link
Copy Markdown
Member

Reviewed at a2f0a5c.

refreshCachedTargetIfProcessChanged (https://github.com/callstack/agent-device/blob/a2f0a5c/apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests+Lifecycle.swift#L203-L224) rebinds currentApp and currentAppProcessIdentifier when the same bundleId's process silently changed, but it clears state with clearRememberedTextEntryTap() instead of the new resetTargetBoundState(), so lastLoggedFastAppGuardLine and lastLoggedGesturePolicyLines carry over from the old process. When the recomputed bundle+state fact or gesture-policy fact matches the stale cached string, the guard or policy line for the new process is silently dropped from runner.log. This is the exact rebind case the PR's own doc comment says every binding site must restate, and it defeats the marker during the kind of investigation this change is meant to support. Every site that binds, rebinds, or drops the target must call resetTargetBoundState() (or clear both fields directly); refreshCachedTargetIfProcessChanged is the one call site that currently doesn't.

testSynthesizedGesturePolicyMarkerWritesOncePerKindUntilTheDecisionChanges (https://github.com/callstack/agent-device/blob/a2f0a5c/apple/runner/AgentDeviceRunnerUITests/RunnerTests+SynthesizedGesturePolicy.swift#L168) only checks the lastLoggedGesturePolicyLines dictionary, never NSLog output, and canUseFastForegroundAppGuard's dedup path has no direct test at all; testResettingTargetBoundStateForgetsTheLastWrittenMarkers only covers the reset, not suppression. Since the dictionary write sits inside the same if as the NSLog call, a change that keeps the write but drops the log gate would still pass everything green, and the reduced-duplicate-logging behavior this PR exists for would be unverified. Every call path that writes lastLoggedFastAppGuardLine or lastLoggedGesturePolicyLines needs a test asserting emission count (not just dictionary state) across a repeated identical fact versus a changed fact.

Not blocking: the PR description says the line still carries the command that observed the change, but the shipped line drops command and the command: CommandType parameter in canUseFastForegroundAppGuard is now unused, so either the description or the dead parameter needs fixing, though this can be taken or left.

I read the diff and the full pre/post RunnerTests+Lifecycle.swift and RunnerTests+SynthesizedGesturePolicy.swift at a2f0a5c; I did not run the Swift unit tests or a live device session. The author reports that 8 tap commands wrote 1 FAST_APP_GUARD line and 1 SYNTHESIZED_GESTURE_POLICY line, but no runner.log excerpt is attached, so that's the author's account, not something I confirmed. What's missing is a live run that forces refreshCachedTargetIfProcessChanged's path — same bundleId, process replaced without an explicit invalidate or relaunch — showing the guard and policy lines restate for the new process once the fix above lands; the steady-state repeat-command case doesn't exercise this path. Smoke Tests were still queued at the time I looked, so I can't say whether they pass; if they do fail, note that Smoke Tests exercise real command dispatch on a simulator and would hit canUseFastForegroundAppGuard, activateTarget, invalidateCachedTarget, and refreshCachedTargetIfProcessChanged directly, so a failure there would be attributable to this diff. Before this can merge, refreshCachedTargetIfProcessChanged needs to call resetTargetBoundState() so a silent process rebind can't suppress its own guard or policy line.

@okwasniewski

Copy link
Copy Markdown
Contributor Author

The iOS smoke failure here is the same lane flake as on #2804 and #2811 (regular depth-1 snapshot must disclose the Simulator AX bridge evidence gap), now on five branches today. This PR only dedupes two log lines in the runner; every other check is green. Evidence collected on #2491.

… they write

refreshCachedTargetIfProcessChanged rebinds the target but only cleared the
text-entry tap, so the new process could lose its guard or policy line. It now
runs resetTargetBoundState. Markers write through runnerMarkerWriter so tests
count emitted lines, and the fast app guard dedup is testable on its own. Drops
the unused command parameter of canUseFastForegroundAppGuard.

Co-Authored-By: Claude <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings September 23, 2026 16:10

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

One or more issues must be addressed before approval.

Get a fresh assessment by requesting another Copilot review.

Review effort: Lite
Findings: 2 High severity · 1 Medium severity

Open (3)
Resolved since last review (2)

clearRememberedTextEntryTap()
snapshotXCTestPenaltyWarmupExemptionPending = false
lastLoggedFastAppGuardLine = nil
lastLoggedGesturePolicyLines.removeAll()
Comment on lines +149 to +151
if lastLoggedGesturePolicyLines[kind] != line {
lastLoggedGesturePolicyLines[kind] = line
runnerMarkerWriter(line)
@thymikee

Copy link
Copy Markdown
Member

Reviewed at f18238c. The fix at RunnerTests+Lifecycle.swift:219 resolves the finding from the earlier review, and the code looks right.

Not blocking: a test that calls refreshCachedTargetIfProcessChanged directly with a new pid for the same bundle would catch a regression on that line that the current tests miss.

Smoke Tests was still running at review time. This diff changes only log output and log-dedup state, so a failure on the known depth-frontier assertion (fixed by #2832) would be unrelated; any other failure needs a look.

@thymikee thymikee added the ready-for-human Valid work that needs human implementation, judgment, or maintainer merge label Sep 23, 2026
@thymikee
thymikee merged commit 81b4760 into callstack:main Sep 23, 2026
12 of 13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-human Valid work that needs human implementation, judgment, or maintainer merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants