feat(ios): the snapshot node carries the field's placeholder - #2961
Conversation
Android nodes carry the field's hint as placeholder since callstack#2927; iOS nodes carried nothing, so a consumer could neither query a field by its placeholder nor tell an empty field (XCTest reports the placeholder as its value) from one holding that text. XCTest exposes placeholderValue on every element and snapshot, and the AX server exposes XC_kAXXCAttributePlaceholderValue. Every iOS producer now publishes placeholder on RawAXNode and PresentedNode: the XCTest tree and query sweeps read placeholderValue off the snapshot or element, the runner's private-AX reader asks for the keypath beside value, and the Simulator AX bridge requests the attribute (source v1.8.0, one recompile per host). An empty placeholder reads as none. The docs name the fact for both platforms.
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
There was a problem hiding this comment.
All reported issues were addressed across 17 files
Reply with feedback, questions, or to request a fix.
Fix all with cubic | Re-trigger cubic
|
Commit 80aa8df: the private-AX reader now sends Not blocking, can be taken or left: is I did not re-run the host-lane XCTest, the snapshot differential, or the live Control Inventory captures the PR body cites. I can't tell whether XCElementSnapshot's keypath mapper actually maps |
…apart in the tests Review follow-ups: the collapsed-tab producer read the placeholder only after its content check, so a placeholder-only field inside a collapsed container was dropped; the placeholder now counts as content. The bridge test used one string for value and placeholder, so it could not tell the two attributes apart; it uses distinct strings and asserts both. A wire-shape test pins that a node without a placeholder encodes no key. The docs stop calling a value equal to the placeholder an empty field, since a user can type that text.
…ulary, read it on text entry only Review follow-ups. The Simulator AX bridge asked for the placeholder attribute in its required set, so a runtime whose vocabulary lacks it would have failed every capture; a mismatch now retries the mapping without the attribute and the capture serves without placeholders. The bridge trims the placeholder the way the runner does, so whitespace reads as none. The element sweeps read a live XCUIElement attribute per element inside their deadline, so they read placeholderValue on text-entry types only; the snapshot producer keeps reading it off the snapshot it already holds.
|
[claude-fable-5-1] responding on behalf of Oskar Thanks. Live Optional items, all taken in fd3ae05 and 9ef0088: the bridge treats the placeholder attribute as optional (a vocabulary mismatch retries the mapping without it, so a runtime lacking it serves without placeholders); the doc says a value equal to the placeholder is an empty field or one holding that text; the bridge trims the placeholder like the runner (the private-AX reader already trimmed through
|
|
The code looks right at 9ef0088. The placeholder now round-trips through the snapshot node the way the earlier review (#2961 (comment)) asked for, and there are no conflicts. Not blocking: could the collapsed-tab sweep at https://github.com/callstack/agent-device/blob/9ef0088/apple/runner/AgentDeviceRunner/AgentDeviceRunnerUITests/RunnerTests%2BSnapshotAcquisition.swift#L402 drop its placeholder read and the I did not re-run the live .privateAX capture; I took the reported backend, runner log line, and name-input value/placeholder pair as stated. I could not exercise the bridge's vocabulary-mismatch fallback, because no runtime lacking XC_kAXXCAttributePlaceholderValue was available. I did not run the host-lane XCTest unit tests; testPresentedNodeOmitsAnAbsentPlaceholder guards the encoder contract and would also pass on the pre-change code, which is expected since it's a contract pin, not a regression test. Three Smoke Tests jobs, Coverage, and Integration are green. One Smoke Tests job (run 36123366332) is still pending, and since the iOS smoke route builds the runner and bridge and runs snapshot captures, which this diff changes, its result counts as related until the log says otherwise; nothing has failed so far. That job finishing green is the only thing left. |
thymikee
added
the
ready-for-human
Summary
Android nodes have carried the field's hint as
placeholdersince #2927; iOS nodes carried nothing. A consumer could neither find a field by its placeholder nor tell an empty field from one holding that text, since XCTest reports an empty field's placeholder as itsvalue. XCTest exposesplaceholderValueon every element and snapshot, and the AX server exposesXC_kAXXCAttributePlaceholderValue.Every iOS producer now publishes
placeholderonRawAXNodeandPresentedNode, absent when empty:placeholderValueoff the snapshot or element (SnapshotEvaluationcarries it);placeholderValuekeypath besidevalue;v1.8.0, so each host recompiles the bridge once.The element sweeps read
placeholderValueon text-entry types only (a liveXCUIElementattribute is one lookup per element inside the sweep's deadline); the snapshot producer reads it off the snapshot it already holds.The kernel field, the unchanged-digest fields, and the response views already knew
placeholderfrom #2927, so nothing on the wire changes shape. 17 files: 9 Swift/ObjC, 5 TypeScript and fixtures, 3 tests and docs.Validation
Tested at
9ef008834f(third commit: optional bridge attribute, trimmed placeholder, text-entry-only sweep reads;pnpm check:affected --run --base c31cc49f6passed on it, and the three host-lane XCTests pass on the rebuilt runner):pnpm check:affected --run --base c31cc49f6: all runnable checks passed,check:xctest-selectionandcheck:packaged-runner-swiftamong them.testSnapshotPresentationPreservesCurrentWireShape(the wire shape now namesplaceholder) and the newtestPrivateAXAcquisitionCarriesTheFieldPlaceholderpass;pnpm test:ios-snapshot-differentialpasses.tree.test.tspins the bridge mapping and that an empty attribute is omitted;protocol.test.tspins the v1.8.0 vocabulary in the native source.dev.e2e.benchmarkControl Inventory, on the rebuilt runner:snapshot --actions --jsonpins the private-AX tier (snapshotQuality.backend: private-ax,reasonCode: requested-backend); the runner log showsAGENT_DEVICE_RUNNER_PRIVATE_AX_SNAPSHOT_USED nodes=58 depth=64for that capture, andname-inputcarriesvalue: "Ada Lovelace"besideplaceholder: "Type your name", withpassphrase-inputandkey-inputcarrying theirs. The XCTest tree producer on an iPhone 17 Pro reportsplaceholderon the three fields (Type your name,Passphrase,Key echo), a filled field included; the v1.8.0 bridge on an iPhone 17 Pro Max reports the same field withplaceholder: "Type your name",role: RCTUITextField,value: "Ada Lovelace".XC_kAXXCAttributePlaceholderValuefails the bridge's existing vocabulary check and falls back to the runner, as any unknown attribute does today.