Add SwiftUI support - #41
Merged
Merged
Conversation
SwiftUI accessibility elements cannot provide alpha or cornerRadius and carry their content as accessibilityLabel, so ViewInspection needs optional fields and a conditional text row. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
SwiftUI's Text/Image/Button never create UIKit views, so the scanner walks the accessibility tree that SwiftUI publishes for VoiceOver (public API only) and classifies elements by their traits. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
SwiftUI targets have no backing UIView, so their monitor buttons attach to the root view at the element's window frame (no scroll tracking) and inspections re-read accessibilityFrame at selection time. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Accessibility-element buttons attach directly to rootView after infoView, so they painted and hit-tested above it whenever their frame overlapped the info panel — obscuring it and stealing its drag gesture. Re-assert bringSubviewToFront(infoView) after attaching all target buttons in show(on:). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
SwiftUI apps have no SceneDelegate, so expose a root-view modifier that starts the monitor on first appear. Screen-change detection keeps using the existing viewDidAppear swizzling, which UIHostingController hits. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
InfoView keeps its UIKit contract (frame-managed size, drag by the overlay) and delegates drawing to a hosted InfoRowsView. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
MonitorOverlay adds SwiftUI accessibility-element monitor buttons directly to rootView, landing them above the launcher button that ViewMonitor.reload() had already added. If a SwiftUI element overlaps the launcher's top-right corner, taps meant to stop measurement hit the element button instead and the launcher's drag gesture is blocked. Re-front the launcher after overlay.show(on:) in the onToggle closure. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Example/ViewMonitorSwiftUIExample was added without being listed in .swiftlint.yml's included paths, leaving it unlinted. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The removed targetView property no longer exists; the test now exercises measurementTarget, so its display name and function name should say so. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
button.onToggle captured its own button strongly while removeLauncherButton() never clears onToggle, forming a button -> onToggle -> button self-cycle. Since reload() runs on every viewDidAppear/orientation change while started, every screen transition leaked the superseded launcher instance. Capture button weakly alongside self, matching the existing weak-self pattern. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
InfoView now renders through a UIHostingController, whose viewDidAppear fires when the overlay is shown. The swizzled hook mistook it for an app screen transition and called reload(), which tore down the overlay that had just been presented and replaced the launcher with a fresh OFF instance - on device the toggle appeared to never turn on. Mark ViewMonitor's internal hosting controller with MonitorInternalViewController and skip transition detection for it. Excluding UIHostingController as a whole would also swallow real transitions in SwiftUI apps, so only ViewMonitor's own controllers are excluded. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
iOS builds the accessibility tree only while an accessibility client is attached, so SwiftUI detection silently found nothing in normal runs. The README workaround (simctl defaults write AutomationEnabled) turned out not to work at all: the tree stays empty even after a simulator reboot with the key set. Add ViewMonitor.enableSwiftUIElementDetection(), which starts tree construction in-process the same way UI test runners do (_AXSSetAutomationEnabled). The implementation is compiled only into DEBUG builds; release builds get a no-op returning false and contain no private-API symbol strings. The SwiftUI example app now calls it at startup, and measurement buttons appear over all SwiftUI elements on a clean simulator with no external tooling. This also unblocks the end-to-end test that was written and deleted earlier in this branch because the tree never materialized in the unit-test host: with in-process activation it materializes immediately, so the scanner is now covered against a real UIHostingController tree instead of only fakes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
When the scanner finds a hosting view but zero accessibility elements, the overlay used to show nothing at all, which is indistinguishable from a bug (and was reported as one). Show instructions in the InfoView instead: call enableSwiftUIElementDetection() or attach an accessibility client. Pure UIKit screens are unaffected. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The documented simctl defaults write AutomationEnabled workaround does not actually build the accessibility tree (verified on iOS 26 simulator, including with a reboot). Document enableSwiftUIElementDetection() as the primary path, keep Accessibility Inspector / VoiceOver as the alternative, and cover both example apps in the device signing setup. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
SwiftUI's List inserts a UICollectionView between the hosting view and the row content, and each row's accessibility elements are published by a CellHostingView inside the cell - a class that does not match the _UIHostingView prefix. Reading only the hosting view's own accessibilityElements therefore found no rows at all: the hosting view's array contains just the collection view (a UIView, skipped to avoid double detection, with no AX children of its own to recurse into). Once a hosting view is entered, read accessibility elements from every view in its subtree instead. Pure UIKit hierarchies keep the previous behavior (no accessibility scan), and elements published by more than one view are deduplicated by identity. Covered by a structural unit test mirroring the List layout and an end-to-end test against a real List in a UIHostingController; rows are measured one element per row, matching how List publishes them. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Taps on areas not covered by a measurement button used to reach the app: actions fired, lists scrolled (leaving SwiftUI element buttons at stale positions), and any resulting screen transition made the reload teardown discard the overlay and reset the toggle to off. Insert a transparent shield between the app and the measurement UI while the overlay is shown. It absorbs app-bound touches; the info panel, SwiftUI element buttons, and the launcher sit above it, and touches aimed at UIKit-target measurement buttons (which live inside the app hierarchy, below the shield) are let through via hit-test requery with a re-entrancy guard. Toggling isHidden during hitTest is not an option: some internal UIKit hit-test paths re-enter the override without checking hidden, overflowing the stack (verified via crash report). To interact with the app (e.g. scroll further down a list), toggle off, move the screen, and toggle on again to re-scan. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…e targets On device, tapping a UIKit-target monitor button (nav bar labels) did nothing while measuring. The shield passed such touches through by re-running the hit test, but the requery traverses _UIHostingView.hitTest, which is not a pure function: for the same point it returned the monitor button on the first call and itself on the second (captured via on-device logging), so the final hit went to the shield and the button never received the touch. Stop relying on hit-testing through SwiftUI entirely: attach UIKit target buttons to the rootView above the shield, the same placement SwiftUI element buttons already use, with frames converted to window coordinates. The shield becomes a plain absorb-everything view (the requery and its re-entrancy guard are gone), and forcing isUserInteractionEnabled on target views is no longer needed. Since buttons are now fixed-position, targets that are effectively invisible (hidden or alpha 0, e.g. the inline nav title label while the large title is shown) get no button - previously their buttons were invisible together with the hidden parent, but at window level they would float over nothing. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Monitor buttons are now fixed at window coordinates for UIKit targets too, so the scroll-following note no longer belongs to the SwiftUI limitations list. Fold it into the general measuring section, and correct the transition behavior: the overlay closes and the toggle returns to OFF, it does not refresh. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Text/Image/Buttonthrough the accessibility elements SwiftUI publishes: newMeasurementTargetabstraction,AccessibilityElementScanner, and overlay integration with cross-framework (UIKit x SwiftUI) distance measurementListrows: List inserts aUICollectionViewbetween the hosting view and row content, so the scanner reads accessibility elements from every view in a hosting subtree (rows measure one element per row, matching how List publishes them)View.viewMonitor()so apps using the SwiftUI lifecycle can start the monitor without a SceneDelegateViewMonitor.enableSwiftUIElementDetection(): iOS builds the accessibility tree only while an accessibility client is attached, so without this call SwiftUI detection finds nothing in a normal run. It starts tree construction in-process the same way UI test runners do. Debug builds only — the implementation is compiled out of release builds (no-op returningfalse, no private-API symbol strings in the binary)InfoViewrows with SwiftUI internally (UIKit contract unchanged)Example/ViewMonitorSwiftUIExample, generated by a Ruby script like the existing example) and build it on CIKnown limitations
.accessibilityElement(children: .combine)regions andListrows measure as one element;.accessibilityHidden(true)views are not detected; font/background/cornerRadius showNoneNotes for reviewers
InfoView's internalUIHostingControllerfiresviewDidAppearwhen the overlay is shown; the swizzled transition hook mistook that for an app screen transition andreload()tore down the overlay that had just been presented — the toggle appeared to never turn ondefaults write com.apple.Accessibility AutomationEnabled 1) does not actually work — the tree stays empty even after a simulator reboot. Replaced byenableSwiftUIElementDetection()Listrows were not detected at all, because row elements are published byCellHostingView(not_UIHostingView-prefixed) inside collection view cells_UIHostingView.hitTest, which is not a pure function — on device it returned the monitor button on the first call and itself on the second for the same point (captured via on-device logging). Buttons now attach above the shield instead, so no touch ever routes through SwiftUI's hit testingUIHostingControlleraccessibility tree because the tree never materialized in the unit-test host. In-process activation unblocks it:SwiftUIElementDetectionTestsnow verifies Text/Image/Button detection and List row detection against real hosting views rather than fakesTest plan
swiftlint --strict0 violations; both example apps buildUIHostingControllertrees end-to-end (VStack content and List rows), the InfoView-triggered teardown regression, the "no elements detected" notice, and the input shield (absorb, button placement above the shield, invisible-target skip, z-order, removal on hide)🤖 Generated with Claude Code