Skip to content

feat: add exact overlay target tracking contract#10

Merged
abrichr merged 3 commits into
mainfrom
feat/overlay-target-tracking-v2
Jul 25, 2026
Merged

feat: add exact overlay target tracking contract#10
abrichr merged 3 commits into
mainfrom
feat/overlay-target-tracking-v2

Conversation

@abrichr

@abrichr abrichr commented Jul 25, 2026

Copy link
Copy Markdown
Member

What changed

  • Adds an additive ControlOverlayFrameV2 / ControlOverlayTimelineV2 contract while leaving V1 unchanged.
  • Carries only normalized browser top-level CSS-viewport geometry, source CSS viewport and DPR, an optional closed action kind, and an exact binding. Native and RDP device-pixel geometry is intentionally outside this V2 schema.
  • Uses a run/export-scoped HMAC reference for private runtime observations instead of a linkable raw observation or screenshot hash.
  • Uses the immutable media SHA-256 plus decoded frame index for published media.
  • Exposes fail-closed helpers that return target tracking only for the exact observation or decoded media frame; missing bindings omit the rectangle.
  • Keeps viewer layout out of runtime state: Desktop, Cloud, and media renderers map the normalized rectangle through their actual content box.
  • Requires every target carried by a media timeline to use a matching exact media-frame binding; live observation-HMAC targets are FrameV2-only.
  • Ships deterministic Pydantic and packaged JSON Schemas for Python, TypeScript, Rust, and other consumers.

Why

The V1 overlay contract intentionally carries control state but no target geometry. Browser and cross-surface viewers need a shared way to render honest target tracking without injecting into the target application, replaying selectors, or inferring motion/timing the runtime did not retain.

The observation HMAC is preferable to a raw frame SHA because raw hashes can link a private observation across evidence boundaries. Immutable sanitized/public media already has an explicit digest boundary, so its exact digest and decoded-frame index are the appropriate binding there.

Consumer impact

V1 behavior and generated V1 schema bytes do not change. Consumers opt into V2 for browser geometry. A V2 renderer must omit target geometry unless its observation HMAC or exact decoded media frame matches; it must not replay a selector, interpolate missing tracking, or trust a producer-authored transform for a future viewer surface.

The conventional feat: title makes this eligible for the repository's next minor semantic release, but this draft does not authorize merge or publication.

Validation

  • uv run pytest -q — 132 passed
  • Focused Ruff check and format check on every changed Python file
  • Deterministic schema regeneration; V1 schema files unchanged
  • Wheel and sdist built successfully
  • Clean built-wheel import and V1/V2 schema smoke passed
  • Built archives contain the four overlay schemas and MIT license; no customer evidence, app-specific recipe, corpus, or private tuning data

@abrichr
abrichr marked this pull request as ready for review July 25, 2026 22:56
@abrichr
abrichr merged commit a1ed02e into main Jul 25, 2026
1 check passed
@abrichr
abrichr deleted the feat/overlay-target-tracking-v2 branch July 25, 2026 22:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant