Skip to content

Keep browser correctness fast and bound scatter timing phases - #154

Merged
tannerlinsley merged 1 commit into
mainfrom
taren/lean-browser-validation
Sep 15, 2026
Merged

tannerlinsley merged 1 commit into
mainfrom
taren/lean-browser-validation

Conversation

@tannerlinsley

@tannerlinsley tannerlinsley commented Sep 15, 2026

Copy link
Copy Markdown
Member

Changes

  • Run first-party browser correctness weekly, with the same 188 cases, viewport/theme matrix, revisions, geometry assertions, and native interaction scenarios. Skip competitor builds and comparative type, bundle, and timing audits in this mode.
  • Run exhaustive comparisons monthly or manually. Keep ordinary PRs unchanged, with opt-in browser labels and targeted manual case runs.
  • Split raw-scatter timing into independent mount and update trials for every renderer. Keep every warmup, sample, correctness check, lifecycle soak, and the 120-second deadline per phase. Retry only a failed phase, and retain its errors and timing evidence.
  • Fix the example issues exposed by the retained checks: responsive dashboard layout, outside radar labels, negative-bar label spacing, pinned mark counts, and cumulative stacked-radial values. Add exact count bounds and focused regressions.

Evidence

The original stress failure was cumulative work, not a stalled operation. Hosted diagnostics completed successive Recharts updates until the whole-cell deadline: phase trace, sample trace.

The hosted first-party sweep passes 188/188 layout cases and 22/22 interaction cases on this head. Summed browser job execution time is 14m 48s, down from 32m 01s in the previous comparative sweep, about 54% less. This excludes queue time and billing rounding. The longest shard fell from 4m 47s to 2m 20s.

The hosted standard benchmark passes all 100 cells with zero correctness failures and zero recovered retries. All four comparison jobs and the bundle policy pass. Stress job execution totals 30m 45s versus 33m 19s before. The longest partition is 11m 07s versus the previous failed 10m 20s run, so this improves completion and total runner time, not the slowest standard partition. Quick PR stress jobs passed in 2m 11s, 1m 31s, 1m 46s, and 1m 11s. The full static graph took 6m 13s on the PR. These single-run observations include runner/cache variance, not a guarantee of steady-state speedups.

Type checking, focused tests, example-source validation, and the existing bundle policy pass. No package runtime, bundle budget, workflow timeout, or npm version changes.

Tradeoff: routine validation no longer checks competitor paint parity or relative geometry similarity. Those checks remain in monthly/manual comparisons. Raw-scatter phase contexts add setup overhead, but avoid discarding and retrying already completed trials.

@coderabbitai

coderabbitai Bot commented Sep 15, 2026

Copy link
Copy Markdown

Review Change StackReview Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Advanced

Run ID: 417c44b8-d9d7-4e9e-a9b9-7a29c6de0fea

📥 Commits

Reviewing files that changed from the base of the PR and between 29ed879 and 051e675.

⛔ Files ignored due to path filters (3)
  • benchmarks/conformance/previews/150-shadcn-bar-negative.svg is excluded by !**/*.svg
  • benchmarks/conformance/previews/181-shadcn-radar-label-custom.svg is excluded by !**/*.svg
  • benchmarks/conformance/previews/189-shadcn-radial-stacked.svg is excluded by !**/*.svg
📒 Files selected for processing (25)
  • .github/workflows/conformance.yml
  • API-FRICTION.md
  • benchmarks/conformance/README.md
  • benchmarks/conformance/cases/127-shadcn-dashboard/dashboard.tsx
  • benchmarks/conformance/cases/127-shadcn-dashboard/styles.ts
  • benchmarks/conformance/cases/143-shadcn-bar-active/case.json
  • benchmarks/conformance/cases/149-shadcn-bar-mixed/case.json
  • benchmarks/conformance/cases/150-shadcn-bar-negative/example.tsx
  • benchmarks/conformance/cases/156-shadcn-line-interactive/case.json
  • benchmarks/conformance/cases/181-shadcn-radar-label-custom/example.tsx
  • benchmarks/conformance/cases/189-shadcn-radial-stacked/case.json
  • benchmarks/conformance/cases/189-shadcn-radial-stacked/example.tsx
  • benchmarks/conformance/catalog-index.json
  • benchmarks/conformance/previews/manifest.json
  • benchmarks/conformance/shared/shadcn-catalog-recharts.tsx
  • benchmarks/conformance/shared/shadcn-data-contracts.test.ts
  • benchmarks/conformance/shared/shadcn-reference-accessibility.test.ts
  • scripts/benchmark/stress-phases.mjs
  • scripts/benchmark/stress-phases.test.mjs
  • scripts/compare-plot-catalog-helpers.mjs
  • scripts/compare-plot-catalog-helpers.test.mjs
  • scripts/compare-plot-catalog.mjs
  • scripts/conformance-workflow.test.mjs
  • scripts/generate-shadcn-cases.mjs
  • scripts/stress-chart-libraries.mjs

Included review availability: Your plan provides up to 4 included reviews per hour; 3 remain after this review.


📝 Walkthrough

Walkthrough

Changes

Conformance execution and cases

Layer / File(s) Summary
Workflow scheduling and invocation
.github/workflows/conformance.yml, scripts/conformance-workflow.test.mjs, benchmarks/conformance/README.md, API-FRICTION.md
The workflow now runs weekly first-party checks and monthly comparisons. Manual runs support suites and targeted case IDs.
First-party conformance harness
scripts/compare-plot-catalog.mjs, scripts/compare-plot-catalog-helpers.mjs, scripts/compare-plot-catalog-helpers.test.mjs
The harness adds --first-party mode, TanStack-only checks, first-party reports, and inspection validation.
Shadcn case rendering and contracts
benchmarks/conformance/shared/*, scripts/generate-shadcn-cases.mjs, benchmarks/conformance/cases/*, benchmarks/conformance/catalog-index.json
Browser data, series counts, radial totals, and geometry limits now match the updated shadcn examples.
Layout fixes and regenerated previews
benchmarks/conformance/cases/127-shadcn-dashboard/*, benchmarks/conformance/cases/150-shadcn-bar-negative/*, benchmarks/conformance/cases/181-shadcn-radar-label-custom/*, benchmarks/conformance/previews/manifest.json
Dashboard container queries, scroll-viewport marking, chart spacing, radar sizing, and preview metadata were updated.

Stress benchmark timing isolation

Layer / File(s) Summary
Timing phase orchestration
scripts/benchmark/stress-phases.mjs, scripts/stress-chart-libraries.mjs
Raw-scatter timing now runs mount and individual update phases in separate browser contexts and aggregates their results.
Timing phase validation tests
scripts/benchmark/stress-phases.test.mjs
Tests cover invalid results, phase order, failure propagation, recovery data, and non-timing workloads.

Priority: ➖ Normal

Estimated code review effort: 4 (Complex) | ~60 minutes

Change: Feature

Merge Risk: ⚪ Minimal · up to 051e6

No concrete merge-blocking risk remains; the updated conformance and timing paths preserve their required contracts.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 4.76% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 21 functions across 16 files. (9 skipped: … Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title directly describes the two main changes: keeping browser correctness validation fast and bounding raw-scatter timing phases.
Full details: Docstring Coverage

Explanation

Docstring coverage is 4.76% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 21 functions across 16 files. (9 skipped: 9 unsupported.)

  • Fix all pre-merge checks with AI
✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch taren/lean-browser-validation

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@nx-cloud

nx-cloud Bot commented Sep 15, 2026

Copy link
Copy Markdown

View your CI Pipeline Execution ↗ for commit 051e675

Command Status Duration Result
nx run charts-workspace:ci-distributed ✅ Succeeded 4m 21s View ↗
nx run charts-workspace:package-check ✅ Succeeded <1s View ↗
nx run charts-workspace:benchmark-check ✅ Succeeded 1m 8s View ↗

☁️ Nx Cloud last updated this comment at 2026-09-15 18:25:57 UTC

@tannerlinsley
tannerlinsley merged commit 327f488 into main Sep 15, 2026
39 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant