Skip to content

Publish the learned-heuristic results: a Research view in the workbench - #161

Merged
guilyx merged 5 commits into
mainfrom
claude/library-demo-plots-webpage-8r8uz4
Aug 4, 2026
Merged

Publish the learned-heuristic results: a Research view in the workbench#161
guilyx merged 5 commits into
mainfrom
claude/library-demo-plots-webpage-8r8uz4

Conversation

@guilyx

@guilyx guilyx commented Aug 4, 2026

Copy link
Copy Markdown
Member

jupyddl.learn shipped in 2.3.0 with its results living only in .docs/ and
a video. The published site — the thing anyone actually reads — said nothing
about the largest addition in the release. This adds a Research view to the
workbench carrying the measured run, and fixes a defect that rendering it
exposed.

The Research view

Built from promo/rl-data.json, the cache the RL video renders from, so the
page and the video quote one measured run and cannot drift apart.
tools/build_web.py distils it into web/dist/research.json; when that cache
is absent the builder emits {} and the view says so rather than showing
numbers from nowhere.

What it reports, all measured, blocksworld trained on 3–6 blocks and judged on
9–13 under GBFS at a 30000-expansion budget:

heuristic coverage expanded seconds cost
learned 1.00 137 0.036 48.2
hff 1.00 518 0.534 51.8
goalcount 1.00 2483 0.154 51.0
blind 0.00

It deliberately does not stop at that table:

  • The per-instance spread, because the held-out set has a heavy tail. Nine
    of ten instances sit between 58 and 227; the tenth is worth thousands, and
    imitation alone could not solve it at all. The mean is close to a report of
    that one instance, so the view shows all ten.
  • Where it loses. On logistics it is beaten by hff roughly 6× — 204
    expansions against 35 — and the reason is exact, not mysterious: that domain
    has two predicates, so the feature vector is 8 numbers and can say how many
    packages are somewhere but not which package is where. Counting is blind to
    topology.
  • What we got wrong. Three corrections, kept on the page rather than in a
    footnote: the objective is flat where search is already good; an improvement
    was credited to a validation split when the perturbation scale changed in the
    same edit and was doing all of it; and the mean was doing the lying.

The defect it exposed

<main> was hidden until the worker reported ready, so every page of prose,
the requirement matrix and this new view sat behind a full-screen spinner
waiting for a 10 MB WebAssembly runtime none of them use.

The shell now renders immediately from the committed bundle — the menus and
the support matrix come from capabilities.json, which carries the same
registries the runtime reports — the boot notice is a status bar rather than a
splash screen, and only the controls that actually run a planner are gated,
now on state.ready. onReady still overwrites everything from the live
interpreter, which stays the authority.

Verified in Chromium: before the runtime lands, 14 planners, 10 heuristics, 21
requirement rows and the full Research view render with no console errors and
the Run button reads "Loading Python…" and is disabled; after it lands the
button enables, the footer reads jupyddl 2.3.0 · CPython 3.14.2 · WebAssembly, and a search returns a 17-step plan.

Tests

  • capabilities.json is pinned to the live registries. It is now rendered
    before Python loads, so a stale bundle no longer merely lags — it states
    something untrue to every visitor for the seconds before the runtime
    corrects it.
  • research.json is pinned to promo/rl-data.json.
  • A regression test that the app shell is not hidden and the gating is on
    state.ready, since that is how this broke once.
  • test_builder_is_reproducible now covers capabilities.json and
    research.json too.

418 passed, 1 skipped; flake8 and black clean.

Also carried

19204ce, unrelated and previously unmerged: the GitHub Release job no longer
requires the PyPI upload to have succeeded. A release records that a version
was cut from a verified commit; PyPI is a downstream channel that can fail for
reasons the code has nothing to do with, and losing the tag and the release to
that would mean no record of the version and a full re-run to get one. build
succeeding is the gate that matters — that is where every check lives.


Generated by Claude Code

claude added 3 commits August 4, 2026 08:02
`github-release` required `pypi` to succeed, so a failed upload took the
tag and the release down with it. That is the wrong coupling.

A GitHub Release records that a version was cut from a commit that
passed every check. PyPI is a downstream channel, and it can fail for
reasons the code has nothing to do with: auth not configured, an outage,
a rate limit. Tying the record to the upload means a transient failure
leaves no evidence the version existed and forces a full re-run to get
it back.

`build` succeeding is the gate that matters -- version guards, strict
metadata check, and a wheel installed clean and made to solve an
instance all live there. The publish stays ordered before the release so
a successful run still reflects it, and a failed one stays red and
visible rather than being swallowed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Meb35zHKsyBkH2sbWoyKMT
jupyddl.learn shipped in 2.3.0 with its results in .docs/ and a video, so
the published site said nothing about the largest addition in the release.
The workbench now carries a Research view with the measured run: imitation
against hff and goalcount, the per-instance spread behind the mean, the
logistics loss and the exact reason for it, and the three claims that
turned out to be wrong.

Its numbers come from promo/rl-data.json, the cache the RL video renders
from, so the page and the video quote one measured run and cannot drift
apart. A test pins them together, and pins capabilities.json to the live
registries.

Rendering the view exposed a defect it did not cause: <main> was hidden
until the worker reported ready, so every page of prose, the requirement
matrix and this new view sat behind a full-screen spinner waiting for a
10 MB runtime none of them use. The shell now renders immediately from the
committed bundle, the boot notice is a status bar instead of a splash
screen, and only the controls that actually run a planner are gated on
state.ready.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Meb35zHKsyBkH2sbWoyKMT
Both are in code added by the previous commit. renderResearch had grown to
107 lines covering four unrelated blocks; it is now a 17-line dispatcher
over researchHeadline, researchTable, researchSpread and researchNarrative.
The button label used a nested ternary where an if/else says the same thing
without the reader having to unpick the nesting.

No behaviour change: the view renders identically before and after, checked
in Chromium against the same measured bundle.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Meb35zHKsyBkH2sbWoyKMT
@mergify
mergify Bot requested a review from sampreets3 August 4, 2026 08:27
claude added 2 commits August 4, 2026 08:34
The `python-version` values in build.yml, pages.yml and release.yml are not
a dependency to keep at latest. They choose which interpreter builds the
wheel, bundles web/dist and cuts the release. The `tests` workflow already
runs the whole supported range on every push, and the wheel is
py3-none-any, so building on the newest interpreter buys nothing while
tying the release pipeline to whatever is newest on the runner. Moving that
pin should be a deliberate edit.

Also moves `config:base` to `config:recommended`; the former is the
deprecated spelling.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Meb35zHKsyBkH2sbWoyKMT
@guilyx guilyx mentioned this pull request Aug 4, 2026
1 task
@guilyx
guilyx marked this pull request as ready for review August 4, 2026 08:40
@cursor

cursor Bot commented Aug 4, 2026

Copy link
Copy Markdown

Bugbot is not enabled for your account, so this pull request was not reviewed.

Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs.

guilyx commented Aug 4, 2026

Copy link
Copy Markdown
Member Author

Two additions since the description was written.

CodeFactor. This PR first came back "1 issue found." The report is 403 without an account, so rather than guess at it again I looked at what my own diff had added that was worth fixing regardless: renderResearch had grown to 107 lines covering four unrelated blocks, and the Run button label used a nested ternary. Splitting the first into researchHeadline / researchTable / researchSpread / researchNarrative (a 17-line dispatcher) and replacing the second with an if/else took the status to "No issues found." — green for the first time on this repository. Re-checked in Chromium: the view renders identically before and after.

renovate.json. Carried here because it is the other half of closing #152. python-version in build.yml, pages.yml and release.yml is not a dependency to keep current — it picks the interpreter that builds the wheel, bundles web/dist and cuts the release, and tests already covers 3.9–3.14 on every push. Renovate marks those PRs immortal, so the rule is what actually stops it recurring; the reasoning is in the rule's description. Also moves the deprecated config:base to config:recommended.

Merged from main after #153, #154, #155, #156 and #159 landed. The artifact-action bump and this branch's release changes touch neighbouring lines in release.yml; they merged cleanly and tag_name / target_commitish survive alongside download-artifact@v8.


Generated by Claude Code

@guilyx
guilyx merged commit 0ca272e into main Aug 4, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants