test(e2e-real): watchdog thermal-hold leg — first live coverage (#349) - #384
Merged
Conversation
tests/run.sh has always driven watchdog() through stubs (systemctl, curl, and _read_temp all faked), so the over-temp stop has never fired against a real sensor, a real systemd, or a real miner. Add a 'watchdog' phase: it lowers max_temp_c below the live reading (the same _read_temp the verb itself calls, with a 5°C margin to absorb the restart's brief dataset-init dip), runs `rigforge.sh watchdog` once, asserts the stop + hold marker + an independent journal witness, then restores config, state files, and the running service under the same snapshot/trap-on-EXIT shape control()/upgrade() already use. The wedge-restart half (2-strike, unreachable API) is deliberately not exercised here — forcing that on a production-adjacent rig risks the same "stranded miner" outcome control()'s own skipped rollback leg calls out; see #276 for that path against a stubbed apply. No rigforge.sh changes — the coverage gate is untouched.
…ail on an unresolvable worker root, clamp the cutoff ceiling (#349)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #349.
Adds a
watchdogphase to tests/e2e-real.sh (also wired intoall, between upgrade and perf): the thermal-hold path run for real for the first time — tests/run.sh driveswatchdog()entirely through stubs, so the over-temp stop had never fired against a real sensor, a real systemd, or a real miner.The leg: snapshot config.json and install the cleanup trap FIRST; read the live temperature via the same
_read_tempthe verb itself calls (so the leg can never disagree with the verb about "the current reading"); lowermax_temp_cto 5°C below it through the normalapplypath; runrigforge.sh watchdogonce; assert three independent witnesses — the over-temp stop log line, the thermal-hold marker file, and a stop/deactivate entry in the systemd journal. Cleanup restores the config snapshot, removes thermal-hold/strike state directly (a config restore alone could leave a stale hold), re-applies, and verifies the service is active — on every exit path. Skips explicitly (never silently) when no temperature reading is available, or when the rig idles too close to the 40°C schema floor for a safe margin. The wedge-restart half stays stubbed-only by design: forcing an unreachable API on a shared rig is the "stranded miner" risk the file already declines elsewhere (#276).rigforge.sh: zero changed lines — no new coverage surface.
What was run:
shellcheck --severity=warningon all tracked *.sh,shfmt -i 4 -d,bash -n, andmake lint: clean.bash tests/run.shforeground: pass (~4 min).sudo bash tests/e2e-real.sh watchdog).🤖 Generated with Claude Code