Skip to content

fix(nodetask): stop re-submitting gov/sidecar tasks every poll#450

Merged
bdchatham merged 1 commit into
mainfrom
fix/nodetask-stop-resubmit
Jul 6, 2026
Merged

fix(nodetask): stop re-submitting gov/sidecar tasks every poll#450
bdchatham merged 1 commit into
mainfrom
fix/nodetask-stop-resubmit

Conversation

@bdchatham

Copy link
Copy Markdown
Collaborator

What

The nodetask controller never advanced status.task.status off Pending, so the == TaskPending submit guard re-invoked SubmitTask on every 15s poll for a task's whole life. Deterministic task IDs make that idempotent only while the sidecar retains the task; if its store is lost, the next poll re-submits — a double-submit / double-deposit hazard for non-idempotent gov proposals (GovSoftwareUpgrade/GovParamChange).

Change

  • Add a TaskRunning value to the TaskStatus enum (additive; the SeiNode plan executor never writes it).
  • Set TaskRunning after a successful Execute, so later reconciles poll instead of re-submitting.
  • Regenerated CRDs/deepcopy; regression test asserts submit-once → poll-only.

By-design notes (resolved for PR 2b)

  • A submit whose status flush then fails still re-submits once on requeue — pre-existing, now the only window (shrunk from every 15s cycle).
  • After this fix, a sidecar store-loss parks the task in Running rather than resubmitting — the safer behavior for non-idempotent proposals; PR 2b's inclusion re-check resolves it.

Standalone bug fix — no seictl dependency. First slice of the SeiNodeTask gov UX work.

Reviewed: idiomatic-reviewer, systems-engineer (correct, ship it). Design: sei-protocol/bdchatham-designs#98.

🤖 Generated with Claude Code

The nodetask controller never advanced status.task.status off Pending, so the
== TaskPending submit guard re-invoked SubmitTask on every 15s poll for the
task's whole life. Deterministic task IDs make that idempotent only while the
sidecar retains the task; if its store is lost the next poll re-submits — a
double-submit / double-deposit hazard for non-idempotent gov proposals.

Add a TaskRunning enum value and set it after a successful submit so later
reconciles poll instead of re-submitting. Terminal transitions are unchanged
(the status switch overwrites unconditionally).

By design (resolved for PR 2b): a submit whose status flush then fails still
re-submits once on requeue (pre-existing, now the only window); and after this
fix a sidecar store-loss parks the task in Running rather than resubmitting —
safer for non-idempotent proposals, resolved by 2b's inclusion re-check.

Reviewed: idiomatic-reviewer, systems-engineer. Design: bdchatham-designs#98.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@cursor

cursor Bot commented Jul 6, 2026

Copy link
Copy Markdown

PR Summary

Medium Risk
Changes sidecar-backed SeiNodeTask lifecycle and CRD validation for task status; reduces double-submit risk for non-idempotent gov proposals but leaves a narrow re-submit window if status flush fails after submit.

Overview
Fixes a bug where SeiNodeTask sidecar work (gov vote, software upgrade, etc.) could call SubmitTask on every ~15s reconcile because status.task.status never left Pending, so the submit guard fired on each poll.

Adds TaskRunning to the shared TaskStatus enum (CRDs/manifests regenerated) and sets it in driveTask immediately after a successful Execute, so later reconciles poll only via GetTask.

Adds a regression test that asserts one submit and continued status polls without re-submit.

Reviewed by Cursor Bugbot for commit bb2d2f8. Bugbot is set up for automated code reviews on this repo. Configure here.

@bdchatham bdchatham merged commit cf7c349 into main Jul 6, 2026
5 checks passed
@bdchatham bdchatham deleted the fix/nodetask-stop-resubmit branch July 6, 2026 16:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant