Skip to content

Prove workflow-to-CHASM backfill cursor replay - #42

Open
chaptersix wants to merge 1 commit into
mainfrom
migration-backfill/evidence
Open

chaptersix wants to merge 1 commit into
mainfrom
migration-backfill/evidence

Conversation

@chaptersix

@chaptersix chaptersix commented Sep 5, 2026

Copy link
Copy Markdown
Owner

Problem

Workflow-backed schedulers store each ongoing backfill's StartTime as an exclusive cursor. The CHASM migration converter currently clears the corresponding LastProcessedTime, so CHASM treats the transferred range as fresh and generates the already-processed boundary again.

Violated invariant

Migration must preserve the remaining occurrence set (exclusive cursor, inclusive end]. Every occurrence already accounted for by V1 must remain excluded, while every later occurrence must remain actionable.

Minimal reproduction

The deterministic test uses a one-minute schedule, cursor 2026-08-01T11:59:00Z, and end 12:00:00Z:

  • Native CHASM continuation emits only 12:00.
  • Migrated continuation emits 11:59 and 12:00.

Ordinary CI runs the native control and skips the known counterexample. Expose it with:

TEMPORAL_RUN_MIGRATION_COUNTEREXAMPLES=1 go test -tags test_dep ./chasm/lib/scheduler \
  -run '^TestMigrationBackfill_MigratedResumesAfterWatermark$' -count=1

Evidence

The production trace is convertBackfillsLegacyToCHASMBackfillerTaskHandler.processBackfill: the converter clears progress, then the backfiller subtracts one millisecond for a fresh inclusive range.

Scope

This layer contains evidence only; it intentionally makes no production change. The opposite CHASM-to-workflow fresh-range boundary is covered by upstream PR temporalio#11878 and is not duplicated here.

Validation

go test -tags test_dep ./chasm/lib/scheduler -run '^TestMigrationBackfill_' -count=1
TEMPORAL_RUN_MIGRATION_COUNTEREXAMPLES=1 go test -tags test_dep ./chasm/lib/scheduler \
  -run '^TestMigrationBackfill_MigratedResumesAfterWatermark$' -count=1

Stack

Rendered failure and sequence artifacts are attached below.

V1 to CHASM backfill cursor replay failure sequence

Deterministic evidence-parent regression failure

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 5, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review Completed 2026-09-05T01:02:38.365158Z c9b3351 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant