internal: resolve state after build phase - #6788
Merged
Merged
Conversation
Reorder so the deployment-state resolution (open direct state, DMS fetch/open, deployment-history enforcement, InitIDs, --select, and the --plan file load) runs after phases.Build instead of before it. The order becomes FastValidate -> Validate -> Build -> resolve-state -> deploy. No behavior change: nothing between FastValidate and Build consumes the opened state, the auto-migration still runs post-deploy, and FastValidate/Validate/Build issue no workspace API calls, so recorded request order is unchanged too. Only the code position moves; the --plan handling is split into a Build-flag toggle (before Build) and the plan load (in the post-build state block). Motivation: prep for moving the terraform->direct migration before deploy (#6749). That migration converts the bundle config into the direct state and must run after Build, because library and ${artifacts.*} references (e.g. `whl: ./dist/*.whl`) are only resolved during Build -- running it earlier bakes unexpanded globs into the migrated state. Co-authored-by: Isaac <no-reply@databricks.com>
denik
changed the title
denik
added a commit
that referenced
this pull request
Sep 22, 2026
#6788 extracts the ProcessBundleRet reorder (resolve state after phases.Build) as a standalone no-op refactor. Stack this PR on top of it: keep this branch's process.go (the same reorder plus the migration wired into the post-build state block), so the diff over #6788 is just the migration behavior. Co-authored-by: Isaac <no-reply@databricks.com>
shreyas-goenka
approved these changes
Sep 22, 2026
Collaborator
Integration test reportCommit: fd10082
Top 3 slowest tests (at least 2 minutes):
|
The --select rejection only needs the pulled engine (stateDesc.Engine), which PullResourcesState sets before Build. Keep it in the first shouldReadState block, matching main, so this reorder does not touch it and --select on terraform is still rejected early. Co-authored-by: Isaac <no-reply@databricks.com>
Collaborator
Integration test reportCommit: 62e3fae
109 interesting tests: 107 FAIL, 2 flaky
Top 50 slowest tests (at least 2 minutes):
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changes
Reorder the steps in
ProcessBundleRet. The single pre-buildshouldReadStateblock is split in two: the state pull (which determines the engine) stays up front, while opening the resolved state moves to after Build.Old: pull-state → open-state → FastValidate → Validate → Build → deploy
New: pull-state → FastValidate → Validate → Build → open-state → deploy
where:
PullResourcesState— determine the engine, setMigratingToDirect, print the deploy notice, and validate--select. Needs only the pulled engine descriptor, so it stays before FastValidate (unchanged).InitIDs, and the--planfile load. Needs the opened/resolved state, so it moves to after Build.The reorder is safe because the two sides are independent:
(
Validatemay issue workspace API calls — that's why it is separate fromFastValidate— but it does not need the state, so running it before open-state is fine.) The auto-migration still runs post-deploy, so behavior is unchanged.Why
Prep for #6749 — move the terraform→direct migration before deploy: that migration converts the bundle config into the direct state and must run after Build, because library and
${artifacts.*}references (e.g.whl: ./dist/*.whl) are only resolved during Build. Running migration before Build would bake unexpanded globs into the migrated state instead of resolved remote paths. Landing this reorder on its own keeps #6749's diff to the actual behavior change.This pull request and its description were written by Isaac.