fix(python-setup): resolve a missing serverless version instead of aborting

[rugpanov]

Why

Pressing the uv-native "set up Python environment" entry with serverless attached
but no serverless version persisted showed:

Select a cluster or serverless compute before setting up the environment.

…and aborted — asking the user to select compute they had already selected, with
no hint that the missing piece was a version. The feature was unreachable for
those projects.

That state is not just a legacy leftover; four live paths produce it:

1. A serverlessComputeId: auto config enables serverless via updateServerless
   without ever opening the version picker.
2. With the feature flag off, selectServerless() enables serverless with no
   version — enabling the flag later leaves the project version-less.
3. A persisted version failing isSupportedVersion (e.g. a hand-edited "7") is
   silently dropped to undefined on load, then dead-ends on the same message.
4. A pre-existing or hand-edited vscode.overrides.json with serverless: true
   and no serverlessVersion key — the original repro.

What

Ask for the version on the spot, using the scorer and picker that already exist,
persist it with the serverless selection so the question is asked once, then
continue the run.

• resolveComputeFrom stays pure but returns a tagged ComputeResolution, with
  needsServerlessVersion as its own state rather than another undefined.
• The resolveCompute seam returns a separate ResolvedCompute union whose
  cancelled case is distinct from none: dismissing the prompt stops the flow
  silently and records nothing, so the no_compute metric keeps meaning "the CTA
  had nothing to do" instead of absorbing deliberate bail-outs.
• Persistence reuses connectionManager.enableServerless(version). That call
  only records the version while serverless is still enabled — then it does not
  re-attach compute, fire a compute-change event, or re-report
  COMPUTE_SELECTED. No ConnectionManager change was needed. Because the
  guarantee is conditional, resolveCompute re-reads the attachment after the
  prompt (below) rather than trusting the pre-prompt snapshot.
• The observation collector moved off ConnectionCommands into
  serverlessVersionObservations.ts, so compute selection and the setup flow
  share one collect → score → pick pipeline.

Prompting sits after the visibility gate and before the attempt is recorded, so
we never prompt where the feature isn't offered and user think-time stays out of
the setup-duration metric. A failed config write still lets the run proceed —
persistence only buys "don't ask again", so losing it shouldn't cost the user the
run they asked for.

The picker doesn't set ignoreFocusOut, so compute can change while it is open.
resolveCompute therefore re-classifies attachedCompute() once the prompt
returns and persists only while the state it asked about still holds; otherwise it
skips the write and returns the fresh resolution, so the run follows what is
attached now. Without that re-read the write is destructive rather than merely
stale: attachCluster calls disableServerless(), so enableServerless takes
its full branch and detaches the cluster the user just picked. The same re-read
stops a version that landed from elsewhere (a concurrent compute-picker run, a
config reload) from being overwritten by the answer to a stale question.

Path 3 above is fixed as a side effect: an out-of-range persisted version now
prompts instead of dead-ending.

Backward compatibility

The whole feature is behind databricks.experiments.optInto containing
environment.pythonSetup, so there is no change for anyone without the opt-in.
The only persisted-state write is serverlessVersion — an already-shipped
overrideable key that selectServerless writes today — into
.databricks/bundle/<target>/vscode.overrides.json. No format change, no
migration. Both unions are internal to python-setup/.

Verification

yarn test:unit — 492 passing, up from 475 on the base commit. The 6
remaining failures (CLI embedding, profile loading) are pre-existing on main
and byte-identical to the baseline captured before these changes.
yarn test:lint clean.

17 new tests cover: prompt + persist; dismissal staying silent with zero
telemetry; the three paths that must never prompt (cluster attached, nothing
attached, serverless already versioned); a failed persist still running; the
collector's per-source guards in both directions; and the three
compute-changed-during-the-prompt cases (cluster attached, everything detached, a
version recorded elsewhere), each of which fails without the post-prompt re-read.

Not yet manually verified against a live workspace — that needs the opt-in plus a
project with serverless: true and no serverlessVersion.

This pull request and its description were written by Isaac.

[rugpanov]

🤖 Integration tests triggered for fc849394 — ⏳ running.
View run

[rugpanov]

🤖 Integration tests triggered for 55129ea5 — ⏳ running.
View run

[github-actions]

If integration tests don't run automatically, an authorized user can run them manually by following the instructions below:

Trigger:
go/deco-tests-run/vscode

Inputs:

• PR number: 2088
• Commit SHA: 1a564a075e1db11294f82b3672b6d6b9e056c3d4

Checks will be approved automatically on success.

[rugpanov]

🤖 Integration tests triggered for 1a564a07 — ⏳ running.
View run