DAB deploys with zero actual config changes still trigger a SETTINGS_CHANGE cancel+restart on continuous pipelines

[geirantonsen]

Describe the issue

A bundle deploy that makes no actual configuration change to a continuous
pipeline still causes the Pipelines service to cancel the pipeline's active
update and start a new one, with reason SETTINGS_CHANGE. This happens on
effectively every deploy of a bundle containing a continuous pipeline, even
when that specific pipeline's resource definition is byte-for-byte unchanged
between deploys.

This looks like the same class of bug as #6076, #6060 and #6315 — an
input-only or computed field on the pipeline resource that isn't registered
as such, so the direct engine perpetually diffs bundle state against a
remote value that structurally can never match, and replays an update every
run. Unlike those, the field responsible hasn't been identified yet; filing
this to track it and get guidance on capturing the right evidence.

Configuration

Minimal repro shape (the exact field causing the perpetual diff is not yet
isolated — happens on any continuous pipeline with a libraries.notebook
based DLT pipeline, no gateway/ingestion-specific config required):

resources:
  pipelines:
    example_stream:
      name: example_stream
      catalog: my_catalog
      target: my_schema
      continuous: true
      development: false
      channel: CURRENT
      edition: PRO
      libraries:
        - notebook:
            path: ../src/pipelines/example/pipeline

Steps to reproduce the behavior

1. databricks bundle deploy -t <target> with a continuous pipeline in the bundle, no source changes since the last deploy.
2. Observe the pipeline's event log / update history immediately after deploy.
3. See a new update start with cause: SETTINGS_CHANGE, cancelling whatever update was previously running.

Expected Behavior

No update call (and therefore no cancel/restart) should occur against a
continuous pipeline when nothing about its resolved configuration actually
changed.

Actual Behavior

The pipeline is cancelled and a new update starts on every deploy.

OS and CLI version

CLI v1.17.0, Linux.

Is this a regression?

Yes, in scope: one continuous ingestion pipeline (Lakeflow CDC gateway) already
showed this behavior under the Terraform-based engine. After migrating this
bundle to the direct deployment engine, every continuous pipeline in the
bundle now exhibits the same behavior — not just that one.

Detailed plan

Can provide a redacted bundle plan -o json -t <target> on request — happy
to capture it if a maintainer confirms this is the most useful next
diagnostic, and to redact accordingly.

Debug Logs

Can provide on request, redacted.

[denik]

Thanks for the report @geirantonsen ! Yes, please post or send to dabs-feedback at databricks.com the output of "bundle plan -o json"

[antgei]

Updated: Now we cannot reproduce the behaviour, neither from local machine nor bitbucket pipeline. Not sure why. Still the CDC pipeline restarts. BTW (ref PR): The channel and edition values are all uppercase in our bundle config.

[antgei]

The CDC pipeline was fixed by explicit setting target in conf. Case can be closed, but still not sure why it self-healed after the first direct mode deployments.