Skip to content

pip extension facts structure prevents correct git merge of MODULE.bazel.lock #4162

Description

@psalaberria002

Summary

The pip extension stores its facts with the index URL as a top-level key and package names one level deeper:

{
  "@@rules_python+//python/extensions:pip.bzl%pip": {
    "dist_hashes": {
      "https://pypi.org/simple/": {
        "package-a": { "wheel_path": "sha256:..." },
        "package-b": { "wheel_path": "sha256:..." }
      }
    },
    "index_urls": {
      "https://pypi.org/simple/": {
        "package-a": "package-a"
      }
    }
  }
}

Bazel ships a git merge driver for MODULE.bazel.lock (scripts/bazel-lockfile-merge.jq) that merges the facts section using shallow_merge, which applies last-wins semantics at the top level only (jq add).

Because dist_hashes and index_urls are single top-level keys shared by all packages, when two branches each add a different package, one branch's entire dist_hashes map overwrites the other's. The merged lockfile is missing the packages from one branch, causing:

MODULE.bazel.lock is no longer up-to-date because the extension
'@@rules_python+//python/extensions:pip.bzl%pip' has changed its facts

Root cause

This is a mismatch between the facts structure and the merge driver's semantics. I filed bazelbuild/bazel#31139 requesting a deep-merge for facts, but the Bazel team considers shallow_merge intentional — deep-merging arbitrary extension facts could be wrong for other extensions — and redirected to rules_python.

Suggested fix

If the facts structure used package names (or index_url + package name) as top-level keys, shallow_merge would correctly preserve entries from both branches:

{
  "@@rules_python+//python/extensions:pip.bzl%pip": {
    "https://pypi.org/simple/ package-a": { "wheel_path": "sha256:..." },
    "https://pypi.org/simple/ package-b": { "wheel_path": "sha256:..." }
  }
}

Any flattening that puts per-package data under distinct top-level keys would solve the problem.

Impact

This affects any team that uses a CI setup where master is merged into PR branches before running tests (common in Jenkins-based pipelines), and where two concurrent PRs each add a Python dependency.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions