Skip to content

feat(rust): flat-crate proto_rust_library + prost/tonic interop fixes#425

Open
pcj wants to merge 12 commits into
masterfrom
feat/rust
Open

feat(rust): flat-crate proto_rust_library + prost/tonic interop fixes#425
pcj wants to merge 12 commits into
masterfrom
feat/rust

Conversation

@pcj

@pcj pcj commented Jun 8, 2026

Copy link
Copy Markdown
Member

Summary

Branch-level work for the Rust toolchain (feat/rust), plus a couple of general improvements to gencopy and root-file sync. The Rust changes converge on a flat-crate convention for proto_rust_library and fix several real-monorepo failures uncovered by running protoc-gen-prost, protoc-gen-prost-serde, and protoc-gen-tonic together.

Rust: flat-crate convention (7932580)

The consuming Starlark macro now exposes all generated types at the crate root, so the gazelle extension drops the corresponding plumbing:

  • No more pkg attribute on emitted proto_rust_library rules — RustLibrary.Pkg() is retained as an internal helper for Name()/Reexports() only (pkg/rule/rules_rust/rust_library.go).
  • No more _rs suffix on crate namesRustCrateName returns e.g. google_api instead of google_api_rs; the crate name is the namespace (pkg/protoc/rust_keywords.go).
  • Flattened extern_path entries — transitive deps use extern_path=.{pkg}=::{crate} and the self-extern override uses extern_path=.{pkg}=crate, dropping the nested ::module::path segments (pkg/plugin/neoeinstein/prost/extern_paths.go).

Rust: three independent prost/tonic failures (1935ee5)

  1. Service-only proto_libraries caused tonic to error with "Tried to insert into file that doesn't exist" because ProtocGenProstPlugin skipped libraries with no messages/enums. shouldApply and outputs now also fire on HasServices() so prost emits a stub .rs for tonic's insertion point (pkg/plugin/neoeinstein/prost/protoc-gen-prost.go).
  2. Non-deterministic plugin order: BUILD attr ordering could place tonic before prost, breaking the insertion. proto_compile.bzl now sorts plugins by name inside the rule impl so CLI order is independent of attr order (rules/proto_compile.bzl).
  3. Path-mismatched packages (e.g. trumid.common.auth at //trumid/common/auth/proto, grpc.health.v1 under //thirdparty/.../grpc/health/v1) failed with mv: No such file or directory. output_mappings previously only handled the Rust-keyword-escape case; generalized to emit mappings whenever RustProtocOutputDir(pkg) differs from pc.Rel (pkg/rule/rules_rust/proto_rust_library.go).

Rust: misc fixes

Root-file auto-sync (5c379a1)

After GenerateRules runs for every package, rewrite two marker-delimited regions at the repo root (no-op when markers are absent):

  • Cargo.toml between # gazelle:proto_rust_members start/end — one entry per package emitting a proto_rust_library.
  • BUILD.bazel between # gazelle:vendor_proto_sources_deps start/end — underlying _lib targets plus every proto_compiled_sources rule.

gencopy: ProTip with the .update target (b655336)

When bazel test //path:foo_proto_compile_test fails on drift, append a one-line ProTip to the mismatch error naming the .update target so it can be copy-pasted into bazel run:

ProTip: to regenerate, run:
    bazel run //proto:foo_proto_compile.update

Label derived from pkg.TargetLabel by swapping the rule-name component with cfg.UpdateTargetLabelName (cmd/gencopy/gencopy.go).

Test plan

  • bazel test //... across the gazelle extension packages
  • Generated rules build in a downstream monorepo with a mix of message-only, service-only, and path-mismatched proto packages
  • protoc-gen-prost + protoc-gen-tonic + protoc-gen-prost-serde co-execute cleanly regardless of BUILD attr order
  • WKT-only proto libraries resolve their extern paths correctly
  • Cargo.toml / root BUILD.bazel marker regions are populated on gazelle run; left alone when markers absent
  • gencopy mismatch output includes the ProTip line with the correct .update label

pcj added 11 commits May 9, 2026 12:20
…deps

After GenerateRules has run for every package, rewrite two marker-delimited
sections at the repo root:

  - Cargo.toml between `# gazelle:proto_rust_members start/end` —
    populated with one entry per package that emitted a proto_rust_library.
  - BUILD.bazel between `# gazelle:vendor_proto_sources_deps start/end` —
    populated with the underlying _lib target of each proto_rust_library
    plus every proto_compiled_sources rule.

Both rewrites are no-ops when the markers (or the target file) are absent,
so existing repos without the markers see no change. Entries are sorted
and deduplicated; the file is only rewritten when content actually changes.
…, nested extern paths

The consuming Starlark proto_rust_library macro now exposes all generated
types at the crate root, so several pieces of namespace plumbing that
the gazelle extension was emitting are no longer needed and have been
removed:

  - pkg attribute on the generated rule (rust_library.go): the macro
    doesn't accept it anymore. RustLibrary.Pkg() is retained as an
    internal helper that still feeds Name() and Reexports() — only the
    SetAttr("pkg", ...) emission is gone.

  - _rs suffix on crate names (rust_keywords.go): RustCrateName returns
    the package with dots replaced by underscores, with no trailing
    "_rs". E.g. "google.api" -> "google_api" (was "google_api_rs"). The
    crate name now IS the namespace, matching the flat-root layout.

  - Nested ::module::path on extern_path entries (extern_paths.go):
    `extern_path=.{pkg}=::{crate}::{module}` is shortened to
    `extern_path=.{pkg}=::{crate}` for transitive deps, and the
    self-extern override `extern_path=.{pkg}=crate::{module}::...` is
    shortened to `extern_path=.{pkg}=crate`. Both reflect the same
    flat-root convention.
When `bazel test //path:foo_proto_compile_test` fails because a checked-in
generated file has drifted from what `protoc` would produce, the existing
error tells you which file pair mismatched and shows the diff, but leaves
the developer to figure out which target regenerates the source-of-truth
copy.

Append a one-line ProTip to the mismatch error message, naming the
`.update` target so it can be copy-pasted into `bazel run`:

  gencopy mismatch "foo.pb.go.gen" vs. "foo.pb.go" (-want +got):
  ...

  ProTip: to regenerate, run:
      bazel run //proto:foo_proto_compile.update

The label is derived from pkg.TargetLabel (the proto_compile rule's full
label, including any external-repo prefix) by replacing the rule-name
component with cfg.UpdateTargetLabelName -- both already populated by the
proto_compile_gencopy_test Starlark rule via gencopy_config().
…mismatched pkgs

Three independent failures surfaced by running protoc-gen-prost,
protoc-gen-prost-serde, and protoc-gen-tonic together across a real
monorepo:

1. Service-only proto_libraries (no messages/enums) caused tonic to
   error with "Tried to insert into file that doesn't exist": tonic
   appends client/server code into prost's {package}.rs at the
   @@protoc_insertion_point(module) marker, but ProtocGenProstPlugin
   skipped libraries with no messages or enums. shouldApply and
   outputs now also fire on HasServices() so prost emits a stub .rs
   for tonic to insert into. Two new tests in protoc-gen-prost_test.

2. Plugin order was non-deterministic: BUILD.bazel attr ordering
   (preserved by bazel-gazelle's standard list-merge) put tonic
   before prost, so tonic's insertion ran before prost created its
   target file — yielding the "Tried to insert into file that
   doesn't exist" / "Tried to write the same file twice" pair.
   proto_compile.bzl now sorts plugins by name inside the rule impl,
   so protoc CLI ordering is independent of attr ordering.

3. proto packages whose path doesn't match the bazel package dir
   (e.g. "trumid.common.auth" living at //trumid/common/auth/proto,
   or "grpc.health.v1" living at //thirdparty/.../grpc/health/v1)
   failed with `mv: ... No such file or directory` because prost
   writes to its proto-package-derived path while proto_compile.bzl
   expected the output at the bazel pkg. The output_mappings
   computation in proto_rust_library.ProvideRule only handled the
   Rust-keyword-escape case (e.g. google.type → google/r#type).
   Generalized: emit mappings whenever RustProtocOutputDir(pkg)
   differs from pc.Rel, regardless of whether keywords are involved.
   Extracted RustProtocOutputDir helper; existing
   RustKeywordEscapeMappings retained for back-compat with the
   exposed Starlark function.
@pcj pcj changed the title feat: update generated rust crate exports to a flat namespace feat(rust): flat-crate proto_rust_library + prost/tonic interop fixes Jun 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant