WIT: declare as a package-scope type, apply with `@annotate`

[yordis]

Picking up #58, which ends with an invitation to bring something concrete. What changed since is that WIT now carries three hardcoded @-attribute productions with three different sets of position rules: gate (#332), implements (#613), and external-id (#672), the last two merged this year. #388 wants a fourth for user-defined gate strings, #454 proposes @non-exhaustive(u16), #235 floats @get/@set, and in that same thread @lukewagner anticipates shared, async, and reentrant as future function attributes. #332's own read in 2024 was that a generic annotation syntax "opens a can of worms that require a lot more thought and discussion". Since then the same surface has grown one hardcoded attribute at a time.

The idea in one sentence: an annotation is a package-scope type, and a usage is a WAVE value of that type. That shape is the first reply on #58, where @lukewagner floated "defining syntax for literal values of all the interface value types" alongside "CapnProto ... explicit declarations and validation". Both halves now exist: WAVE is moving into this repository in #639, and Cap'n Proto style declarations need only a placement, which is what #694 is for.

Declaring:

package tooling:rust@1.0.0;

@annotation(declarations = [record, variant, enum])
type derive = list<string>;

@annotation(declarations = [record-field, func])
type rename = string;

@annotation(declarations = [func], retention = binary)
type alias-of = list<string>;

The payload type does what a bespoke parameter list would: record for named arguments, list<T> for multiplicity, option<T> for optionality, enum or flags for a closed vocabulary. The reason it is a type and not a parameter list is that deployed annotations outgrow parameter lists. google.api.http carries a oneof over six verbs plus a repeated HttpRule additional_bindings field referring to its own type. A oneof is a variant, and recursion needs a name to point at, so neither survives flattening into an argument list. Google also got several bindings per method from a repeated field inside the payload rather than from a repeatable option, which is most of why I think list<T> is enough and no repeatability flag is needed. Two from my own protos, on positions: a UUID-derivation option attaches both to an enum and to individual enum values and takes a payload type imported from another file; a NATS one declares two annotations in one package with a different payload type per position, which is what declarations is for and which protobuf makes mandatory in practice since you have to pick something to extend. Both declarations and retention are named fields on the marker, and that argument list is the extension point: fields can be added over time without another grammar change, so a first cut does not have to carry all of them, and declarations is the most readily deferred. Deferring it also defers declaration-kind below, the largest piece of new machinery here, so the mechanism could land on payload types alone and grow position rules later.

Two things get conflated in that kind of staging though, and only one of them is reversible. Adding a field later is free. Deciding what omission means is not, so it has to be settled from the start even if the field itself arrives later. Java's @Target and C#'s AttributeUsage both left omission meaning effectively everywhere, and neither can narrow that default now without breaking existing code. So I would rather require declarations from the get-go, since that is the only version of this with no irreversible default in it. Deferring it stays available, but only while annotations are gated and unreleased: declarations would have to become required before the feature stabilizes, because a released bare @annotation fixes "anywhere" as the answer permanently.

Applying, where the argument is a WAVE value:

@annotate(tooling:rust/derive, ["Hash", "Eq"])
record object-id {
  @annotate(tooling:rust/rename, "objectID")
  id: string,
}

@annotate(tooling:rust/alias-of, ["get-environment"])
initial-cwd: func() -> option<string>;

Only string payloads are available without #639, since string-literal is already in the grammar (WIT.md:1118) and @external-id already takes one. Structured payloads are what needs WAVE.

On namespace, nothing changes about who owns bare @word, and that is the point. #332 already established the invariant: "all @[tag name]s (other than @feature and @since) are still reserved after this PR since they are all rejected by the WIT parser." The set is now four with @external-id. This proposal takes two more, @annotation on the declaration and @annotate on the usage, and then stops: everything user-defined is named by a package-qualified path passed as an argument, never by a bare word, so unknown bare names stay a hard error and the reserved space stays the platform's alone to expand over time. That is what I think makes this a safe expansion rather than a competing one, and it matters because the queue is long: #388, #454, #235 and the shared/async/reentrant set above can all still land as bare @words later without colliding with anything a user wrote. Package-qualified identity also makes collisions between user annotations impossible, with versioning from ordinary package deps rather than a registry. On typed versus untyped I am taking the answer already reached in #58's thread, where @Pauan documented Rust's token-soup rule and concluded "that flexibility is probably overkill for Wasm".

retention is the part I most want feedback on, and I do not think it is a new axis. #58 drew it on 2022-12-21, separating annotations "only meant to be meaningful to a particular language or host", fine "as long as we say you can always strip these sorts of annotations", from ones that "logically extend the URL, adding a data payload that is passed to the host", and ending "perhaps this use case should be considered separately". The thread went quiet. Per-annotation retention is that same line, drawn by the annotation author instead of the spec picking a side and shipping two features. #307 wants the same for an instance-reuse hint, a defined section rather than a custom one so it "couldn't be indiscriminately stripped", and Smithy is the deployed version: @stevelr noted in that thread that @sensitive is consumed at runtime by logging libraries while @required is codegen-only.

For binary encoding I would start from what exists rather than #58's custom-section-versus-name fork. Explainer.md:2613 already defines attribute ::= versionsuffix | implements | externid on both import and export, and Explainer.md:1107 says attributes there are ignored by type checking. That satisfies #58's round-trip requirement ("Wit should be co-expressive with component types so that we can render an arbitrary component's type as Wit and also do a rough roundtrip"), and #672 shipped with round-tripping in wasm-tools. The caveat I would want help with is @alexcrichton's point on #613: attribute work has already forced bindings generators off the WIT AST onto a nominal post-processed form, so a retained annotation has to survive that too.

On positions, I would want attachment allowed everywhere from the start, including where @ is rejected today. Verified against wasm-tools 1.255.0: gates work on interface and world items, on resource declarations and on resource methods, and are a parse error on record fields, variant cases and enum cases, which is exactly where #58's motivating examples live (deprecate a field, gate a new case), where #312 wants parameters and where #454 wants non-exhaustiveness. Unrelated bug noticed while checking: WIT.md's resource-method production has no gate prefix, yet wasm-tools accepts gates there and WASI interfaces depend on it. Happy to send that one-line fix separately.

Positions I am taking rather than asking about, so they can be argued with:

• No propagation in the general mechanism. #559 wants @since on an interface to reach its items and @lukewagner raised the same during #332, but that debate is about what silence should mean for an ungated item, which is gate semantics rather than metadata inheritance. Better settled in #559.
• No repeatability flag and no defaults. list<T> and option<T> cover multiplicity and optionality without a merge policy, and on defaults @badeend's argument in #15 applies: a default that participates in subtyping has to be known to glue code, which makes it a type-system feature.
• Not proposing @get/@set from #235. That stays its thread's call; this would only be the mechanism it could ride.
• #256 fits as the thing it was always the fallback for. #273 deleted a doc-comment @ production with "if it takes us too long to make a real proposal, we could go with your soft reservation idea". @bbb651's point in #533 is the reason to prefer the grammar: comment-level annotations are not syntax errors when malformed, so every tool ends up with its own dialect.

What I am unsure about:

1. The encoding details, given the round-trip requirement and the nominal-form constraint above.
2. Whether declarations belongs in a first cut, or whether the mechanism should ship on payload types alone and add position rules once there is deployment experience. I care less about which than about not letting a bare @annotation mean "anywhere" on the way there, since that is the part that cannot be taken back. If the field does land, it is cleanest as list<declaration-kind> with no universal case, since any and [record, enum] cannot both be values of one type, and protobuf gets by without one.
3. declaration-kind itself, if declarations lands: a closed enum of WIT declaration forms (record, func, record-field, variant-case, and so on). Nothing unified exists today; wit-parser has TypeDefKind, WorldItem and FunctionKind as fragments. Worth defining as general reflection rather than something annotation-private?

If the shape seems reasonable I am happy to write it up the way #58 asked, syntax in WIT.md, encoding in Binary.md, semantics in a new Annotations.md, alongside a wasm-tools PR the way #672 was done. The 🏷️ attributes are not released yet, so this seems like the moment to ask whether the next one should be general.

[lukewagner]

Having had #58 stewing in the back of my mind for the last few years, I've come to believe that we definitely need some sort of user-defined annotation mechanism in WIT for things that we can't or shouldn't "bake in" to the CM (as we've done, for good reason, with implements and the other built-in annotations) as well as for early pre-bake-in experimentation.

What had stumped me in #58 was how to resolve these two tensions:

1. do we have a simple, but structure-less, "list of key+value strings" or do we do something more-structured and if so... what?
2. is it just a free-for-all global namespace of annotation names or do we do something fancier like all these other IDLs seem to do and if so... what?

But I think your proposal has pretty great answers for both:

1. we use WIT value types and WAVE literals, which are already implemented in various places, are already being proposed to standardize (Add WAVE format to this repository #639), and already have other reasons to define as a subset of WIT (eq-bound-value imports/exports 🪙; maybe default param values)
2. we reuse the existing namespace:package/item scheme we're already using for interfaces and worlds and, with WIT: allow type declarations at package scope #694, structural types

To your open questions:

• I'd definitely prefer to leave declarations out of a first-cut and save it as a follow-up feature to be independently motivated with use cases. I can't imagine a (no-argument) @annotation having any other default than all, so it seems like a fine default (if we want to allow all at all).
• Because we'll need to use a typeidx and valueidx to encode @annotate, I think this belongs as a new attribute (alongside implements et al) which means it goes in the import/export definition (in the text and binary format), roundtrips, and wouldn't get stripped by default.
• Not-stripped-by-default seems like a reasonable default retention policy, so I'd also prefer to leave retention out of a first-cut, waiting for motivating use cases for a follow-up feature addition.

I'm interested to hear what other folks think, but personally I'm pretty excited by the general shape of this and would be happy to help spec it out (along with #694) if it's seeming like the right general approach.