From 8f047d4fc729c465fd9b77ca0966a64c2528d680 Mon Sep 17 00:00:00 2001 From: Till Varoquaux Date: Wed, 10 Jun 2026 22:45:18 -0400 Subject: [PATCH 1/2] PEP 9999: Shorthand syntax for Annotated type metadata --- peps/pep-9999.rst | 482 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 482 insertions(+) create mode 100644 peps/pep-9999.rst diff --git a/peps/pep-9999.rst b/peps/pep-9999.rst new file mode 100644 index 00000000000..2366c8234ec --- /dev/null +++ b/peps/pep-9999.rst @@ -0,0 +1,482 @@ +PEP: 9999 +Title: Shorthand syntax for Annotated type metadata (Draft Proposal) +Author: Till Varoquaux +Sponsor: Ivan Levkivskyi +Discussions-To: https://discuss.python.org/t/106888 +Status: Draft +Type: Standards Track +Topic: Typing +Created: 19-Apr-2026 +Python-Version: 3.16 +Post-History: 19-Apr-2026 + +Abstract +======== + +This proposal introduces a shorthand syntax for ``typing.Annotated`` using the +``@`` operator. This change reduces verbosity for type annotations with +metadata, benefiting libraries like **Pydantic**, **FastAPI**, **Typer**, and +**SQLModel**. + +Motivation +========== + +Since its introduction in :pep:`593`, ``Annotated`` has become the standard +mechanism for attaching context-specific metadata to types. It is widely +embraced by libraries such as **Pydantic**, **FastAPI**, and **SQLModel**. +Historically, these libraries embedded metadata within default values, a pattern +that was concise but problematic for type checkers and runtime defaults:: + + # The older, now discouraged pattern + class User(BaseModel): + id: int = Field(gt=0) + name: str = Field(min_length=3) + +The transition to ``Annotated`` cleanly separated type from metadata but +introduced visual noise and cognitive overhead. Library authors often surface +"special" types, like ``PositiveInt`` or ``EmailStr``, to hide this verbosity. +These aliases are discoverable but inherently limited. Users needing unique +constraint combinations must fall back to the full ``Annotated`` syntax:: + + from typing import Annotated + from pydantic import BaseModel, Field, PositiveInt + + class User(BaseModel): + # Concise but limited + age: PositiveInt + + # Verbose fallback required for specific constraints + id: Annotated[int, Field(gt=0, le=1000)] + name: Annotated[str, Field(min_length=3, max_length=50)] = "Anonymous" + +This creates a jarring experience. ``Annotated`` is core to the ecosystem but +remains "hidden" and difficult to use directly. The proposed shorthand bridges +this ergonomic gap. It restores the conciseness of earlier patterns while +adhering to the modern ``Annotated`` standard. By reducing overhead, this +proposal encourages developers to leverage the full power of type metadata:: + + from pydantic import BaseModel, Field + from fastapi import Query + + class User(BaseModel): + id: int @ Field(gt=0, le=1000) + name: str @ Field(min_length=3, max_length=50) = "Anonymous" + age: int @ Field(gt=0) + email: str @ Field(pattern=r".*@.*") + + async def read_items(q: (str | None) @ Query(max_length=50) = None): + ... + +Rationale +========= + +Developer Ergonomics and Ecosystem Alignment +--------------------------------------------- + +Pydantic and FastAPI now recommend ``Annotated`` over embedding metadata in +default values. However, the resulting code is significantly more verbose. +This proposal restores the earlier pattern's conciseness while adhering to the +modern ``Annotated`` standard. + +Sebastián Ramírez (author of FastAPI, Typer, and SQLModel) noted that a shorter +syntax without extra imports would benefit users. By making the "correct" way +the most ergonomic, we reduce the incentive for discouraged patterns. + +This ergonomic barrier was notably evident in the withdrawal of :pep:`727` +(Documentation Metadata). The extreme verbosity of the syntax in function +signatures was a primary factor in its community pushback. A native shorthand +makes such metadata-heavy standards significantly more viable. + +Conceptual Consistency and Precedent +------------------------------------- + +The ``@`` operator signifies "decoration" or "attachment of metadata" for +functions and classes. Extending this to type expressions leverages that +mental model: just as a decorator attaches behavior to a function, the ``@`` +operator attaches metadata to a type. + +This follows the precedent set by :pep:`604` (``|`` for ``Union``) and +:pep:`585` (generics in built-ins). These PEPs moved common typing constructs +into native operators, making the type system feel like a first-class part of +the language. + +This syntax also draws inspiration from other languages with strong metadata +ecosystems, notably Java. In Java (formalized in `JSR 308 `_) +and other JVM languages, the ``@`` symbol is standard for type annotations:: + + public class Person { + @Column(length = 32) + private String name; + } + +While the exact syntax differs (Python's ``@`` operates inline on the type +expression rather than decorating the declaration), the visual association +between the ``@`` symbol and type-level metadata will be familiar to many +developers. + +Implementation and Performance +------------------------------- + +Making the syntax built-in eases runtime metadata use by removing ``typing`` +module import overhead. This aligns with the trend toward accessible runtime +type introspection. + +The proposed syntax is straightforward to implement. Prototypes for Mypy, +Pyright, and Ruff are compact. Since ``@`` is already a valid expression +operator, these tools do not require parser changes. They handle the new syntax +during semantic analysis. Ruff has already prototyped a ``pyupgrade`` rule +(``UP051``) for automated conversion. This enables large codebases to +migrate to the new syntax with minimal manual effort. + +CPython prototype testing confirms that libraries like ``typer`` and +``pydantic`` work out of the box. + +Specification +============= + +The proposed syntax uses the ``@`` (matrix multiplication) operator to attach +metadata to a type:: + + # Current syntax + x: Annotated[int, Range(0, 10)] + + # Proposed shorthand + x: int @ Range(0, 10) + +Operator Precedence +------------------- + +The ``@`` operator has higher precedence than the ``|`` operator (bitwise OR, +used for Unions in :pep:`604`). Parentheses are required when attaching +metadata to a Union type: + +- ``int | str @ Metadata`` is equivalent to ``int | Annotated[str, Metadata]`` +- ``(int | str) @ Metadata`` is equivalent to ``Annotated[int | str, Metadata]`` + +This matches the standard precedence of ``@`` and ``|`` in Python expressions. +The most common union pattern, ``Optional``, works naturally: + +- ``int @ Field(gt=0) | None`` is equivalent to + ``Annotated[int, Field(gt=0)] | None`` + +Flattening and Associativity +---------------------------- + +The ``@`` operator is left-associative. When multiple metadata items are +chained, the resulting ``Annotated`` object is flattened. + +Specifically, ``T @ m1 @ m2`` is strictly equivalent to +``Annotated[T, m1, m2]``. It must not resolve to a nested structure such as +``Annotated[Annotated[T, m1], m2]``. This mirrors the existing runtime +behavior of ``typing.Annotated``. + +This flattening also applies when the left-hand operand is an existing +``Annotated`` type, regardless of how it was constructed:: + + Annotated[int, m1] @ m2 # AnnotatedType(int, m1, m2) — flattened + +Runtime Behavior +---------------- + +The ``@`` operator produces a ``types.AnnotatedType`` instance, a new built-in +type implemented in C. The existing ``typing.Annotated`` is unified with this +type: ``typing.Annotated[X, Y]`` returns the same ``types.AnnotatedType`` +object as ``X @ Y``:: + + >>> type(int @ Field()) is type(Annotated[int, Field()]) + True + >>> typing.Annotated is types.AnnotatedType + True + +An ``AnnotatedType`` object exposes the following attributes: + +- ``__origin__``: The base type (e.g., ``int``). +- ``__metadata__``: A tuple of metadata items. +- ``__args__``: The tuple ``(origin, *metadata)``, for compatibility with + ``typing.get_args()``. +- ``__parameters__``: Lazily computed type variables contained in the type. + +The ``repr()`` of an ``AnnotatedType`` uses the shorthand syntax:: + + >>> int @ Field(gt=0) + int @ Field(gt=0) + +``__copy__`` and ``__deepcopy__`` are not supported on ``AnnotatedType`` +objects and will raise ``AttributeError``. This is a deliberate design +choice to avoid ambiguity around shared mutable metadata. + +``AnnotatedType`` objects support pickling via ``copyreg``, reconstructing +through ``AnnotatedType[origin, *metadata]``. + +``None`` on the left-hand side is accepted and uses ``NoneType`` as the +origin:: + + >>> None @ Field() + None @ Field() + +Supported Left-Hand Operands +----------------------------- + +The ``@`` operator is implemented by adding ``nb_matrix_multiply`` to the +metatype (``type``) and to several typing-related types. The operator fires +when the left-hand operand is any of the following: + +- A type object (any class, including built-ins like ``int`` and ``str``) +- ``None`` (treated as ``NoneType``) +- ``types.UnionType`` (e.g., ``int | str``) +- ``types.GenericAlias`` (e.g., ``list[int]``) +- ``typing.TypeVar``, ``typing.ParamSpec``, ``typing.TypeVarTuple`` +- ``typing.TypeAliasType`` +- ``types.AnnotatedType`` (for chaining) + +For all other left-hand operands, the operator returns ``NotImplemented``, +allowing normal ``__matmul__`` dispatch to proceed. + +Parsing and Grammar +=================== + +This proposal requires no changes to the Python grammar. Because ``@`` is +already a valid operator, it is natively parsed as a binary operation. The +shorthand is resolved during semantic analysis, entirely bypassing the need +to patch grammar files or update the parser. + +How to Teach This +================= + +In Python, the ``@`` symbol already has an established association with +metadata through decorators. The annotation shorthand extends this +intuition to the type system: ``int @ Field(gt=0)`` reads as "``int``, +decorated with ``Field(gt=0)``." + +For beginners, the key rule is simple: **in a type annotation, ``@`` means +"with this metadata."** The full ``Annotated[int, Field(gt=0)]`` syntax +remains available and is entirely equivalent for those who find it clearer. + +For experienced developers, the precedence rules follow standard Python +operator precedence (``@`` binds tighter than ``|``), and chaining +``T @ m1 @ m2`` flattens exactly as nested ``Annotated`` does. + +Documentation and teaching materials should introduce the shorthand alongside +``Annotated``, not as a replacement. The longhand form is still preferred in +contexts where multiple metadata items are passed as a group and chaining +would be unwieldy. + +Backwards Compatibility +======================= + +Forward References and Deferred Evaluation +------------------------------------------- + +Under :pep:`749`, annotations are lazily evaluated. The ``annotationlib`` +module provides several formats for retrieving annotations: + +- ``Format.VALUE``: Fully evaluates the annotation. Raises ``NameError`` + if any name is unresolvable. +- ``Format.FORWARDREF``: Wraps unresolvable names in ``ForwardRef`` objects. + However, compound expressions using operators (``@``, ``|``) produce an + opaque ``ForwardRef`` string containing the entire expression. +- ``Format.STRING``: Returns the raw source text with no evaluation. + +For the ``@`` operator, ``Format.FORWARDREF`` is insufficient. Consider:: + + class Model: + ref: "NotYetDefined" @ Field(gt=0) + +Under ``Format.FORWARDREF``, this produces +``ForwardRef('"NotYetDefined" @ Field(gt=0)')``. The metadata ``Field(gt=0)`` +is trapped inside the unresolved string and cannot be inspected until the +forward reference is resolved. This is a blocking issue for libraries like +Pydantic and FastAPI, which inspect metadata at class-definition time. + +This proposal introduces a new format, ``Format.FORWARDREF_STRUCTURAL``, +which evaluates compound type expressions **structurally**. When a name +cannot be resolved, only that name is wrapped in ``ForwardRef``; the +surrounding operators are still evaluated. The example above produces:: + + AnnotatedType(ForwardRef('NotYetDefined'), Field(gt=0)) + +The metadata is immediately accessible. This format also resolves the +pre-existing issue with ``|`` unions, where ``"Foo" | int`` under +``Format.FORWARDREF`` produces ``ForwardRef('Foo | int')`` instead of the +structural ``ForwardRef('Foo') | int``. + +Interaction with PEP 563 +^^^^^^^^^^^^^^^^^^^^^^^^^ + +Under :pep:`563` (``from __future__ import annotations``), all annotations +are stored as source-code strings and evaluated via ``eval()`` on access. The +``@`` shorthand works correctly in this context: ``eval("int @ Field(gt=0)")`` +triggers the metatype's ``nb_matrix_multiply`` and produces an +``AnnotatedType``. + +However, ``FORWARDREF_STRUCTURAL`` reconstruction from PEP 563 strings is +coarser than from :pep:`749` thunks. When a name is unresolvable, the +``ForwardRef`` may wrap a call expression (e.g., ``ForwardRef('Field(gt=0)')``) +rather than just a name. :pep:`749` provides a strictly better experience and +is the recommended path forward. + +Operator Overloading +-------------------- + +The ``@`` operator is currently used for matrix multiplication +(``__matmul__``). The shorthand is implemented by adding +``nb_matrix_multiply`` to the metatype (``type``), so it applies when a +**type object** (class) appears on the left-hand side — not when an instance +does. + +This means ``int @ Field()`` produces an ``AnnotatedType``, while +``42 @ something`` is unaffected and follows normal ``__matmul__`` dispatch. +Crucially, ``ndarray @ Field()`` (using the **class** as a type annotation) +also produces an ``AnnotatedType``, even though ndarray *instances* define +``__matmul__`` for matrix multiplication. This is the desired behavior: applying the +``@`` operator to a class object evaluates as type metadata; applying it to +an instance performs arithmetic. + +The only case where the shorthand does not apply is when a class has a +**metaclass** that defines ``__matmul__``. In that case, the metaclass's +operator takes priority via standard Python MRO dispatch. This is an obscure +edge case unlikely to arise in practice. + +typing.Annotated Migration +--------------------------- + +This proposal replaces the pure-Python ``typing._AnnotatedAlias`` class with +a native C implementation (``types.AnnotatedType``). ``typing.Annotated`` +becomes a reference to this C type rather than a special form with a custom +metaclass. + +The private ``typing._AnnotatedAlias`` class is retained as a deprecated +compatibility shim. Code using ``isinstance(x, typing._AnnotatedAlias)`` +will continue to work but emit a ``DeprecationWarning``. The shim is +scheduled for removal in Python 3.23. + +Code that should be updated: + +- ``type(ann).__name__ == '_AnnotatedAlias'`` → use + ``isinstance(ann, types.AnnotatedType)`` or + ``typing.get_origin(ann) is Annotated`` +- ``typing._AnnotatedAlias(origin, metadata)`` → use + ``Annotated[origin, *metadata]`` or ``origin @ m1 @ m2`` + +Backporting via typing_extensions +---------------------------------- + +Unlike ``X | Y`` (which could be backported by ``typing_extensions`` using +``__or__``), the ``@`` shorthand requires changes to the metatype +(``type.__matmul__``), which cannot be patched from pure Python. The +shorthand is therefore only available on Python 3.16+. The existing +``Annotated[X, Y]`` syntax continues to work on all supported versions and +should be used when backwards compatibility is required. + +Rejected Ideas +============== + +Mandatory List Variant +---------------------- + +The syntax ``Type @ [ann1, ann2]`` was considered to group metadata and avoid +chaining ambiguities. While clearer in some contexts, it was deprioritized in +favor of the cleaner ``Type @ ann1 @ ann2``. + +List-based syntax +----------------- + +An alternative syntax using list literals, such as ``[int, Metadata]``, was +rejected due to runtime semantics. In Python, a list literal evaluates to a +mutable ``list`` instance. Allowing lists as type annotations would break the +assumption of runtime checkers (like Pydantic) that annotations evaluate to +valid type constructs or ``GenericAlias`` objects, not arbitrary data +structures. + +Scientific Computing Conflict +----------------------------- + +Critics note that ``ndarray @ Metadata`` visually resembles matrix +multiplication on a type whose instances are heavily associated with that +operation. However, the ``@`` +operator distinguishes between **type objects** and **instances**: ``ndarray`` +(the class) appearing in a type annotation is a type object, and ``@`` +produces an ``AnnotatedType``. An ``ndarray`` instance appearing in an +expression still uses NumPy's ``__matmul__`` for matrix multiplication. + +Since type annotations and arithmetic expressions occupy distinct syntactic +positions, this is a visual concern rather than a runtime conflict. + +Divergence from Type Theory +--------------------------- + +Unlike ``Union`` or ``Generics``, using an operator for metadata is a +Python-specific ergonomic choice rather than a standard type-theoretic +construct. This follows the pragmatic precedent of :pep:`604`. + +Usage Examples +============== + +Pydantic Validation +------------------- + +The shorthand excels in data validation scenarios:: + + from pydantic import BaseModel, Field, HttpUrl + from annotated_types import Len + + class Project(BaseModel): + name: str @ Field(title="Project Name") @ Len(1) + url: HttpUrl @ Field(description="The project homepage") + stars: int @ Field(ge=0) = 0 + +FastAPI Dependency Injection +---------------------------- + +In FastAPI, the shorthand simplifies complex parameter definitions:: + + from fastapi import FastAPI, Header, Depends + + app = FastAPI() + + @app.get("/secure") + async def secure_endpoint(token: str @ Header(description="Authentication token")): + return {"status": "authorized"} + +SQLModel and Database Definitions +--------------------------------- + +SQLModel relies heavily on ``Annotated`` to define column properties. The +shorthand syntax makes these definitions significantly cleaner:: + + from sqlmodel import SQLModel, Field + + class Hero(SQLModel, table=True): + id: (int | None) @ Field(primary_key=True) = None + name: str @ Field(index=True) + secret_name: str + age: (int | None) @ Field(index=True) = None + +Reference Implementation +======================== + +Prototype implementations are available for the following tools: + +- **CPython:** `CPython at-type-annot `_ +- **CPython (with annotation-lib structural forward references):** `CPython forward-stringifier `_ +- **Mypy:** `Mypy at-type-annot `_ +- **Mypyc/ast_serialize:** `ast_serialize at-type-annot `_ +- **Pyright:** `Pyright at-type-annot `_ +- **Ruff:** `Ruff at-type-annot `_ + +References +========== + +- `Discussion on Python Discourse `_ +- :pep:`563` -- Postponed evaluation of annotations +- :pep:`585` -- Type hinting generics in standard collections +- :pep:`593` -- Flexible function and variable annotations +- :pep:`604` -- Allow writing union types as ``X | Y`` +- :pep:`727` -- Documentation metadata in typing (Withdrawn) +- :pep:`749` -- Implementing PEP 649 + +Copyright +========= + +This document is placed in the public domain or under the +CC0-1.0-Universal license, whichever is more permissive. From c2374091bf8aeb0ba625cd0c467bf86075bfccc8 Mon Sep 17 00:00:00 2001 From: Till Varoquaux Date: Thu, 11 Jun 2026 22:24:30 -0400 Subject: [PATCH 2/2] Address review feedback --- peps/pep-9999.rst | 35 +++++++++++++---------------------- 1 file changed, 13 insertions(+), 22 deletions(-) diff --git a/peps/pep-9999.rst b/peps/pep-9999.rst index 2366c8234ec..05ed84e2432 100644 --- a/peps/pep-9999.rst +++ b/peps/pep-9999.rst @@ -32,7 +32,7 @@ that was concise but problematic for type checkers and runtime defaults:: id: int = Field(gt=0) name: str = Field(min_length=3) -The transition to ``Annotated`` cleanly separated type from metadata but +The transition to ``Annotated`` cleanly separated types from metadata but introduced visual noise and cognitive overhead. Library authors often surface "special" types, like ``PositiveInt`` or ``EmailStr``, to hide this verbosity. These aliases are discoverable but inherently limited. Users needing unique @@ -125,7 +125,7 @@ The proposed syntax is straightforward to implement. Prototypes for Mypy, Pyright, and Ruff are compact. Since ``@`` is already a valid expression operator, these tools do not require parser changes. They handle the new syntax during semantic analysis. Ruff has already prototyped a ``pyupgrade`` rule -(``UP051``) for automated conversion. This enables large codebases to +for automated conversion. This enables large codebases to migrate to the new syntax with minimal manual effort. CPython prototype testing confirms that libraries like ``typer`` and @@ -201,14 +201,10 @@ The ``repr()`` of an ``AnnotatedType`` uses the shorthand syntax:: >>> int @ Field(gt=0) int @ Field(gt=0) -``__copy__`` and ``__deepcopy__`` are not supported on ``AnnotatedType`` -objects and will raise ``AttributeError``. This is a deliberate design -choice to avoid ambiguity around shared mutable metadata. - ``AnnotatedType`` objects support pickling via ``copyreg``, reconstructing through ``AnnotatedType[origin, *metadata]``. -``None`` on the left-hand side is accepted and uses ``NoneType`` as the +``None`` on the left-hand side is accepted and uses ``None`` as the origin:: >>> None @ Field() @@ -218,16 +214,9 @@ Supported Left-Hand Operands ----------------------------- The ``@`` operator is implemented by adding ``nb_matrix_multiply`` to the -metatype (``type``) and to several typing-related types. The operator fires -when the left-hand operand is any of the following: - -- A type object (any class, including built-ins like ``int`` and ``str``) -- ``None`` (treated as ``NoneType``) -- ``types.UnionType`` (e.g., ``int | str``) -- ``types.GenericAlias`` (e.g., ``list[int]``) -- ``typing.TypeVar``, ``typing.ParamSpec``, ``typing.TypeVarTuple`` -- ``typing.TypeAliasType`` -- ``types.AnnotatedType`` (for chaining) +metatype (``type``) and to several typing-related types. The operator is +supported for any left-hand operand that currently supports the ``|`` +operator for making a union. For all other left-hand operands, the operator returns ``NotImplemented``, allowing normal ``__matmul__`` dispatch to proceed. @@ -288,10 +277,12 @@ is trapped inside the unresolved string and cannot be inspected until the forward reference is resolved. This is a blocking issue for libraries like Pydantic and FastAPI, which inspect metadata at class-definition time. -This proposal introduces a new format, ``Format.FORWARDREF_STRUCTURAL``, -which evaluates compound type expressions **structurally**. When a name -cannot be resolved, only that name is wrapped in ``ForwardRef``; the -surrounding operators are still evaluated. The example above produces:: +This proposal introduces a new format, ``Format.FORWARDREF_STRUCTURAL``. +This format assumes typing semantics and evaluates compound type expressions +**structurally**. It always returns an ``AnnotatedType`` for ``@``, a union +for ``|``, and a ``GenericAlias`` for subscripting. When a name cannot be +resolved, only that name is wrapped in ``ForwardRef``; the surrounding +operators are still evaluated. The example above produces:: AnnotatedType(ForwardRef('NotYetDefined'), Field(gt=0)) @@ -348,7 +339,7 @@ metaclass. The private ``typing._AnnotatedAlias`` class is retained as a deprecated compatibility shim. Code using ``isinstance(x, typing._AnnotatedAlias)`` will continue to work but emit a ``DeprecationWarning``. The shim is -scheduled for removal in Python 3.23. +scheduled for removal in Python 3.18. Code that should be updated: