Skip to content

nail down the final validity rules: references and unions #2337

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
RalfJung wants to merge 2 commits into
base: master
Choose a base branch
from
Open

nail down the final validity rules: references and unions #2337

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
93d25cc
nail down the final validity rules: references and unions
RalfJung Aug 23, 2026
c8ce01e
make 'inhabited' check account for dynamic information in metadata
RalfJung Aug 24, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 12 additions & 4 deletions src/behavior-considered-undefined.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -145,10 +145,15 @@ r[undefined.validity.struct]
* A `struct`, tuple, and array requires all fields/elements to be valid at their respective type.

r[undefined.validity.union]
* For a `union`, the exact validity requirements are not decided yet. Obviously, all values that can be created entirely in safe code are valid. If the union has a [zero-sized] field, then every possible value is valid. Further details are [still being debated](https://github.com/rust-lang/unsafe-code-guidelines/issues/438).
* For a `union`, there are no validity requirements. All byte sequences are valid union values.

r[undefined.validity.reference-box]
* A reference or [`Box<T>`] must be aligned and non-null, it cannot be [dangling], and it must point to a valid value (in case of dynamically sized types, using the actual dynamic type of the pointee as determined by the [metadata]). Note that the last point (about pointing to a valid value) remains a subject of some debate.
* A reference or [`Box<T>`] must be aligned and non-null, it cannot be [dangling], and the pointee type `T` must be *inhabited*.

The exact classification of inhabited types is unspecified, similar to the size and alignment of types.
However, types that can be constructed from safe code are definitely inhabited.
For unsized types, this check considers dynamic information from the metadata:
In particular, if `T` has an unsized tail of slice type `[U]`, and if `U` is uninhabited, and if the length encoded in the metadata is non-zero, then the pointee is uninhabited.
Comment on lines +155 to +156

@WaffleLapkin WaffleLapkin Sep 8, 2026 •
edited by rustbot
Loading

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like making inhabitedness depend on metadata values is not right / clean.

I would rather make this a separate restriction on the value of the metadata. That is, I would say something along the lines of "A reference or Box<T> must ... have valid metadata" and "slice metadata for a slice type with uninhabited elements, it must be 0".

This allows adding other restrictions in the future, such as the one mentioned by @scottmcm that slice metadata can at most be usize::MAX / size_of::<T>() (for non-ZSTs).

View changes since the review

@RalfJung RalfJung Sep 8, 2026 •
edited
Loading

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We generally pretty consistently treat the metadata as telling us the "actual type" that we use to inspect the pointee. I think it would be odd do do something else here. It would look like an arbitrary set of rules, rather than just something that falls out of a more general principle.

This allows adding other restrictions in the future, such as the one mentioned by @scottmcm that slice metadata can at most be usize::MAX / size_of::() (for non-ZSTs).

As I mentioned in reply to Scott, that restriction is already present. It follows from the fact that the reference has to be dereferenceable for its actual dynamic size.

@WaffleLapkin WaffleLapkin Sep 8, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm. It still feels weird to me to do it this way for slices, but if we are consistently doing it this way I suppose that is better.

I suppose the reason it feels weird to me is that in the compiler "inhabitedness" is a property of types, and does not concern the value in any way. If we would like exploit this UB in the compiler, we would have to add an arbitrary rule in the form of "if the tail is a slice of an uninhabited type, assume(len == 0)".

In a way, considering the "actual type" feels more arbitrary to me.

That being said, I imagine we have to consider the "actual type" for trait objects, at which point it's better do the same thing for slices too.

(I'd resolve this conversation, but it appears I don't have permissions for that)

@RalfJung RalfJung Sep 8, 2026 •
edited
Loading

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The compiler can only statically approximate all these properties using the types, yes. But that's expected, everything on this page is written assuming full knowledge of the current dynamic state (think: Miri, MiniRust). The compiler might get better at approximating things and that should not change the spec!

IOW, I think you're thinking too much like a compiler writer and not enough like a language specifier. :)

@WaffleLapkin WaffleLapkin Sep 9, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

guilty as charged I'm afraid ^^'

@RalfJung RalfJung Sep 9, 2026 •
edited
Loading

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we would like exploit this UB in the compiler, we would have to add an arbitrary rule in the form of "if the tail is a slice of an uninhabited type, assume(len == 0)".

To also reply to this specifically -- the compiler does not have to exploit this UB if we think it's too weird or doesn't fit the current compiler architecture. But I would like to keep the door open to the compiler potentially exploiting this UB in the future, as the compiler architecture may change and make this less weird. Maybe one day it becomes trivial to make it so that the length metadata of &[!] is annotated with range information saying it has to be 0. This seems quite plausible, don't we already encode that the length is at most isize::MAX for non-zero-sized types?

@asquared31415 asquared31415 Sep 9, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in many contexts we do encode that the length is 0..=floor(isize::MAX/size_of::<pointee>), in particular function arguments. in a few context we do not (due to perf regressions and dubious benefit), and we never do for unsized tails, it's only &[T].

for example, fn meow(x: &[u16]) is i64 noundef range(i64 0, 4611686018427387904) for the length arg, but struct Meow { val: [u16] } fn meow(x: &Meow) is a plain i64 noundef.

@WaffleLapkin WaffleLapkin Sep 10, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@RalfJung I was not arguing against adding arbitrary rules to the compiler (it's full of them!), I was more so arguing that it should be fine to have them here too (which I don't believe anymore); thanks for explaining your view though :)

@asquared31415 do you know why we are only adding range metadata for &[T] and not &Meow? Seems like an oversight... Turns out this metadata is calculated with a special case in cg_llvm: https://github.com/rust-lang/rust/blob/c4c4a576936e9e67717d0deb8e74e02dd5dd10de/compiler/rustc_codegen_llvm/src/abi.rs#L561-L579 As the comment suggests, I don't think this is a good way to handle this, hm...

@asquared31415 asquared31415 Sep 10, 2026 •
edited
Loading

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I looked into it as a part of another PR, and it ended up regressing codegen in most cases, with minimal benefits: rust-lang/rust#159921 (comment) unfortunately I don't have numbers on hand, but llvm was making some bad assembly involving 64 bit immediates and sometimes even using a whole extra register (in a contrived test, but I'm unsure if it would go away even after inlining in normal usage)

The attributes are set here in rustc_codegen_llvm, where it matches only Ref(Slice(T)) types, and there's even a note that it's suboptimal.

During the aforementioned PR, I even hit the layout cycles that the comment alludes to, so it's definitely still an issue as of july-ish.

@asquared31415 asquared31415 Sep 10, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@scottmcm might have some additional context as the original author of that hack. They mentioned rust-lang/rust#152843 could help get some of the layout info we would like on DSTs, but IIRC I also touched that code in similar ways (with nuw and/or nsw on that math) and ended up getting similar "llvm suddenly emits 64 bit literals and extra registers".


r[undefined.validity.wide]
* The [metadata] of a wide reference, [`Box<T>`], or raw pointer must match the type of the [unsized tail]:
Expand Down Expand Up @@ -195,8 +200,11 @@ r[undefined.validity.const-provenance]
> };
> ```

r[undefined.validity.undef]
**Note:** Uninitialized memory is also implicitly invalid for any type that has a restricted set of valid values. In other words, the only cases in which reading uninitialized memory is permitted are inside `union`s and in "padding" (the gaps between the fields of a type).
> [!NOTE]
> The definition above implies that uninitialized memory is invalid everywhere except inside `union`s and in "padding" (the gaps between the fields of a type).

> [!WARNING]
> Just because a value is *valid* does not mean that it is *safe to use*. A value being *valid* merely means that creating the value does not cause immediate undefined behavior; such UB can still be caused later, even by safe operations. For instance, consider that `&str` pointing to initialized non-UTF8 data is *valid* as defined above, but passing such a value to a safe function can cause undefined behavior. As a more extreme example, consider that `&[u8]` pointing to allocated but uninitialized memory is *valid*, but one can trivially cause UB simply by accessing an element of the slice. Generally, only values you can construct in safe code are *safe to use*.

[`bool`]: types/boolean.md
[`const`]: items/constant-items.md
Expand Down
Loading