Rust, Reflection and Access Rules | Hacker News Reader

Rust, Reflection and Access Rules

fractalfir.github.io

40 pointsmisonic22 comments

It seems a bit weird to me to dedicate all this space to talking about access to private fields and respecting field access rules, and not even suggest the idea that a reflection API should actually provide different levels of information depending on where you're invoking it from. Which is to say, if I'm writing code inside the module that defines a type, then I should be able to reflect on that type's private fields as my code can otherwise access those private fields. And if I'm writing code from outside the module then I shouldn't be able to reflect on those private fields.

One way to accomplish this would be to have reflection add a private method to a reflectable type that returns a mirror that includes information on the private fields. This does require the capability to reflect to be opt-in (and I think this article is assuming that reflection is globally enabled on all types rather than being opt-in), but reflection could plausibly be done as something like

  #[derive(Reflectable)]
  struct Foo {
      pub i: i32,
      x: i32
  }

such that this expands to something like

  struct Foo {
      pub i: i32,
      x: i32
  }

  impl Foo {
    fn private_mirror() -> Mirror {
      /* construct mirror for all fields */
    }
  }

  impl Reflectable for Foo {
    fn mirror() -> Mirror {
      /* construct mirror for public fields */
    }
  }

This way code from within the module can call `private_mirror()` to get a mirror to pass to reflection APIs (and Mirror could implement Reflectable to return itself like how Iterator implements IntoIterator) in order to do things like implement serialization.

If I learned right now that Rust had a reflection API then I’d assume that it’s fundamentally unsafe when it comes to handling non public fields. For exactly the reasons the article lists.

I think the API with safe public reflection and unsafe private reflection is easy to motivate.

The author doesn’t really provide any reason why it’s a bad idea other than “people might abuse it”. The SemVer argument I don’t understand: the versioning is for the public API. Nobody promises that somelib 1.0.0 and 1.0.1 behave remotely the same if I peek behind the curtain.

The SemVer argument is that a 1.0.0 to 1.0.1 bump shouldn't cause any downstream code to stop compiling. In this case this really just means that reflection shouldn't allow you to construct an instance of a type from its public fields if the type is annotated with #[non_exhaustive]. This is also an argument against even having an unsafe reflection API that allows access to private fields, because being able to construct an instance of a type from its private fields means depending on something that SemVer normally protects you from, and an API being unsafe isn't supposed to be an opt-out for SemVer.

You should probably have access to all fields you would otherwise be able to see (eg pub crate or private within the same module)

I didn’t follow that argument, where would reflection be used here? Was the problem that if parentlib 1.0.0 used reflection on childlib 1.0.0 then potentially bumping childlib to 1.0.1 would cause it to completely break the functionality of parentlib?

In that case yes, that’s what I’d expect would happen. And I’d still think it’s useful functionality.

SemVer says that updating childlib from 1.0.0 to 1.0.1 cannot cause parentlib to stop compiling. If the update does cause dependents to stop compiling then this must be a major version update.

Rust does have some subtle holes in its rules for SemVer here where there are changes you can make in a minor version that have the capability of breaking your caller (and this is sort of ignored by the logic that the caller could have been more explicit in how it called the method, either using Universal Function Call Syntax or, in the case of inferred types breaking, specified an explicit type, but this does still cause issues from time to time), but overall it tries hard to stick to this rule. So any official Reflection API needs to take SemVer into account. Being able to access private fields from outside the crate that defines the type is always going to be a really big SemVer issue as private fields otherwise do not affect the public API of a crate (only the presence or absence of private fields on a type matters to public API). Similarly, outside of the defining crate, reflection can't allow for the creation of a #[non_exhaustive] type from its public fields as the type may gain more fields in a non-breaking update.

What would this change be that made parentlib stop compiling?

Wouldn’t changes to the child lib just potentially stop parentlib from running properly (when an expected private field is missing or similar)?

> Reflection interacts with the safety features of Rust in a somewhat counter-intuitive way. Those interactions force any reflection API to obey certain rules.

I know enough Bevy and Rust to know that Bevy does have their own Reflection library (https://docs.rs/bevy_reflect/latest/bevy_reflect/), but I don't know the internals of it, anyone happens to know that who can compare it to what the author is ideaing about? As a Bevy user and library author, the Reflect API Bevy uses is simple enough at least.

Isn't reflection at odds with "zero-cost abstraction" principle?

We're talking about compile-time reflection here, not runtime reflection.

Not clear on the problem.

In Rust, each struct type, to be serialized, must have a serialization and deserialization function of its own. There's a macro to generate these by calling the appropriate serialization or deserialization function for each field.

For the common types, there's a set of standard serialize and deserialize functions. Here's the list.[1] Those handle the special cases around Vec, Mutex, and such.

The author doesn't make a strong case for reflection for other purposes.

[1] https://docs.rs/serde/latest/serde/trait.Serialize.html

I prototyped https://github.com/emberian/serde-reflect if anyone is interested

I get what the article is saying. I enjoy Rust, as I enjoy dabbling in other more esoteric programming languages, like Zig. But one thing I miss from the days of early programming is the hacker spirit, in a way. As it master, I should be able to force my machine into giving me access of private fields. I want to be able to poke holes in stuff, to break things and at times, I want the ability to do something stupid just because I want to see what would happen. I feel the same for overly strict compilers. The number one thing I hate about Zig is the compiler treating me like a child when I have unreferenced variables.

Modern languages should enable what older languages couldn‘t. They shouldn‘t get in my way needlessly.

unsafe is there when you need it, you can poke all the bytes and hack the planet as much as you want as soon as you acknowledge the language can't keep you safe anymore.

"As it master, I should be able to force my machine into giving me access of private fields."

If you're writing the code then you can do whatever you want with it. But if I'm writing the code you're using, I want the power to express "the end user cannot use this private field".

You trying to access the private fields in the library I or someone else wrote would be like trying to change a book someone wrote or a painting someone made. It's not the computer restricting you, it's the author of the code.

EDIT: This response was not my finest take.

I think the agreement is rather “if you change the private field then there are no longer any guarantees of anything”. Which is fine, I think. And obviously with or without reflection anyone can already modify the private field.

It's the shortest path to Hyrum's Law, certainly.

> And obviously with or without reflection anyone can already modify the private field.

This bit, I don't understand. Wouldn't that require unsafe?

EDIT: Just to say: Abstract data types usually require upholding some invariants which all the operations on those data types must uphold at entry/exit. If anyone can do whatever, then what's left othen than just a bag of bytes?

Yes. I’m 100% talking about unsafe Rust now (for private fields).

That is: I see reflection only as a way of making unsafe bytemucking a bit more ergonomic. Its nothing you can’t already do. It’s a well behaved object with invariants 99% of the time, then for some edge case/cheat it’s not. Much like the difference between safe and unsafe already is. Most of the time you rely on the guarantees, but occasionally you throw the guarantees out. And again: this IS already in an unsafe context by necessity. This has been the case in every language with encapsulation and reflection (C#, Java, …) since forever. And it’s a super useful thing. I can’t see why the encapsulation in (unsafe) Rust is more important? Is it because it has no runtime to provide guard rails where the JVM/CLR would still throw exceptions when I use my tampered object with broken invariants?

Oh, I see...

I think I was thinking about reflection in an entirely different way. When I think Reflection I think mostly static and generative reflection -- so you can introspect on data structures, derive [whatever] from that, generate code, whatever... but you still cannot do [whatever] to the thing you reflected on. (Other than whatever the language says you can do as a public consumer of that API/structure)

So you can, say, add serialization code, generate property test instances, whatever.

What you cannot do is do anything to "invade" the thing being reflected on and change its meaning.

This sort of thing seems you should be able to do already at compile time instead of using reflection?

But sometimes you might writing a debugger. Reflection is essentially including all the "debug symbols" that you need in order to build a debugger into your application.

No language since assembly enables new features that you could not have done in assembly.

Languages are invented to enforce standard and build higher level abstraction aka disable feature.

Reply on news.ycombinator.com