Hacker News Reader

Comment by snek_case | Hacker News Reader

> rust demands that I cross every last t before I can run it at all.

It's worse than that IMO. Rust makes it very awkward/impractical to have cyclic data structures, which are necessary to write a lot of useful programs. The Rust fans will quickly jump in and tell you that if you need cycles, your program is wrong and you're just not a good enough programmer, but Maybe it's just that the Rust borrow checker is too limited and primitive, and it really just gets in the way sometimes.

Some of the restrictions of the Rust borrow checker and type system are arbitrary. They're there because Rust currently can't do better. They're not the gospel, they aren't necessarily inherent property that must always be satisfied for a program to be bug free. The Rust notion of safety is not an absolute. It's a compromise, and a really annoying, tiresome drain on motivation and productivity sometimes.

View on HN

The basic model of Rust is to move use-after-free from a dynamic, runtime check to a static, compile-time check. But to keep the static checks from being Turing-complete, you need to prohibit arbitrary cycles while something like a tree (or other boundable recursion) is doable. So Rust not being able to check cyclic data structures isn't a "Rust currently can't do better" situation, it's a "Rust just can't do better" situation.

What Rust's intended solution for that is that you add in data structures that do the dynamic checking for you in those cases. But the Rust library doesn't provide anything here that's useful (RefCell is the closest alternative, and that's pretty close to a this-is-never-what-you-want datatype), which means your options are either to use integers, roll your own with unsafe, or try hard to rewrite your code to not use cycles (which is usually a euphemism for use integers anyways). The problem here, I think, is that there is a missing data structure helper that can sit in between integers and references, namely something akin to handles (with a corresponding allocator that allows concurrent creation/deletion of elements).

missing data structure helper -- didn't you already just name-check that though, since that's basically RefCell .. or if you're willing to roll the dice... UnsafeCell (aka "trust me I know what I'm doing")?

What you essentially want for the user to not write any unsafe code is this kind of interface:

   trait Allocator {
     fn allocate<'a, T>(&'a self, init: T) -> Handle<'a, T>;
     fn deallocate<'a, T>(&'a self, handle: Handle<'a, T>);
     fn read<T>(&self, handle: Handle<'_, T>) -> impl Deref<T>;
     fn write<T>(&self, handle: Handle<'_, T>) -> impl DerefMut<T>;
   }

&'a RefCell<T> is pretty close to a definition of Handle<'a, T>, except that Rust provides no implementations of allocate and deallocate that take a const instead of a mut reference for self. Trying to make an allocator that lets you safely deallocate something requires a completely different implementation of Handle<'a, T> than what RefCell can provide, and even if you're fine without deallocation, allocation with a const ref still requires unsafe to get the lifetime parameter right.

Can you clone a Handle? If so, how do you handle using a clone after freeing it? If clones are refcounted, how do you handle cycles?

There are several different ways you can implement a Handle, depending on what features you want; the most important part of its implementation is that `fn is_valid(handle: Handle) -> bool` is possible. The simplest implementation is a (pointer, generation) pair, which can be packed into a u64 pretty easily even for 64-bit systems; every allocation and deallocation increments the generation counter in the allocator, and is_valid is thus implemented by checking if the allocator's generation matches the claimed generation for a Handle. This kind of Handle is effectively a Copy implementation (not merely Clone!).

Effectively, handles are like weak pointers in that you can detect when the underlying object has been freed, but unlike weak pointers, there's no need for a reference counter to know when to deallocate the object--the object is freed when the allocator itself dies, or it can manually be freed earlier. It is possible to write code that will attempt to use the freed object, and the compiler will be happy, but the runtime will detect that it has been freed and panic instead. (RefCell does something similar, except it only detects violations of multiple readers xor one writer requirement, not overall lifteime). You can also add other wrappers around Handles to automatically free those Handles on scope exit, but the point is you can now have multiple references to an object that can be upgraded to a mutable reference if you desire.

Did you get a look at https://github.com/rust-lang/rfcs/pull/3446 at all?

I'm not in the habit of regularly following new Rust RFCs, so I'd have no way of knowing about something made just last week. :-) But I'm taking a look now.

I don't tend to follow them either, but I've been frustrated by the lack of progress on allocator_api, and I came across this yesterday after looking into that. I only mention it because the Handle stuff in there looked tangentially related, though it's talking about something quite a bit different than you were.

> The Rust fans will quickly jump in and tell you that if you need cycles, your program is wrong and you're just not a good enough programmer

I have absolutely never heard a Rust fan say this. AFAICT the fact that cyclic data structures are hard to write is widely accepted within the community as one of the negative tradeoffs of the language.

If you’re talking to people who claim that any language is better than all others in every possible way, for every possible use case, then they are zealots whose opinion can be ignored.

I would never tell you that you are wrong to have cyclic data structures. But there are reasonable workarounds like using handles into an array to do it, which of course re-creates some of the same problems as pointers, but not the worst ones, and is often a positive for performance on modern hardware due to improved data locality.

Or you can use reference counted types and take a small performance hit.

Or use unsafe and git gud.

> currently can't do better

The limitations are an inherent consequence of basic tenets of Rust's design. Rust wouldn't be Rust anymore if you fixed them.

> Some of the restrictions of the Rust borrow checker and type system are arbitrary. They're there because Rust currently can't do better. They're not the gospel, they aren't necessarily inherent property that must always be satisfied for a program to be bug free. The Rust notion of safety is not an absolute. It's a compromise, and a really annoying, tiresome drain on motivation and productivity sometimes.

Yeah, but this actually seems consistent with the philosophy behind Rust: to take away the tools a programmer needs for creativity, so they couldn't do potentially costly mistakes, as applicable to big teams in huge corporations. Another commenter in this thread put it nicely: the borrow checker is a straitjacket for the programmer.

It's not meant to foster creativity, it's meant to be safe for big business and novice employees.

> It's not meant to foster creativity, it's meant to be safe for big business and novice employees.

Interestingly, my experience is the opposite.

I find that the "straightjacket" is extremely precious during refactorings – in particular, the type of refactorings that I perform constantly when I'm prototyping.

Compared to this, I'm currently writing Python code, and every time I attempt a refactoring, I waste considerable amounts of time before I can test the interesting new codepath, because I end up breaking hundreds of other codepaths that get in the way and I need to go through the testsuite (and pray that it contains a sufficient number of tests) hundreds of time until the code is kinda stable.

Which is not to say that Rust matches every scenario. We agree that it doesn't, by design. But I don't think that the scenarios you sketch out are the best representation of what Rust can/should be used for and can't/shouldn't be used for.

Basically it should be left for scenarios where any kind of automatic memory management isn't allowed, either for technical reasons, or because it is a lost battle trying to change the mindset of the target group.

For everything else there are more productive options.

Cyclic data structures are implemented easily with unsafe. Like non-cyclical ones (Vec for example). The difficult part is to make a safe API to that. This difficulties are not of syntactic nature but design difficulties. You need to think through your use cases for such a struct and to devise an API that supports them.

This is more difficult than C++ way "just do it". With C++ you will solve the same problems but on a case by case basis as they come into view. With Rust you need to solve these problems upfront or do a lot of refactoring later. There are upsides and downsides in both approaches, but it is clear that Rust is not good to sketch some code quickly to see how it will do.

It is still possible to do it quickly with Rust in a C++ way by leaking usafety everywhere and passing raw pointers, but I think it is still easier to do it with C++ which was designed for this style of coding.

Reply on news.ycombinator.com