Hacker News Reader

Unsafe Rust is an esoteric language without iron-clad guarantees, and type-level programming and async Rust is an esoteric metalanguage (https://hirrolot.github.io/posts/rust-is-hard-or-the-misery-...). For example, matklad made a recent blog post on "Caches In Rust" (https://matklad.github.io/2022/06/11/caches-in-rust.html). The cache is built around https://docs.rs/elsa, which is built around https://docs.rs/stable_deref_trait/latest/stable_deref_trait..., which is unsound for Box and violates stacked borrows in its current form: https://github.com/Storyyeller/stable_deref_trait/issues/15

There is a recurring trend of sound C programs turning into unsound Rust programs, because shared mutability is often necessary but it's difficult to avoid creating &mut, and Stacked Borrows places strict conditions on constructing &mut T (they invalidate some but not all aliasing *const T).

I don't think this is a great example of "sound C program turning into unsound Rust program". The crate isn't "unsound" in the way a C program would be - it's unsound in the sense that, given either 'unsafe' elsewhere or changes to how Rust constructs work (that are not guaranteed) a consumer of this crate could accidentally violate one of the necessary guarantees.

For a Rust program the bar is "has to be safe, even if some other part of the program uses unsafe". That seems like it's arguably a higher bar than C where everything is already "unsafe" in that same way.

I invested a lot of time porting some parsing code I had written to Rust, with the vision that Rust is the memory-safe future. The code I was porting from used arenas, so I tried to use arenas in Rust also.

Using arenas required a bunch of lifetime annotations everywhere, but I was happy to do it if I could get provable memory safety.

I got everything working, but the moment I tried to wrap it in Python, it failed. The lifetime annotation on my struct was a problem. I tried to work around this by using ouroboros and the self-referencing struct pattern. But then I ran into another problem: the Rust arena I was using (Bumpalo) is not Sync, which means references to the arena are not Send. All of my arena-aware containers were storing references to the Arena and therefore were not Send, but wrapping in Python requires it to be Send. I wrote more about these challenges here: https://blog.reverberate.org/2021/12/19/arenas-and-rust.html

You might say "well don't use an arena, use Box, Rc, etc." But now you're telling me to write more complicated and less efficient code, just to make it work with Rust. That is a hard pill to swallow for what is supposed to be a systems language.

I did the Crafting Interpreters book to learn Rust after I did The Rust book.

I faced the same problem as you. Somehow it felt like Arenas went beyond Rust's philosophy and added huge amounts of complexity to the interpreter.

Tree traversal and mutating the environment of a `block` internally was an issue I spent like 2-3 days on. I was porting Java code to Rust after all. Somehow got it working in a Rust way. I used unsafe at one place. But I was left heavily unsatisfied. Something about graphs/trees and Rust don't match up.

I understand where you are coming from. Rust and Graphs/Trees is a hard problem. Somehow it goes beyond the way we think about Graphs.

I’ve written a JVM in Rust where I was pretty much unable to work without unsafe, and while I know the general consensus is that it is a sin to use, I managed to get away with a few usages wrapped tightly inside a safe API. Sure, I had memory problems during writing, but with MIRI and some debug assertions they were not hard to hunt down and I really only had to get that few lines of code inside unsafe blocks right.

What I’m trying to say with all that, do not be afraid to use unsafes in Rust. It is part of the language for a reason. Sure, do use ARC for some non-performance critical whatever, because it frankly doesn’t really matter. But where it matters, and you decided to use a low-level language, then go for unsafe if that’s the only reasonable way. The result will still be much safer than the other low-level languages. I believe the problem here is the same what C++ tried to achieve: making people believe it is a high level language. That is just dishonest, and really should not be the goal of Rust.

I respect the effort. I won’t argue against such hard-earned experience.

You can tell pyo3 that your type isn't Send and then it'll panic if the object is accessed from multiple threads. Given that that's the only safe option, that seems fine? You say that that's not acceptable for a production library but I don't see the issue.

You have the same restrictions in C++ except with worse consequences.

> You have the same restrictions in C++ except with worse consequences.

In C++ it is safe because the arena is only used from one thread at a time.

To model this C++ pattern in Rust, what I would really want is:

1. Arena should be Sync, and not use interior mutability.

2. Arena::alloc() should do a dual borrow: (a) a mut borrow of the Arena metadata, only for the duration of the alloc() call, and (b) a non-mut (shared) borrow of the Arena data.

Because this kind of split borrow cannot be expressed in Rust AFAICS, (2) is not possible, so (1) is not feasible. This forces Bumpalo to be !Sync, which makes a direct Rust port of the C++ pattern impossible.

I've heard this called the "factory problem" for Rust: you cannot easily make a factory type in Rust that returns references, because if the create() operation mutates the factory, then the returned reference will have a mutable borrow on the factory.

The alternative would be to make a truly thread-safe arena/factory, which could be Sync with interior mutability, but that is an efficiency compromise due to synchronization overhead.

Rust has been my primary language for the past 5 years, but it's moving in a direction that gets it farther away from my own values about what software ought to be like. As more features are added to the language, the ways they interact with each other increases the overall complexity of the language and it becomes hard to keep up.

I really like the safety guarantees that Rust provides and I want to keep enjoying them, but the language -- and more importantly, its ecosystem -- is moving from something that was relatively simple to a weird mish-mash of C++, JavaScript, and Haskell, and I'm keeping an eye out for a possible escape hatch.

Zig, Odin, or Hare are not on the same plane of existence as Rust when it comes to out-of-the-box safety (or, at the moment, out-of-the-box suitability for writing production-grade software), but they are simpler and intend to remain that way. That really jives with my values. Yes, this means that some of the complexity of writing software is pushed back onto me, the programmer, but I feel that I have a better shot at writing good software with a simple language than with a complex language where I only superficially understand the features.

> have a better shot at writing good software with a simple language than with a complex language where I only superficially understand the features.

That's exactly how I feel too. No matter how much I use Rust, I find it nearly impossible to claim I understand a lot of its features. Zig OTOH is basically what C would look like if designed today! The improvements it offers over C are very compelling to me... it remains to be seen if the lack of formal guarantees that Rust gives still makes Zig programs similarly as buggy as C programs, but my current impression is that Zig programs are going to be very far away from C's in terms of safety issues... the features the blog post mention go a long way.

I think you are too dismissive of the importance of simplicity. Programming is hard. That Rust takes away certain problems doesn’t change that. A lot of coding is just reading and understanding some code. If you have problems understanding some code then I hat is also code you are. Or likely to not catch bugs in.

A compiler cannot be a substitute for your brian. The ability to read code and think clearly about it is a massively important feature because humans at the end of the day are the ones who have to understand code and fix it.

It depends on the person. Programmers are different. Rust works great for some. To me it looks too much like C++ which is something I want to put behind. I know it is a different language but it has a lot of that same focus as C++ that leads to slow compilers and complex looking code.

If I was younger I might have put in the effort, but I am not willing to make the same wrong bet I did with C++. I sunk so much time into perfecting C++ skills and realizing afterwards when using other languages that it was a huge waste.

Readable code is important? Then keep in mind the context: low-level programming and writing Safe Rust or writing in some not-quite-memory-safe language because one would rather be bitten by undefined behavior now and again rather than have to learn the borrow checker.

Knowing for sure that the code you write is at least memory safe is a certain kind of readability win and I don’t see how anyone can claim that it’s not.

> ... I am not willing to make the same wrong bet I did with C++. I sunk so much time into perfecting C++ skills ...

What are your top complaints about C++? What parts/patterns are wasteful, and must be avoided?

I have encountered C++ a couple of times in my career. And both those times I was barely able to survive in the short periods of time I spent in those jobs. I'm pretty good at C, but for the life of me, I just can't deal with the hidden behaviours of C++.

So far, I have seen only one example of well-written, understandable C++ code: LLVM. I dabbled in learning LLVM for a side project, and the tutorials on writing compiler passes, and the LLVM's own code, seems to use only the rudimentary features of C++ (primarily, single-inheritance). And this absence of complex C++ features made me feel comfortable looking at, reading, and understanding the LLVM code.

I am of the firm belief now that good code can be written in any bad language, and bad code can be written in any good language. Perhaps those couple of times that I encountered difficult C++ codebases, they were just instances of bad code, and should not be used as an indictment of the C++ language.

Good code => readable, understandable, maintainable, extensible, rewritable. Bad code => !Good code.

> LLVM's own code, seems to use only the rudimentary features of C++ (primarily, single-inheritance). And this absence of complex C++ features made me feel comfortable looking at, reading, and understanding the LLVM code.

Uh, what parts of LLVM were you reading? Because LLVM uses pretty much every C++14 feature allowed by their coding standard. The fact that you felt like no complex machinery was being used is actually a testament of the powers of abstraction in C++.

> Of course Rust does the same thing with certain non-memory-safety bug checks like integer overflow.

The problem with getting lost too much in the ironclad certainties of Rust is that you start forgetting that simplicity (papa pia) protects you from other problems. You can get certain programs in pretty messed up states with an unwanted wrap around.

Programming is hard. Rust is cool, very cool, but it's not a universal silver bullet.

Nothing Is Perfect is a common refrain and non-argument.

If option A has 20 defects and option B has the superset of 25 defects then option A is better—the fact that option A has defects at all is completely besides the point with regards to relative measurements.

But if Option A has 20 defects and takes a lot of effort to go down to 15 defects, yet Option B has 25 defects and offers a quick path to go down to 10 defects, then which option is superior? You can't take this in isolation. The cognitive load of Rust takes a lot of defects out of the picture completely, but going off the beaten path in Rust takes a lot of design and patience.

People have been fighting this fight forever. Should we use static types which make it slower to iterate or dynamic types that help converge on error-free behavior with less programmer intervention? The tradeoffs have become clearer over the years but the decision remains as nuanced as ever. And as the decision space remains nuanced, I'm excited about languages exploring other areas of the design space like Zig or Nim.

> But if Option A has 20 defects and takes a lot of effort to go down to 15 defects, yet Option B has 25 defects and offers a quick path to go down to 10 defects, then which option is superior?

Yes. If you change the entire premise of my example then things are indeed different.

Rust eliminates some defects entirely. Most other low-level languages do not. You would have to use a language like ATS to even compete.

That’s where the five-less-defects thing comes from.

Go down to ten effects? What are you talking about?

I'm not strongly opinionated on Rust specifically but I'm not sure:

> Rust eliminates some defects entirely.

Is really a true premise, and to the extent it is true, is not a clear to me that it makes Rust better or safer than languages who don't eliminate this class of bugs.

Unsafe exists, is widely used, and importantly is used in places where the hairiest versions of these bugs tend to live anyways.

For safe code, I think it's clear that what we as programmers really care about is "how much does the language reduce the number of defects weighted by severity". If Rust reduces the number of memory bugs but increases other types of bugs due to increased complexity, that might not be a net win.

Personally my guess is that Rust is a net win in this regard, but I don't think we have any evidence of that.

Another dimension to remember here is velocity.

I've worked in heavily monadic Scala codebases where changes in object structure require minor refactors due to strict typing in the codebase. Changes that could be small in other languages would necessitate changes in our monad transformer stacks and have us changing types throughout the project. This eventually lead to engineer anxiety ("small changes take forever so I'm not willing to work on this codebase and explain to my manager why a schema change is taking so long") and aversion to project ownership.

This codebase did materially have fewer defects than other, looser, codebases my team worked with. We got paged fewer times for this codebase than others. Unfortunately, other than a handful of engineers who loved working with monadic FP and were also busy, nobody wanted to touch the code and once the original authors left the team, the codebase went untouched. You could make the case that management should have let these engineers take more time to make these changes but in the meantime, other teams at the company built up high reliability ways of working in other languages and paradigms with higher velocity and similarly low defect rate.

Defects alone aren't everything. You need to look at the big picture.

Yes, yes, yes. This was already covered in my original comment.

> you don’t ever have to even think about those kinds of things as long as you trust the compiler and the Unsafe code that you rely on.

Forgive if I don’t give the lawyerly disclaimer in all of my follow-up comments.

Yes, you have to be able to trust the Unsafe code that you depend on.

Rust does no eliminate memory errors.

200+ memory safety errors were found in rust crates: https://www.infoq.com/news/2021/11/rudra-rust-safety/

I love rust, I think there's a good chance zig in its current state isn't the answer, but saying rust is totally memory safe is wrong. You drop into unsafe and people make the same errors C/C++ devs make.

The surface of exposure is much lower in Rust than in C/C++ however. It is unlikely you are writing your entire program in unsafe Rust, so you can still get a significant benefit from memory-safe safe Rust.

In C/C++ world, _everything_ is unsafe, so you can't even limit your exposure in the safer parts because you can do unsafe operations anywhere.

While true, that depends pretty much on which libraries you depend on, and the npm like ecosystem doesn't help in that regard.

Give me a break. In my original comment:

> you don’t ever have to even think about those kinds of things as long as you trust the compiler and the Unsafe code that you rely on.

Your comment is not some kind of gotcha. I can’t be bothered to give the long-form disclaimer in all of my follow-up comments. Read the context.

>If option A has 20 defects and option B has the superset of 25 defects then option A is better

Only if "defect count" is what you care for.

What if you don't give a fuck about defect count, but prefer simplicity to explore/experiment quickly, ease of use, time to market, and so on?

Then you don't want Zig or Rust. Use a language with a GC. Exploratory programming is a lot more pleasant when you don't have to worry about calling free() at the right time. I've had success with PHP and Elixir for productive, exploratory programming, not just because of their GCs, but also because they both support REPL-driven development and hot code reloading.

>Then you don't want Zig or Rust.

I might still want Zig just fine. E.g. because I know C well, and it's a better C. Or because of the ease of interfacing with C/native libs. And several other reasons.

> Then you don't want Zig or Rust. Use a language with a GC.

Zig allows custom allocators pretty much everywhere, right? Would it be impossible to provide it with a GC-based allocator for increased convenience at the cost of a little performance for programs (or parts of programs) where convenience is preferred? Perhaps libgc could be an inspiration here.

Then just use C? Heck, if you really don't give a fuck about defects, you can just have all your code in main(). You really can't beat that in terms of simplicity to explore/experiment quickly, ease of use, and time to market.

>Heck, if you really don't give a fuck about defects, you can just have all your code in main(). You really can't beat that in terms of simplicity to explore/experiment quickly, ease of use, and time to market.

I probably can "beat that", because e.g. breaking things into functions increases simplicity and being able to explore/experiment quickly.

Whereas eliminating defects with a type/lifetime checker you need to hand-wrestle, not so much.

Yeah what if you don’t care about memory safety bugs. Indeed.

Zig keeps overflow checks in the main release mode (ReleaseSafe), Rust defines ints as naturally wrapping in release. This means that Rust is not a strict superset of Zig in terms of safety, if you want to go down that route.

I personally am not interested at all in abstract discussions about sets of errors. Reality is much more complicated, each error needs to be evaluated with regards to the probability of causing it and the associated cost. Both things vary wildly depending on the project at hand.

> This means that Rust is not a strict superset of Zig in terms of safety, if you want to go down that route.

Fair.

> I personally am not interested at all in abstract discussions about sets of errors.

Abstract? Handwaving “no silver bullet” is even more abstract (non-specific).

releasesafe is the main release mode because zig has a small community that is largely ideologically aligned with it.

I have absolutely no faith that, in a future where Zig is popular, it will remain so. "well it passed our unit tests and it didn't fall down when we fuzzed it, so lets squeeze a couple free % extra perf out of it and ship" already in this post we have people talking about how safer mallocs or sanitizers are too much of a hit to expect people to use in the wild.

>A meta point to make here but I don’t quite understand the pushback that Rust has gotten.

The non-CS "human" answer to this is that so much of tech and programming is unfortunately tied to identity. There are developers who view their choices as bordering on religion (from editors to languages to operating systems and so on) and across the entire industry you can see where some will take the slightest hint that things could be better as an affront to their identity.

The more that Rust grows and winds up in the industry, the more this will continue to happen.

Yes. I’m as guilty of this as anyone else.

This is a concise summary of why I'm betting on Rust as the future of performant and embedded computing. You or I could poke holes in it for quite some time. Yet, I imagine the holes would be smaller and less numerous than in any other language capable in these domains.

I think some of the push back is from domains where Rust isn't uniquely suited. Eg, You see a lot of complexity in Rust for server backends; eg async and traits. So, someone not used to Rust may see these, and assume Rust is overly complex. In these domains, there are alternatives that can stand toe-to-toe with it. In lower-level domains, it's not clear there are.

> I think some of the push back is from domains where Rust isn't uniquely suited. Eg, You see a lot of complexity in Rust for server backends; eg async and traits. So, someone not used to Rust may see these, and assume Rust is overly complex. In these domains, there are alternatives that can stand toe-to-toe with it. In lower-level domains, it's not clear there are.

The big win for rust in these domains is startup time, memory usage, and distributable size.

It may be that these things outweigh the easier programming of go or java.

Now if you have a big long running server with lots of hardware at your disposal then rust doesn't make a whole lot of sense. However, if want something like an aws lambda or rapid up/down scaling based on load, rust might start to look a lot more tempting.

It's pretty simple. Rust's safety features (and other language choices) have a productivity cost. For me I found the cost surprisingly high, and I'm not alone (though I'm sure I'll get replies from people who say the borrow checker never bothers them anymore and made them a better programmer, let's just agree there's room to disagree).

Although I'm a big fan of safety, since experiencing Rust my opinion is that low-pause GC is a better direction for the future of safe but high performance programming. And there's also a niche for languages that aren't absolutely safe which I think Zig looks like a great fit for.

After having seeing GC after GC fail to live up to expectations... I'm still voting for Rust. So much more control over wether you even want to allocate or not. I know where you are coming from but, I see it differently I guess.

Whether you allocate or not is a property of the language, not the GC. A lot of GC'd languages encourage or even force allocations all over the place. But maybe we could do better in a new language.

I see a lot of people put rust down for what it is today, and then in the next breathe wistfully wonder about the future of zig, or some other language, etc. It's just not fair.

Rust today let's you avoid allocations, if you want smart pointers you can have them today, I'd you want a GC inside rust, you can have that today as well. Don't need to wait for another language lol.

Enough of Rust is set in stone now that it will never be the language I want. There's nothing fair or unfair about it. That's just the way it is. I still think it is a good language.

Please read the fricking context.

My comment was about preferring other new, low-level languages over Rust when they don’t give the same safety guarantees. If you can deal with a GC then fine—my comment has got nothing to do with that.

So it was much, much more narrow than making a case for Rust in general.

Rust and GC both eliminate certain defects. And if you can use a GC then you don’t need Rust (w.r.t. memory safety).

Admittedly maybe I could have made it more clear that my comment does not make an argument against new low-level capable languages when used with some kind of automatic memory management scheme, like I guess Nim.

What Rust does is incredibly cool and impressive.

But as someone that's dabbled a bit in both Zig and Rust, I think there's a lot of incidental complexity in Rust.

For example, despite having used them and read the docs, I'm still not exactly sure how namespaces work in Rust. It takes 30s to understand exactly what is going on in Zig.

Can you explain what you mean by your namespaces comment? AFAIK, Rust has modules and crates, not namespaces.

Namespace is a more generic name for what modules and crates in Rust provide.

The module system is something that some people struggle with for a while, but every feedback I've heard so far is "it actually makes complete sense, I don't know why I didn't get it earlier". There just seems to be dead-end somewhere that people frequently think themselves into.

Yeah, it was quite mind-bending when I tried to learn it. I still don’t understand why they went in that direction.

The premise of eliminating entire classes of errors in the abstract is nice and all, and definitely something we should do, but it isn’t the sole deciding factor in choosing an implementation. language:

- If the language is not well known, that’s bad. It will be harder to hire a proficient team. More time will be spent on learning the language. - If the syntax is needlessly verbose, that’s bad. It increases the chance for typos and time spent fixing typos to get things to compile. Eventually it leads to ide completion based programming which results in the degradation of the skill set of the pool of programmers that know that language. - If the concepts the language uses are difficult to manipulate and remember, it takes more time to engage with any given piece of code - If you often need to drop into unsafe modes that’s also not great because now you effectively are using two languages not just one. The safe language and the unsafe language. they interact but play by totally different rules. yikes.

I think rust is a great language and the borrow checker is an amazing innovation. However I think rust has a lot of warts that will harm its success in the long run. I think the next language that leverages the borrow checker idea but does so with a bit better ergonomics will really take off.