Hacker News Reader

Comment by scotty79 | Hacker News Reader

From my very limited expeirience with Rust I noticed that it becomes way more easy and laid back language when you just skip using references and lifetimes nearly completely and just wrap everything in Rc<>. Then you are getting expeirience of fairly high level language with a lot of very cool constructs and features like exhaustive pattern matching and value types with a lot of auto-derived functionality.

Does Rc<> help this much with async as well?

View on HN

`Rc` freezes the value as long as it's shared unless you use interior mutability. It's fine when you want to share immutable data, but if you ever need to mutate its contents you will have to deal with awkward cases and situations, and at runtime!

Wrapping everything in `Rc` is not a good default strategy. Instead figure out an architecture that works well with the borrow checker! This normally means thinking hard about your data structures and the ownership model of your application.

I love Rust precisely because of this. You can leverage the compiler to drive your development and design. If it's hard, then it's probably wrong!

Are there any good resources for design patterns in Rust? I've gone through Rust by example and am writing mid-size rust programs now but would love to check my work against the experts.

There's The Unofficial Rust Design Patterns Book...

https://rust-unofficial.github.io/patterns/

(More generally, there's the Little Book of Rust Books to find things like that: https://lborb.github.io/book/ )

...and someone asked about good code to learn from on Reddit a day or two ago and was advised to read basically any code by dtolnay or burntsushi, two of the big wizards who feel like they've written half the Rust ecosystem at times.

Rust also just enforces good C++ practices, so you could just get yourself familiar with the 40 years history of C++.

> I love Rust precisely because of this. You can leverage the compiler to drive your development and design. If it's hard, then it's probably wrong!

Yes, I had the same experience writing code in Rust and Haskell. If the design works nicely with the language, then also the design is better, more modular, simpler dependencies and easier to understand and extend.

I'll stick to my Rc's, thanks. Limiting yourself to immutable data is a good idea anyways for all scenarios where it is possible.

> If it's hard, then it's probably wrong!

Trees and graphs?

The borrow checker gets to actively tell you that your family tree has some inbreeding!

What the problem with trees? They are trees, not cyclic graphs.

You probably need Arc<> in the context of async, but if it's a normal closure, Rc<> will do.

However, that's not needed always. You largely wanna move stuff into closures in my experience, since the closure ends up owning the data.

Arc comes with a small performance overhead (Rc too, but less so I think?), so you don't want to use it all the time. When using Arc, you'll usually need a Mutex too, which adds its own cost, etc.

There are ways to partially avoid this costs, but they can add complexity (eg channels).

Sometimes you might be able to use an Async-flavor of the synchronization types instead, which come with more acceptable performance compromises in Async contexts.

If you are going to wrap everything in Rc, just use kotlin or c-sharp or python?

And suffer a 100 MB runtime?

If anything, Rust with refcounts for everything is much closer to Swift.

Ah didnt realize swift has no runtime. Thanks!

As I remember, it uses an evolution of Objective-C's Automatic Reference Counting feature.

Every language has a runtime, it is what supports the basic language infrastructure, including Swift.

https://github.com/apple/swift/blob/main/docs/Runtime.md

Because Rust feels even nicer than Python and C# (I don't have any experience with kotlin but from what I've seen it's as decent as Python at least) and even with Rc<> Rust is quite light-weight and fast.

One of its selling points is zero-cost abstractions and pretty powerful abstractions at that.

Rust has advantages over C#, even for high level programming:

* Rust traits are more flexible than C# interfaces, especially when combined with generics (implementing traits for foreign types, associated types, each method can have its own constraints, conditional trait implementation, #derive)

* Rust has much stronger thread safety guarantees (absence of data races, preventing access to a mutex's data without locking)

* Options are cleaner than null (though at least C# supports non-nullable reference types nowadays)

* Cleaner error handling through `Result`s. In C# you either have to use exceptions (discouraged for errors that are expected to happen), or use ad-hoc approaches which don't benefit from syntax sugar like the `?` operator.

* Support for discriminated unions ("enums")

* I like cargo better than msbuild+nuget ("it just works", feature-flags + conditional dependencies, distribution as source instead of binaries)

On the other hand C# has better IDEs and you don't have to deal with the rigors of ownership and the borrow checker. In particular I struggle with LINQ style functional programming in Rust, since the lifetimes of closures and the fields they close over are difficult to reason about.

I haven't used Kotlin, but expect it to be similar to C# in its strengths and weaknesses. Python is not an option for me, since I like static typing.

I haven't used C#, so I can't comment on the IDEs there.

However, it's surprising to me that you can get a better experience than VScode + rust-analyzer. It feels like magic when I use it.

I don't agree that people should avoid exceptions for expected errors. The idea that they should has twisted the error handling landscape into a pretzel and is responsible for some ugly bits of Rust design.

Exceptions are a mechanism for error handling that comes with control flow changes.

But Rust already has perfectly nice control flow mechanisms (look at std::ops::ControlFlow for example) and perfectly nice error handling. So you can use either, or both, as necessary, you don't need this particular Frankenstein's Monster.

Using Exceptions for things that aren't actually exceptional is perverse.

> Using Exceptions for things that aren't actually exceptional is perverse.

In python using exceptions for some control flow is actually accepted practice if you don't go overboard.

I think it's better that way because it turns them into something familiar instead of pushing them back to some extraordinary circumstances.

The thing that C++ does with exception means that programmers mind is encouraged to stay on happy path because when he'll walk off this path he'll have to deal with this exception handling monster.

So it's better to either not have exceptions and expose unhappy paths to programmer clearly like Rust or Go does, or familiarize exceptions and their handling so programmers can use them easily in all circumstances like in Python.

Just don't do what C++ and similar do. Offer them as a heavy and weird side mechanism reserved only for super exceptional cases.

That is precisely what C++ does.

C++ exception handling is nothing special, it’s just surrounded by a lot of FUD, premature optimization, outdated benchmarks and cargo cult practices. Like you say, limited exposure adds to all this mystery around it.

If you follow “rule of zero” they just work. The problems come when you start implementing your own destructors “to handle exceptions”, which unfortunately seem to be a very common practice in the wild.

If you don't need the low-level control of Rust then I think Scala would check all of your boxes (aside from distributing dependencies as code instead of binaries). Specifically, an effect library like ZIO looks a lot like Rust without the complexity of managing lifetimes (because you have a GC).

> Python is not an option for me, since I like static typing.

Have you checked out mypy recently (past few years)? With all the strict flags, it's pretty damn hard to get anything but bulletproof static typed code to pass. The only major downside is it lacks higher kinded types at the moment, and there is a PR in the works to add that. But it has all of the other accoutrements you'd expect from a modern type system: generics, structural subtyping, co/contravariance.

There are way more downsides than that.

1. No recursive types, so you can't express a lot of basic things like a "Json" type. This ends up coming up a lot more than you might imagine.

2. Not great support for every pep ie: Protocol

3. Generics are extremely confusing in mypy. Generic classes suffer from the lack of recursive types I'd mentioned as well.

4. Implicit 'Any' everywhere unless you use the strictest settings, which I don't believe anyone does on real code because of the above issues. I've never once managed to get a real codebase to typecheck with mypy.

Honestly mypy works as a somewhat fancy linter, not as a type checker.

Correct me if I'm wrong, but:

- GC pauses, Rc<T> does not, it's simply a deallocation by the last reference holder

- Rc<T> still has predictable memory allocation/deallocation, GC is not guaranteed to run until needed

- More efficient memory use, no need to keep track of allocated objects

- The rest of the Rust language is a pleasure to use (imo)

- Python is slow for certain applications, and not concurrent

- Kotlin is kind of more niche than Rust, it's very popular in the Android community, like C# is popular on Windows

Just my 2 cents, I love progamming in all languages. There are some use-cases where a high level language is absolutely the way to go. For others, Rust provides much more control with a handy escape hatch. Absolutely don't go wrapping everything in Rc<T> like a madman, only when the reduced complexity is more beneficial than dealing with references/lifetimes.

> GC pauses, Rc<T> does not, it's simply a deallocation by the last reference holder

While the last bit is true, it may actually end up having to do lots of work. Think large object graphs where the last reference to any of it goes out of scope.

Also, as a matter of terminology: Rc is a form of garbage collection. It's also a pretty bad general GC strategy at that -- which is why one doesn't just slap an Rc on everything.

> GC pauses, Rc<T> does not, it's simply a deallocation by the last reference holder

There are pauseless GCs (e.g., Azul for JVM), and Go kicked off a trend of super-low-latency GC. Also, RC deallocation is O(N) while GC is usually O(1) (ignoring pedantry about how RC is a type of GC). Further, RC can't handle cycles automatically.

> Rc<T> still has predictable memory allocation/deallocation, GC is not guaranteed to run until needed

Deterministic GC exists, but admittedly isn't widespread. For most non-critical real time systems (e.g., video games) a low latency GC is probably sufficient.

> More efficient memory use, no need to keep track of allocated objects

I'm not sure if this is true? Presumably each RC has an int for its reference counter? I'm not sure what the bookkeeping overhead is for tracing GCs, but I'm guessing it's not O(N)?

> The rest of the Rust language is a pleasure to use (imo)

Agreed, but the borrow checker affects everything so this is a pretty small consolation in practice.

> RC deallocation is O(N)

What is N here? Number of allocations?

Yes.

> I'm not sure what the bookkeeping overhead is for tracing GCs, but I'm guessing it's not O(N)?

I don't know about these days but, historically, the received wisdom was "Take the memory your algorithm should need and double it, to leave room for floating garbage and bookkeeping overhead. Otherwise, your performance will suffer as the GC is forced to run more frequently than is ideal."

("Floating garbage" being the jargon for stuff that's inaccessible but not yet collected.)

Naive synchronous reference counting can lead to large pauses as well. What happens when you drop the last reference to the root of a 10,000-node search tree?

You do 10,000 reference deferments and free()s.

Reference counting might feel more incremental than GC, but really is not. There are tricks you can use, but you're better off with a fast, modern, pauseless real GC that comes with tons of other benefits.

Look: modern GCs simply don't have pause time issues. The problem has been solved. We have concurrent marking and concurrent sweeping. Stop living in the 1990s!

> Naive synchronous reference counting can lead to large pauses as well. What happens when you drop the last reference to the root of a 10,000-node search tree?

> You do 10,000 reference deferments and free()s.

You do all that work at the precise point where the last reference was dropped.

What people who complain about GC pauses dislike is the GC causing pauses in completely unrelated threads, including these threads that aren't even allocating or deallocating anything at that moment.

Reference counting implies non deterministic frees. If you knew who the final owner of an object was, you wouldn't need reference counting.

While it is true that many (most?) GCs have STW sections, it's not at all clear that it is a bad tradeoff.

> You do all that work at the precise point where the last reference was dropped.

True, but rarely meaningful in practice. People don't anticipate spikes at implicit deallocation sites. On the contrary, they typically use RCs to share data without worrying about exact destructure time (i.e. same reason as GC), and thus will be caught off guard in either case. In fact so much so that even experts frequently introduce accidental RC cycles (causing hard-to-debug leaks).

In the cases where you actually want destruction to be deterministic (e.g. large allocations, graceful teardown, kernel resources etc), neither a traditional GC nor RCs are a good solution.

The simplicity and elegance of Rusts RAII scoped ownership model largely goes away with async, due to the unavoidable "Arc-hell".

I wonder if you could just pass this 10000 tree node root to another thread that will just drop it.

This should result in no pause for main thread.

Apparently you need Arc for that not Rc which shouldn't have much overhead over Rc in reasonable scenarios.

Yes, doing big drops in a subthread is a good way of avoiding pauses when dropping large objects of any kind.

Which is, of course, largely what concurrent GC is about.

Yes, that is how C++/WinRT handles cascading deletion of COM instances, however when you are going down that route, it is basically a poor man's tracing GC.

However it just for the stuff you need it for. It's not all encompassing. It's not mandatory part of the language.

True, and unfortunately a reason why we keep having CVEs related to memory corruption.

There is a thread I came across on hacker news that does exactly that https://news.ycombinator.com/item?id=23362518 - this question reminded me of it

> More efficient memory use, no need to keep track of allocated objects

RC uses a traditional malloc/free-style heap, which requires bookkeeping for allocations. It also allocates an _additional_ word to track the reference count, which leads to poor cache behaviour (even in single-threaded programs, but especially in multi-threaded programs). Bumping/copying nursery incurs no bookkeeping overhead.

While I understand that with Graal and .Net you can ostensibly make native, static binaries for Kotlin or C#, I'm very skeptical that it works well in practice. In particular, I'm guessing it will feel like swimming upstream, fighting an ecosystem and build tooling which mostly assume you're running on a VM. And then there's the question of performance...

And of course, with Python your code will run 100x slower than either of the above, your dependency management will suck, you won't be able to statically compile, and to top it all off everyone will give you recommendations for your ailments which take a long time to try out and inevitably will fail miserably for one glaring reason or another. By way of example: "just rewrite the slow bits in C" -> you rewrite the slow bits in C -> your code is now slower because the marshaling costs exceed the gains + your build system is dramatically more complex and you have the sheer joy of debugging segfaults and undefined behavior (yeah, I know Cython exists).

With the modern JVM you actually have to work quite hard to write native code (Rust/C++/C) that outperforms an equivalent Java/Kotlin/Scala implementation. And it is quite easy to perform worse with a native implementation. Of course that is subject to various caveats:

1. Anything running on the JVM will need a 50-500ms of startup time.

2. The JVM implementation will not reach max performance until the runtime has optimized and JIT'd the relevant code paths.

3. There is memory overhead of the VM itself so your runtime will (all else equal) probably require more memory.

If you do need to use Graal to generate native images though it can actually be quite nice. You can run locally (or in benchmarking environments) on the VM and get all the tooling and metrics that come along with that, but build a native binary to actually deploy. I agree that it can be kind of a pain though.

> With the modern JVM you actually have to work quite hard to write native code (Rust/C++/C) that outperforms an equivalent Java/Kotlin/Scala implementation.

For what kind of code? I've never experienced comparable performance between JVM and C++ implementations, and I've had the misfortune of writing a couple parallel implementations in recent years where it was literally an "apples to apples" comparison. C++ is faster by default and isn't particularly close, even if you ignore Java's startup and warmup time. I've never seen a native implementation run slower than a JVM one in my entire career, which has included a lot of Java.

This is the expected outcome and easily explainable in technical terms. Performance in modern systems is dominated by memory handling efficiency, where C++ is very strong and JVM is not.

Those are some very large qualifications, though, right? I think a better way of saying this is something like “you actually have to work quite hard to write native code that outperforms an equivalent Java/Kotlin/Scala implementation for a specific kind of long-running process which can amortize the startup costs and the JIT time.” And that can be true! (Though in my experience naïve Rust often substantially outperforms naïve Java/C#/etc.)

The other thing is that true warmup (not just startup) time to reaching optimization is a lot weirder and less predictable than people realize. See e.g. [1] and [2] for a fascinating analysis (caveat: about a decade old) of the actual warmup characteristics of a number of different VMs (Graal, HHVM, HotSpot, LuaJIT, PyPy, TruffleRuby, and V8). Spoilers: there are bizarre deopt cliffs, scenarios where it appears to optimize and then de-optimizes if you run it long enough, and programs which just never optimize… and this is on benchmark tests!

[1]: https://tratt.net/laurie/blog/entries/why_arent_more_users_m...

[2]: https://tratt.net/laurie/blog/entries/why_arent_more_users_m...

That's a fair point. But I think that "long-running processes which can amortize the startup cost and JIT time" covers a very large swath of software development.

Then you are essentially introducing a verbose, unoptimized garbage collector?

And you still run into all sorts of ergonomic issues when you inevitably need to dereference your Rc<> pointer. That said, I sympathize with the parent's desire for a proper Rust-lite with a GC: C-family syntax, great tooling, great documentation, great standard library and ecosystem, native+static compilation by default. Of course, someone will ignore those criteria and come in suggesting OCaml/Reason...

> Of course, someone will ignore those criteria and come in suggesting OCaml/Reason...

I hoped Go to be such a language, but it failed to fulfill my needs by throwing away all the PL knowledge that humanity has accumulated for decades. I still mourn for the missed opportunity by Google.

I considered mentioning "Go", and while it would be nice to have a Go with generics + sum types, it's also very nice having Go as it is today. In other words, it would be nice to have a "Go with those things" and a "Go without them" (predictable rebuttal: "But if Go supports those things, you would only have to use them when you wanted to!" <- not true, because in the real world we have to use the libraries available to us).

Frankly though, the type system is absurdly overemphasized. Squabbling over type systems is penny wise and pound foolish if you're missing the fundamentals. Allow me to quote myself from a thread a couple of days ago:

> Agreed. It still surprises me that so many other languages fail at the fundamentals (minimal learning curve, static binaries, fast builds, reproducible dependency management, great tooling, great stdlib + ecosystem, etc) and yet many devotees of those languages have positively hyperventilated about Go's error handling and type system for 12 years. Go is finally getting generics and I'm sort of cautiously excited about it (like I was when Apple added the TouchBar to MacBook Pros), but any net benefit is going to be positively negligible in comparison to the degree in which Go raised the bar on the fundamentals.

https://news.ycombinator.com/item?id=29179681

Java (and other JVM languages), as well as C# to a certain extent address the points you raise. Secondly, it's not true that golang has a minimal learning curve (I've seen senior engineers write bad golang code when they're onboarded - it takes time to learn the golang way of doing things and its quirks), and it's not even a fundamental goal to have.

I disagree. Firstly, it’s widely agreed upon that Go has quite a lot lower learning curve than just about any other language. Your senior engineer anecdotes sound like outliers.

With respect to Java and C#, I addressed a similar question here: https://news.ycombinator.com/item?id=29181361

Even where those languages have nominally improved, it is often so difficult in practice that virtually no one bothers to use the improvements. For example, while Java and .Net technically can do AOT, static compilation, virtually no one does, instead preferring to put up with runtime dependencies (including the runtime itself at a minimum). It’s ticking a box so on paper they compare better to Go (or Rust) but the practical experiences remain leagues apart.

Again, having a lower learning curve is a non-goal for any serious project. All languages have their quirks and approaches to expressing concepts, and golang is no different.

Java already has a low latency GC (ZGC), and with GraalVM being more and more used, more frameworks are starting to allow building native binaries (e.g. quarkus.io and micronaut.io and helidon.io). Once Spring gets on board, then a huge part of the ecosystem will.

Secondly, if the only concern is packaging and deploying a single file, that's been possible for a long time now (uber jars), and made easier recently with jlink and jpackage.

All "serious" golang projects I saw use some sort of testing suite (like testify), because the built in testing library is so verbose it's basically useless for anything but simple use cases.

And a built in HTTP sever is not a big deal. It's literally a one line maven import, and many such options exist now.

My employer is a big user of golang, and for all company projects, we have to use a framework they built to write code (basically Spring or ASP.NET DI reinvented (but poorly), plus handling some of golang's bad design decisions around error handling and propagation - but it's far from perfect there's only so much they could do). Not to mention having to use bazel to build them and resolve dependencies (and they say golang compiles quickly, lol). All serious projects will end up in that state sooner or later, and in Java land, it's all available from the start and extremely mature and battle tested.

You're right that the experiences are leagues apart, but that's in favor of the JVM. golang has nothing close when it comes to monitoring and observability and continuous profiling. Nor the tunability of the JVM to select the best GC based on the application type.

Have you looked at D? (or perhaps zig - but I think D would be closer to a "sane" go).

Sounds like D, actually.

As D is making gc optional, I'm guessing rust will evolve ADT crates that makes it easier to do massive parallell processing via message passing. But like with C, I'm not sure most of that should be "part of the language" - might not be something you need for your bootsector or ABS break system controller...

Take a look at Kotlin Native. It compiles to native code, the standard library is pretty good, and the latest versions have introduced a GC so it's now much easier to convert Java to it (Java can be rewritten into Kotlin automatically at the source level).

However I'd just use Graal native image. Most libraries work and it's far easier to write a config or tweak one that doesn't, than not have access to the library at all!

Yes. It's a kind of a garbage collector for people who don't need garbage collector.

If you need to do a lot of small allocation or have many circular references that you want to drop wholesale then you need a proper GC.

Even in some other cases you might benefit from proper GC. However Rust doesn't have a standard proper GC yet. It might have it as library some day. Some simple ones like https://docs.rs/gc/0.4.1/gc/ already exist.

But Rust is such a great language for many reasons. It would be a shame to not use it just because it doesn't have a great GC yet or because you don't have patience for borrow checker.

You can get those cool features in languages such as OCaml/F#/Scala/Haskell, and then you don't have to worry about managing memory because you have a GC.

Sure but all of them are weird. Rust is something you can pick up in an afternoon if you have expeirience in C++-like languages.

From the point of view of somehow that has taught C++ as TA, worked with C and C++ during several years in some heavy contexts, an afternoon won't do it.

Reference counting is mostly a cop-out for people who can't design code properly and reason about ownership and lifetimes.

It essentially makes your code Java and defeats the point of the language.

I would expect most decent teams to reject your code during code reviews.

Reply on news.ycombinator.com