Hacker News Reader

Comment by fnfjfk | Hacker News Reader

This article could be 1 sentence: “it has sum types” :)

I find programming in a language without sum types is like rock climbing in ankle weights. I just can’t model many types of data correctly, and I hate using product types for something that is clearly a sum type.

Even if Rust didn’t make any safety guarantees over other languages (although sum types provide safety guarantees through exhaustiveness!) I would still think Rust helps a lot because of that.

I guess Swift and Kotlin are basically “it has sum types” languages too, although they both come with baggage (Apple-only in practice, JVM).

View on HN

Sum types are fantastic and I’ll join you singing their praises all day long. But this article is about rust’s memory model & memory safety - which is independent of sum types.

Unsafe blocks in rust are the only place you can dereference pointers. There are whole classes of memory bugs which are impossible to code up in safe rust code because of this.

As an example of how this helps, a couple years ago I was working on a complex & clever rope data structure. My code segfaulted - and I can count on one hand the number of times my rust code has ever done that. A segfault like that can only come from memory corruption or dereferencing invalid memory - both of which are only possible in unsafe code. And the bottom of the stack trace was a safe function, so I knew I’d corrupted memory somehow. These bugs are really hard to find in C - basically the worst case for debugging because the problem could be literally anywhere. But in rust I knew the bug was in one of only two unsafe blocks in the ~thousand or so lines of my library. And one of those blocks was executed not long before the crash.

5 minutes later I had found and fixed the bug. In C, I’ve had similar problems take hours to days to fix.

Bugs like this aren’t possible at all in Go or C# because of their memory model. But in those languages you pay for safety by sacrificing performance. The rust I’ve written & benchmarked so far somehow faster than the equivalent C code. Which is pretty wild!

> These bugs are really hard to find in C - basically the worst case for debugging because the problem could be literally anywhere.

> 5 minutes later I had found and fixed the bug. In C, I’ve had similar problems take hours to days to fix.

Modern tooling like a memory synthesizer and good debugger makes this kind of debugging relatively easy.

In some case, the bug would have been detected even before you run with proper warnings.

> Bugs like this aren’t possible at all in Go or C# because of their memory model.

Except in their respective FFI or unsafe block of code.

> But in those languages you pay for safety by sacrificing performance.

That's a bit of a strong statement. Even if we accept (i don't) the performance aspect, GC languages give more safety,convenience and dev prod. than rust and at a significantly less complexity overload.

> As an example of how this helps, a couple years ago I was working on a complex & clever rope data structure. My code segfaulted - and I can count on one hand the number of times my rust code has ever done that.

This might simply be an indication that you were using the wrong tools in the first language. Most native languages (D, C++ etc...) have facilities to avoid unsafe behavior. Use them.

I've never heard of memory synthesizers.

I'll cede the point that some of Rust's advantages are merely psychological. But I would argue that most of us are tired, frustrated, and working on teams where we have neither the energy nor authority nor teaching experience to make everyone learn the right tools. I've had to teach some coworkers how to use a profiler. And then remind them to use it.

Therefore mere psychological benefits still improve outcomes.

It's easier to memorize "cargo run" than to memorize the vocabulary quizzes for Python or Kotlin tooling. (A gradle is used to serve soup. A wheel is part of a car.) Rust produces one binary by default, and I don't even have to think about how to bundle a runtime. That means I also don't need to teach anyone what Docker is.

I have a hunch this actually matters. Everyone who's learning to program right now will be deep in their work in 20 years. So every software that's succeeded in my lifetime - JavaScript, Python, Unity 3D, etc., has succeeded by appealing to noobies and then lasting long enough that everyone likes it. In the very long term, a low barrier to entry might matter as much as a high skill ceiling.

> I've never heard of memory synthesizers.

I assume it's a typo for memory sanitizer (i.e., ASAN).

> Even if we accept (i don't) the performance aspect, GC languages give more safety,convenience and dev prod. than rust and at a significantly less complexity overload

At the limit, I think the performance benefits of rust are pretty clear. The performance ceiling of rust seems to be pretty unambiguously higher than that of Go, Python, C#, javascript, etc. But yeah - no contest on rust being way harder to learn and use. I’ve been programming rust for years now, but I still reach for typescript when prototyping because I still find it much faster when making prototypes. Especially when async is involved.

> This might simply be an indication that you were using the wrong tools in the first language. Most native languages (D, C++ etc...) have facilities to avoid unsafe behavior. Use them.

Can you give some examples of those tools? The “first language” I mentioned above was C. What equivalent does C have to rust’s memory safety? Or C++ for that matter? Are smart pointers in C++ that ubiquitous and good that memory corruption is a thing of the past?

> At the limit, I think the performance benefits of rust are pretty clear. The performance ceiling of rust seems to be pretty unambiguously higher than that of Go, Python, C#, javascript, etc.

Not to be pedantic, but "at the limit" performance is not a very good metric. Programming is an engineering exercise, so IMO performance delta per unit of development work is better characterization if we are comparing programming language.

At the limit C# supports inline assembly, so at the limit it could be match rust pounds for pounds with enough efforts, but that's hardly a useful comparison.

In general, a lot of the modern performance differences between programming languages is caused (from the programming languages perspective) by the object model each programming language use. Python/Javascript are in a class of their own and i don't think belong in performance conversation against statically type languages. Can't comment on go cuz i don't know much it.

On C#, the memory model can be surprisingly close to rust. The combination of typed IR for efficient monomorphization + the the ability to use stack allocated struct, mean that C# code can be quite close to rust code. And then the differences are mainly a consequence of LLVM backend vs ruygenJIT or AOT.

To say nothing of the advantage that a GC can have by having a global view of memory allocation/deallocations.

Added to that the fact that the time a rust eng. is spending modeling his ownership relationships and converting then into something the borrow checker can/accept could be spend improving C# perf further.

> Can you give some examples of those tools? The “first language” I mentioned above was C. What equivalent does C have to rust’s memory safety? Or C++ for that matter? Are smart pointers in C++ that ubiquitous and good that memory corruption is a thing of the past?

I am not too familiar with C, but i don't a think any new tool is needed : wrap all the unsafe operation (naked array accesses, allocation etc...) in a safe API and only use the safe API...

In C++, its trivial to avoid memory unsafe operation between smart_pointer, &reference, for_each for iteration etc... etc... Manipulating raw memory in "modern" C++ is mostly a choice than a necessity these days.

It is trivial yet there are plenty of CVE exploiting the failure to do so, as people being people, always take shortcuts in the name of performace, without bothering to use a profiler.

> Can you give some examples of those tools?

ASAN (address sanitizer), UBSAN (undefined behaviour sanitizer), Valgrind.

All of these only detect runtime violations, so they can find bugs but don't prove their absence. On top of that I got a sufficient number of false negative or even bugs that wouldn't reproduce/wouldn't give interesting information with these tools that I can confidently tell you they don't even approach the guarantees provided by Rust.

There are also static analyzer like cppcheck and clang tidy. They do find some useful issues, but they:

1. Have false positives 2. Are a hassle to add on top of existing projects 3. Have false negatives, so a code that passes is not necessarily devoid of UB.

> Are smart pointers in C++ that ubiquitous and good that memory corruption is a thing of the past?

Smart pointers in C++ effectively prevent some classes of double free errors and some classes of leaks (like all RAII, I might add). They do very little against use-after-free in the general sense, because as soon as you take an observer pointer/reference (the equivalent of a Rust borrow), you have to either:

1. Forfeit safety by handling its lifetime on your own 2. Forfeit performance by unnecessarily cloning data (and also forfeit correctness in some cases, as the "sea of shared pointers" approach of some codebases create object identity issues and memory cycles).

The standard library is also full of footguns that can cause UB, such as calling `vector::front()` on an empty vector, that come up a lot in practice. I think cppcheck was able to flag some of these but I'm unsure of this off the top of my head.

Lastly C++ has a slew of very ugly errors like ODR violations that do happen in practice (albeit rarely) and that are completely outside the purview of tools (the last one a former colleague wrote would generate completely garbage information in asan, and would cause valgrind itself to segfault. The error cost the afternoon of two engineers for what was a typo causing a namespace to be early closed in a file... The kind of abhorrent error that causes to avoid these languages as much as I can).

I developed in C++11 for almost 10 years, and I am now a professional Rust developer. Productivity, developer experience, and correctness are night and day, Rust's advantage.

The problem is getting people to use that modern tooling, even static analysis is hardly used and lint was created in 1979.

This is what memory safe systems languages offer, there is no way around avoiding such tooling, it is part of the language and needs to be explicitly disabled.

> The problem is getting people to use that modern tooling, even static analysis is hardly used and lint was created in 1979

Maybe; rest to see if using modern tooling is easier or harder on the average programmer than learning a new language.

> This is what memory safe systems languages offer, there is no way around avoiding such tooling, it is part of the language and needs to be explicitly disabled.

The programming language is just one part of the chain. The goal is to detect/prevent bad code pattern and address them. There is no rules about this happening at the language level.

Oh I understand how safe helps and all, just wanted to sing Rust’s praises as a less low level memory-focused language too.

> Bugs like this aren’t possible at all in Go or C#

As a point of order, such bugs are possible for Go. Eg, hashmap's behaviour becomes undefined if you access from multiple threads at the same time.

Admittedly, that's a Go peculiarity. Making the programmer deal with concurrency on collections undoubtedly gets it a speed boost, but it's not the tradeoff most gc languages take.

The beauty of Rust, is you don't need to trade speed or memory for concurrency.

> hey both come with baggage (JVM).

I'm always surprised to hear people say this, when my perspective on the JVM is that it's a modern technical marvel several decades in the making with advantages like several unparalleled GCs and an ecosystem with extensive off-the-shelf & highly performant code available for any domain.

I don’t work on server code, and I think a lot of discussions are really focused on server-side work. I have done desktop, mobile, and developer tools (CLIs and “scripts”).

If I was working on a server thing Kotlin would seem really good. In my work it is obviously applicable to Android, but not so much elsewhere.

I think Rust is nearly perfect for developer CLIs, you don’t really need to deal with the thorny memory and lifetime stuff because most things are allocated on the stack and the entire program is synchronous. clap is very good. Performance, especially startup performance, is incredible.

You can use GraalVM Native Image and Kotlin/Native to compile Kotlin to a binary executable, which is fairly fast to start up but does not have all of the high-performance JVM features.

JVM is a marvel at making OOP code merely 10% or 50% slower than C, but I find procedural/data-oriented programming to be quite a lot clearer, faster, and more predictable (with respect to performance) and I don't need nor a huge runtime nor an expert to tune it nor a complicated toolchain to bundle it in with my application (or otherwise ensure that the properly tuned runtime is installed on my target).

To get it to that level of performance, you need to have expert java programmers, and the result will be hard to maintain.

For instance, you can’t use String on hot paths. Also, NPEs can happen almost anywhere, and the compiler doesn’t enforce thread safely (by which I mean “statically checked to be data race free”)

Also, the 10-50% slower rule of thumb only applies to throughput. Tail latency is generally much worse than that.

It depends on what you're doing. In C you're going to be hampered by, again, the ecosystem.

There is nothing comparable in C as performant as all of the streaming tooling built up around Kafka for example.

So your can benchmark percentages all you want, but there are a lot of things you just can't do in C that you can on the JVM (without 100 developers and three years)

> There is nothing comparable in C as performant as all of the streaming tooling built up around Kafka for example.

This is an ill-chosen example.

librdkafka is as or more performant than the Java stuff.

As a server, Redpanda blows Kafka out of the water (but this is rarely any bottleneck with brokers).

Yes, if you take the time to hand roll a replacement like Redpanda did it will be faster (though to your point that seems fairly pointless given the constraints imposed by IO)

I was referring more to the general ecosystem of Flink/Spark/KStreams/KTable etc.

this seems truthy but isn't in practice. a lot of work, my perf optimization team @ redpanda does (yes we have a full team chasing tail latencies) is spent on CPU optimization, debouncing, amortizing costs, metadata lookups, hash tables, profilers, etc. so there is a lot of additional work after the IO layer which a decent async eventing thing can get you to get good perf.

What I mean to say is that in my experience the time spent round-tripping to Kafka is more than the time it takes Kafka to do whatever I'm asking it to do. So at least for my use-cases a faster Kafka would be of no benefit.

Now if it means I can run fewer smaller brokers that's awesome.

def. that should be the case, we have iot companies pushing us on a single pthread a a few megs of ram.

> Flink/Spark/KStreams/KTable etc.

Then it's still a bit of an untruth since all of those use RocksDB for their high-performance storage layer, which isn't in Java. You could maybe (maybe!) argue for some ergonomics (though especially hard to argue for Spark!) but performance still goes plainly to C (/ C++).

We're not talking about the systems you build on top of here. We're talking about the code. You write yourself. The fact that rocks is written in c is irrelevant to the fact that in order to use the ecosystem I'm describing you need to be on the JVM or your experience will be at best subpar.

Sure, if you ignore offloading all the hard work to a C library, we can pretend any language is as fast as any other. It's a boring conversation then though - might as well use Python...

Not to mention you still pay a real performance penalty for being unable to make direct use of RocksDB features, because Kafka is shoving a million "abstraction" and cache layers in there and you just want your your damned merge operator to apply.

I don't agree Kafka off the JVM is subpar. Kafka Streams has a lot of sharp edges from "abstractions" that aren't, Spark is a godawful developer experience and tuning it for performance is black magic for 99% of data engineers, and Flink keeps dithering about how consistent/stable baseline features like queries will be. You can stand up very good apps for a huge set of common cases just as quickly in Go or C++, and not have to deal with Kafka Streams's funny opinions about threads/tasks/consumers.

You can write procedural/data-oriented code in Java.

As long as you hire C/C++/Rust programmers to write it.

Sort of (Java still doesn't have value types), and even then you still have most of the cons of a big runtime.

Always funny to see OOP being thrown around like it's some sort of bogeyman. C++ is OOP, so is Rust (minus inheritance), yet both are as fast or faster than C.

As compared to what... JVM is neither particularly advance nor particularly performant.

Can't quite comment on the GCs though

> JVM is neither particularly advance nor particularly performant

I'm not exactly a fan of Java, to put it mildly, but it is pretty efficient these days. To the point that it sometimes amazes me.

I don't want to sound like i am downplaying you understanding of the JVM. But i still have to ask : efficient as compare to what ?

From having played around the JVM, and spent quite a bite of time optimizer large spark workload , from the limited Monomorphization, to memory and cache inefficient object model, to the limited inlining depth, absence of code compaction ( for i-cache performance), the poor registor allocator etc... the JVM is really not up there.

I think it only shines when compare to things like go/python/javascript that the JVM can appear to be great.

Compared to doing distributed systems in DCOM/CORBA using C and C++ for example, that is one of the reasons it took off.

Many that like to bash Java EE, have no clue how bad it was before it came to be, it wasn't only Sun's marketing pushing it.

It isn't by accident that RMI had interoperability with CORBA.

I am not sure how this is an answer the my argument. However saying that distributed system in JAVA EE are more performant that in C++ ( i am no familliar with DCOM/CORBA but i do know other libraries and system ) is news to me.

And again, the success of JAVA in the 90 and 00 can also be explained by better tooling, safer language,better x-platform stories,better libraries etc... etc...

Saying that java/jvm are not particularly performant is not bashing ... java/jvm have other great things going on... performance just isnt one of them

I guess if what one is measuring FPS for AAA, then surely not.

Naturally not having a clue about how to write proper Java code (for loops instead of System.arraycopy()), use profilers, use JIT caches with PGO feedback, not understanding how to allocate off heap, not using JNI to avoid writing everything in C or C++ with their own set of issues, doing background stuff on main threads,...also helps some Java applications not being performant.

Have you used it since Java 8?

There's a reason the entire streaming ecosystem (Kafka/Flink/Spark/Etc.) has built up around the JVM.

JVM memory consumption is prohibitive for small applications and overall the JVM is very memory inefficient. If you can, avoid Java for anything that isn't a long running server process

Have your worked with the JVM since Java 8?

Is memory util on servers still really a consideration in 2023 for most workloads if performance is otherwise good?

> If you can, avoid Java for anything that isn't a long running server process

Isn't that the JVM's niche?

Memory utilization on servers is absolutely a consideration in 2023. At work, we're looking seriously at migrating some key parts of our infrastructure from Java to Rust precisely because of factor (as well as startup time, throughput).

As someone working on systems that handle trillions of message per day in Java, I find this hard to relate to. Is it bad code? Some critical infrastructure flaw? Misunderstanding of memory options settings or how they work in containers?

Startup time is definitely a concern. Good luck on the throughput front though. Java libs (like Netty) still outpace what's available in Rust. You may end up with your own code being more performant, but the things you rely on externally being slower. This was our experience when trialing re-writes in Rust.

It's probably RDD (resume driven development). It's always fun to rewrite in the latest new hype, and I don't have anything against Rust - I think it's great in the domains it is targeting.

> Is memory util on servers still really a consideration in 2023 for most workloads if performance is otherwise good?

The way that cloud scaling works is that as you increase RAM you also increase CPU, which costs a fortune. You can go for 'memory optimized' instances but even still, there are costs associated with memory that aren't just the RAM modules, unfortunately.

Plus with container orchestration it's increasingly important to pack your services into a single instance, and memory can veryyyy quickly become the constraint - in fact, it might be the one I've seen most often.

I don't find this, but maybe it's something K8's helps us get away from?

For our workloads we seem to need more CPU when we need more ram anyways.

No, you'd run into it with k8s more, if anything. But it is workload dependent.

I don't want several GCs, though.

Yeah, I've always derided the having of options as well. I find it best when you cannot toggle between various options to find the best set of tradeoffs for your use case.

Yes, for example, it's much better when the language completely takes away your ability to specify memory layouts or allocation patterns because it will never pick stupid ones.

Value types are in the works for the JVM.

I'm pretty sure I had this exact conversation five years ago, and believe I will be having it five years from now.

This is a huge deal, I find it so difficult to choose a language that doesn't have sum types and destructuring via pattern matching for the kinds of projects I like to work on (compiler/interpreter implementation).

IMHO the one downside with sum types in a performance-oriented systems programming language (or as I prefer to call them 'tagged unions' because that makes it clearer how they actually look like in memory) is that they are inherently cache-unfriendly, because they always take up as much space as the largest wrapped type (plus 8 bytes for the tag). When you have an array of those you need to be aware of the potential 'waste' between array items.

(now of course an automatic transformation into a structure-of-arrays would be pretty cool, I actually wonder what Zig's MultiArrayList does with tagged unions).

PS: in general I agree though, for instance in Typescript they are fantastic, but there I usually don't care much about memory layout and performance.

That depends on the sizes of the objects. If the objects in the union are generally small, cache coherence is better than if you have polymorphic objects allocated on the heap.

a good allocator could mitigate most of the allocation effects

Unlikely, because the allocator would need to know things only the compilers knows, and the compiler in turn needs to know how the allocator lays out the data in memory?

Some of that could probably be solved with comptime code (like Zig's MultiArrayList container I mentioned above).

> Unlikely, because the allocator would need to know things only the compilers knows

Such as ?

Such a 'smart allocator' can't just work with a size and alignment, it also needs to know the internal structure of objects so that it can arrange them differently (maybe to remove padding bytes, maybe to turn arrays-of-structs into structs-of-arrays. And in turn the compiler needs to know how the 'allocator' has rearranged things in order to generate the code which accesses the data.

I also wasn't thinking about moving parts of a struct out into heap allocations, I think in most situations that would be worse than having to deal with padding. Instead at least have the programmer think about how the internal memory layout of sum types might affect performance in specific situations instead of just using them everywhere and in mass because they are convenient.

I think parent meant transforming a vector<A+T> into vector<A>*vector<T>

What's the alternative to sum types? What does a "structure of arrays" look like as it pertains to sum types? Are the arrays sparse or dense (and how is either going to be better for cache coherence)? Where does the tag live? In its own array?

> What does a "structure of arrays" look like as it pertains to sum types?

https://news.ycombinator.com/item?id=36195425

If you are dealing with always-same-type arrays then you can express so as a sum type of arrays, not an array of sum types.

Unfortunately, that isn't what cache unfriendly means. You're concerned about wasting memory bandwidth. Your array is trivial to prefetch.

AFAIK the prefetcher doesn't help much if it's mainly busy with prefetching padding bytes of the 'inactive' unions (and I think even a smart prefetcher would at least always load whole cache lines which could be packed with active data instead). Of course that's also a bandwidth problem, you're wasting bandwidth loading data into the cache which then is ignored.

The overhead isn't always 8 bytes. When the type uses the naive representation, the overhead will generally match the alignment. For example an Option<u8> occupies 2 bytes, 1 of which is for the tag.

And if the type has some illegal values, these can sometimes be used to discriminate the type. When that's possible, the union will have the same size as the largest underlying type with no overhead for a tag. For example Option<&T> matches the size of &T, since it uses the illegal null value to represent None.

Rust sum types do not take up that much extra space, in fact often the overhead is completely compiled away. Lots of hard work to keep the wasted memory to a minimum, and they are not pointers, so there isn't an extra memory hop or anything.

> plus 8 bytes for the tag

The tag can often be stored inline in the data, so doesn't necessarily require extra space.

> because they always take up as much space as the largest wrapped type

Yeah... but what's the alternative?

The alternative would be for Rust to generate machine code for the following

    enum Foo {
        Bar,
        Qux([i32; 1024]),
    }

as if the user had written

    enum Foo {
        Bar,
        Qux(Box<[i32; 1024]>),
    }

I don't think it's a good idea to do that kind of silent transformation, instead teach the developer about their options. That way the application's performance doesn't suddenly shift after changing a few parts of unrelated code that accidentally causes some threshold to be crossed that changes the performance characteristics of binary being produced.

Today, clippy does what I think is the right thing: it teaches the user

  warning: large size difference between variants
   --> src/main.rs:1:1
    |
  1 | / enum Foo {
  2 | |     Bar,
    | |     --- the second-largest variant carries no data at all
  3 | |     Qux([i32; 1024]),
    | |     ---------------- the largest variant contains at least 4096 bytes
  4 | | }
    | |_^ the entire enum is at least 4100 bytes
    |
    = help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#large_enum_variant
    = note: `#[warn(clippy::large_enum_variant)]` on by default
  help: consider boxing the large fields to reduce the total size of the enum
    |
  3 |     Qux(Box<[i32; 1024]>),
    |         ~~~~~~~~~~~~~~~~

When you combine this with rust-analyzer's hints for the sizes of items, it makes it much easier to spot this kind of stuff and resolve it automatically while developing, too.

This is why I wish there was more momentum behind Hare[1]. It's basically C with sum types, namespaces, and other conservative improvements on C.

[1] https://harelang.org/

Eh, Hare expects you to write and rewrite collections like hashmaps yourself. It just looks like a less flexibe and less useful Zig.

And more complex, in implementation and usage. I like Zig too but I don't think there will ever be very good IDE integration (navigation, find usages, refactoring) given its flexibility with regard to the comptime stuff.

As a C++ dev I really don't follow. I don't remember any times where I thought something like a Union would help much.

Where have you found so much use for them?

Option/Optional/Maybe is the most obvious one, as a basic example have you interacted with a API that sends some JSON where a field can be null or not? How would you represent that and distinguish it from a field that is always a string?

Not many options, pointers most likely.

That's just std::optional<> vs std::string, isn't it?

Yes, and std::optional is a janky implementation of a sum type in a language that doesn’t have them. The ergonomics are bad, and it is easy to make your program blow up if you forget a check.

In a language with sum types like Rust, Kotlin, and Swift, if you’re following the default “happy path” of coding, you are not allowed to forget a check. The compiler will not let you.

They're extremely useful in capturing the "business rules" for a given domain, particularly when it's a restricted domain: if something can only be `A | B | C`, being able to make that explicit, have it checked exhaustively everywhere by the compiler, and be able to dispatch on it easily using `match` is really powerful.

That's because it comes with a statically-tracked "tag" (so: sometimes called a "tagged union") that the compiler can use to check you. You can write the same thing with a `union` in C or C++, with an `enum` to tag it, but you're doing that by hand rather than having it built in.

Also: Where a struct or class's relationship over its members is the product of all of the types which make it up, a tagged union/enum is the sum of them. A common example: instead of a bunch of optional fields on a single class to model the different possible combinations, modeling something as only the legal set of combinations of fields. I recommend Scott Wlaschin's 'designing with types' series (uses F♯, but applies to any languages with 'sum types') for a great intro and then deep dive on the ideas here: https://fsharpforfunandprofit.com/series/designing-with-type...

I'm pretty fluent in C++ and Rust. You don't find much use for them in C++ because they're a relative pain in the ass without language support. std::variant is a sum type, but when's the last time you used it? I bet you had lots of fun with visitors and explicit type checks.

The most motivating example is Rust's Result<T, E>. Either you have a "good" type T, or some error E occurred. Want to bubble this error up? Just do `my_result?` to get the underlying T or return E early. This idiom is pervasive and makes error handling via return codes very easy to use.

It also makes it feasible to have types that have no "default" value. In C++ you have std::unique_ptr<T> which, for various reasons, has to have a nullptr state, or else it's frustrating to use. In Rust, owned pointers Box<T> don't have an empty state. If you need a nullable pointer, you use Option<Box<T>>, which is either Some(my_box) or None. The fact that Box<T> is always pointing to a valid object is a stronger guarantee than C++ smart pointers, and is only ergonomically possible because of the Option<T> sum type.

optional<non_null_unique_ptr<T>> would be completely feasible in C++.

It is just that std::optional and unique_ptr do not have the right semantics out of the box, but it is not hard to roll your own.

But yes, I wish C++ has good builtin sum types and proper destructuring via pattern patching. Maybe one day....

The problem with adding a non_null_unique_ptr<T> to C++ is that moving needs to leave behind a valid value.

You are right, but you could make it not movable (and add destructive mode hooks for optional and containers).

It is still possible to return non-movable, non-copyable types from functions.

But I guess ergonomics would not be good.

I love sum types, and i find it very hard to work without them when i am in C++.

However your examples i don't think are particularly convincing.

The examples you gave can mostly be done with std::variant<> and are simply examples are unions. (Result == std::expected)

In term of non nullable smart pointer, it would be easy to enforce the the non nullability in the constructor, and have the same ergonomy as Option<Box<T>>. In fact you can do something like std::expected<std::reference<T>> but why would you instead of just std::unique_ptr and handle the nullable case.

What makes sum type powerfull is to have full Algebraic DataType (tags + sum types + product types) + pattern matching + good compiler errors.

A Union in itself is not a sum type. Rust even has unions:

https://doc.rust-lang.org/reference/items/unions.html

The unions of Rust are tagged unions that can hold data:

https://doc.rust-lang.org/book/ch06-01-defining-an-enum.html

which is immensely powerful in data modeling.

The main difference comes with the syntax sugar that's built into the language (and the additional information the compiler has).

Sum types are also quite nice to bridge the gap from a 'dynamically typed mindset' into a 'statically typed mindset'. Sum types basically provide the ability that a variable can assume different types during its lifetime (? although I don't know if such 'shape-shifting' is allowed in Rust), so it almost feels like Python or Javascript, but since the compiler 'knows' sum types it's all still completely typesafe.

In Typescript it's convenient and frankly quite nice, in a programming language like Rust where performance matters I'd be less enthusiastic though.

> I don't know if such shape-shifting is allowed in Rust

I think so! You mean this?

    let mut x: Result <bool, String> = Ok (false);
    dbg! (&x);

    // Get a reference to the bool and mutate it
    if let Ok (b) = &mut x {
        *b = true;
    }
    dbg! (&x);
    
    // "shape-shift" to the other enum variant
    x = Err ("No boolean :(".to_string ());

    dbg! (&x);

It runs in Godbolt https://godbolt.org/z/q6jqYdYh1

You can make a tagged union in any language, and that can often be useful, but when it is a standard part of the language you prevent any accidental cases of:

1. failing to check for sigils, because nobody uses a sigil when you have sum types. This would have prevented a famous exploit in SUDO a couple years back where someone forgot to check for the special negative value on an input parameter.

2. Eliminating null pointer exceptions or anything similar. You can't forget to check, you can still choose to have this kind of error, but you have to do it explicitly.

Consider something like a function to open and give you back a file handle. In many languages such a function will return a pointer to a file, or null if there is a failure, or perhaps throw an exception if there is a failure. If you neglect to check for null, or forget to handle the exception, you have a problem. With sum types, the natural thing is to return something like a Result type, which is EITHER a file, or an error. And you have to check which it is, to get the File out.

This is one of these things that you don't know you're missing until you actually start actively using. Back in the day when I was writing a lot of C++ I was saying the same thing as you.... until I wrote a significant program in Haskell, where sum types are also very common. And once it clicks it's hard to go back.

I flipped through one of my Rust projects and the best use case I saw was for serializing messages for disk / network / IPC:

    // This derive macro expands to a visitor pattern during build
    
    #[derive (serde::Serialize)]
    enum Message {
        QueryFoo (QueryFoo),
        MutateBar (MutateBar),
    }
    
    struct QueryFoo {
        foo_id: i64,
    }
    
    // If we receive a Message::MutateBar from outside our process, 
    // it's a compile error for us to accidentally access the field `QueryFoo::foo_id`.
    struct MutateBar {
        bar_id: String,
        baz_value: u32,
    }
    
    fn send_msg_as_json (tx: &mut Sender, msg: &Message) -> anyhow::Result <()> {
        // The JSON serialization / deserialization is generated, too
    
        tx.send (serde_json::to_string (msg))?;
        Ok (())
    }

It's possible to represent this in C++, and you can integrate FlatBuffers or whatever to auto-generate the ser/de code, but then you're not writing C++, you're writing some weird DSL and you're fighting the build system and you're teaching juniors how to install it.

I haven't seen anything in C++ that's both as easy and as airtight as a Rust enum with serde's automatic serialization macro on the top.

FWIW using std::variant (or whatever is your in-house variant) to store message types is very common.

Also in well designed low level binary protocols message types have a discriminant header already so often common to wrap them in an ad hoc-variant which doesn't have a separate discriminant.

We do not use unions very often in C/C++ because they are not safe. You have to know how to interpret it; the compiler can't and won't save you if you read the memory as the wrong type. You usually keep a variable set to the type the union is, and check it at run time.

Some languages do that wrapping and checking for you, automatically. Everything from Ada (variant records) to Haskell has them. You cannot mishandle such objects by misinterpreting them as the wrong type; it's a compile-time error in those languages to do so.

Imagine you are parsing something. In my opinion, the most natural way to express this, is a function that takes an array of characters and returns an array of parsed objects or tokens, where each object may be of a different possible type. In C, you'd probably use a union of structs, and set which type of struct as you parse each object. This is error prone and verbose, and the compiler can't double-check it for you. It's all automatic in a language with sum types. Quite efficiently too; most of the runtime checks can be optimized away.

Try representing JSON in C++. Sure, you can figure out a way that works, but the C++ code for printing out the JSON is much uglier than a language with tagged unions.

That is a consequence of lack of reflection, not sum types.

sum types can often eliminate the need for reflection, and can have lower runtime cost.

Compile time reflection doesn't have runtime cost.

Compile time reflection doesn't work for this case.

Template metaprogramming techniques as poor man's reflection prove otherwise.

Additionally that is one of the examples on the ongoing C++ reflection papers.

Then there are a couple of examples in D, Common Lisp, Scheme, C# code generators.

No? A JSON object could have any type at run time. You cannot do anything at compile time to figure out the right type.

Please point to a specific library if you disagree.

Java has sum types for a while now, and quite ergonomic with records.

Kotlin also runs on JS and Wasm these days, not just the JVM.

provided the language later lets you split them apart and declare a branch of code as only dealing with one subtype

Reply on news.ycombinator.com