Rust for JavaScript Developers – Pattern Matching and Enums | Hacker News Reader

Rust for JavaScript Developers – Pattern Matching and Enums

sheshbabu.com

109 pointsrkwz39 comments

Enums are my favorite type in Rust. You can do so much with them, and even more using 3rd party crates such as strum [1], allowing you to do things such as iterate over variants and decorate each variant with custom properties. How is this useful? Here's one example: an enum's variants can map to the values of a reference table in your relational database. When you seed your db, iterate over the enum variants and load into the ref table the properties associated with each variant (the properties you associated using strum). When you query values from the reference table, resolve the value to the enum type. You'll never have values in your table/model that are out of sync with your enum in rust.

[1] https://github.com/Peternator7/strum

Having used enums with large teams across many distributed systems over the years, enums are my least favorite type in that setting, because it's very easy to write an enum that does not tolerate unknown/new values (they usually don't by default, which seems to be the case for Rust as well). This often makes the system brittle to new changes because subsystems have very strict expectations of data schema, even when they don't really need to care about it (e.g. when they're just passing data around).

I've seen a lot of attempts on these teams to ensure these mistakes aren't made (special parsing libraries, using default fallback values, etc.), but someone always manages to sneak in an enum somewhere where they shouldn't and then something blows up 6 months later when a new value is introduced. Enthusiastic engineers will spend a lot of time trying to figure out how to add an over-strict enum, because hey, who doesn't want type safety?

Never trust anything coming in over the network. Always decode with the possibility of failure. I.e., a decoder function from a byte array would return a `Result<EnumType, Error>` where `Error` should usefully describe the failure.

Sorry to say but JavaScript is the only ecosystem I've seen where people blithely `JSON.parse` anything they get over the network and directly access the JSON structure expecting everything they want to be there without checking.

Enum types are supposed to be strict. Decoding raw data into enums can always fail but the failure should be handled at a higher level.

I had some problem like this using protobuf2 enums

The way protobuf handles enums, it recommends having an "UNKNOWN" as value 0, as when deserializing from the wire any unknown enum value gets turned into value 0.

So it's recommended to have something like

enum FailReason { UNKNOWN_ERROR = 0, ERROR_1 = 1, ERROR_2 = 2, etc }

But someone long ago made the enum I was dealing with look like this

enum { SUCCESS = 0, ERROR_1 = 1, ERROR_2 = 2, etc. }

But this means I can't easily add a new error case, as it'll get interpreted as a success unless I carefully roll out the change to the receiving servers first.

But if I wasn't dealing with a distributed system I would like to have exhaustive checking.

I don't think that's protobuf's fault, it's simply a side effect of the common pattern in C of using memset when you initialise structs. I don't think there's a really good way around it other than remembering to think like a C programmer every now and then.

I guess one option is to replace the endpoints that use the enum, but that seems heavy handed if it's wide spread.

I am not sure this has anything to do with enums. When sending serialized data over the network you always have to have logic for dealing with the situation where what you get isn't quite what you expected. Not doing that is what causes things to break, not using enums.

In my experience, these kind of bugs happen overwhelmingly with enums. The nature of enums makes this mistake pretty easy to make. Enums are otherwise fantastic tools, I’m just pointing out that in certain common contexts they have some ergonomic problems.

I'm not sure I understand. If you have some code which doesn't care about the types which it is passing around, why isn't it generic?

Also, I'm not sure if you understand Rust enums. They should probably be called union types, because an enum in Rust signifies that the value can be one of a number of different types. This is extremely useful for code which has to handle a few different things differently. It's much more useful than the standard concept of an enum in languages such as Java and Rust, where it's just a dressed up integer flag.

Rust prevents the program from compiling if a new enum variant was not handled, unless one uses a catch all in the match expression. Did the codebase have an over-usage of catch alls? Or were you using a different language that doesn’t catch enum errors during compilation?

I think the key words here are "distributed system". Just because HEAD has the new enum used everywhere doesn't mean all the deployed code everywhere can handle the new enum value.

Imagine you have two servers communicating with

enum Packet { Ping, Pong, }

Where the semantics are that if you receive a ping, you say pong, and if you see pong, you say ping.

Suppose we deploy this out to several servers, with code built on 2020-7-12. Now on 2020-07-13 you commit to head a new enum value `Stats`, which returns a new kind of reply.

If you send outgoing requests in the same 2020-07-13 build, you're going to have a bad time, as your fleet is going to start at 100% 2020-07-12 and switch over to 100% 2020-07-13, but during that switch you'll have some 50% 50% split. If a new server sends a request to an old one, you'll have a crash.

Instead you have to add receivers in 2020-07-13, but not use it yet. Roll it out, make sure it's past how far you might roll back, then you can deploy a 2020-07-20 build which makes calls using the new enum.

If you skip that step, you're going to have a bad time. Maybe you want to instead have a default case handler instead of crashing, but that means each of your enums has to know about "UNKNOWN" and you're deserialization error has to be future proof.

Any time you break an API that accepts serialized data it's going to crash if you try to use it with something that is on a different version of the API. How is this any different with enums than it would be with changing a string to a float, or anything else?

Related, capnproto has a corresponding feature (which it calls unions), but being a dsl for defining _protocols_, you obviously have to deal with the possibility of introducing new variants as the protocol evolves.

I wrote the Haskell implementation. It maps unions to variants (what Rust calls enums), BUT it always generates an extra 'unkown' variant, which gets used whenever the variant is one not recognized by the generated code.

In this case the value proposition of variants/enums, including exhaustiveness checking, is still really useful -- it can not only deal with the possibility of more variants added in the future, it forces you to handle that possibility, which imo is the best of both worlds.

But yes, in general, when modeling data that comes from the outside world, you need to about overfitting.

What happens when you have a mix of new and old binaries being incrementally rolled out?

Good point, but isn’t that a more general code compatibility issue that applies to many things beyond just enums?

> This often makes the system brittle to new changes because subsystems have very strict expectations of data schema, even when they don't really need to care about i

I deal a lot with this, because I interface with software outside my control. In the case of Rust I find super easy to solve, just use this litte trick:

https://serde.rs/attr-flatten.html

#[derive(Serialize, Deserialize)] struct User { id: String, username: String,

     #[serde(flatten)]
     extra: HashMap<String, Value>,
 }

Serde is very good. Near all you want is there.

this is a great recipe

What I'm referring to are fields that must be one of the expected values. It seems like you're talking about a scenario that doesn't meet that criteria. One of five values or null is simply an Option<MyEnum>.

Question for the author:

Why does Rust destructuring require the struct name? The example uses:

  let Person { name, city } = person;

This seems a little verbose since I would expect the type system to know that person is of type Person already.

Is this because destructuring syntax is trying to be compatible with the pattern matching syntax?

You will find pre-RFCs[1][2] in https://internals.rust-lang.org/ about this, as well as for anonymous structs[3].

It isn't in the language yet due to a combination of subtle issues that need to be considered before a final design can be reached, trying to keep the grammar clean, accepting that explicitness and verboseness now is better than hastily moving towards something that seems better but that can introduce technical, documentation or understandability issues.

> Is this because destructuring syntax is trying to be compatible with the pattern matching syntax?

It's not trying to be compatible, it is pattern matching syntax. If the language changed to, for example, allow the following

    let _ { name, city } = person;

Would also allow

    match opt_person {
        Some(_ { name, city }) => {}
        None => {}
    }

[1]: https://internals.rust-lang.org/t/pre-rfc-struct-constructor...

[2]: https://internals.rust-lang.org/t/pre-rfc-implied-type-for-p...

[3]: https://internals.rust-lang.org/t/pre-rfc-anonymous-struct-a...

Not the author, but yes, this is because the thing between "let" and "=" is a pattern, the exact same thing used in match, and in other places. For example, function arguments!

  fn takes_person(p: Person) {

  fn takes_person(Person {name, city}: Person) {

But I wouldn't inherently say that's the reason, it's also because structs are nominally rather than structurally typed.

I appreciate the author’s use of code style to make it easier to read the examples. While JS is usually camelCase they choose to use snake_case for functions like operate_drone.

I also appreciate the lesson of converting discriminated unions in JS and to rust - that’s my favorite pattern from the TypeScript community.

Another great crib sheet for JS folks to think about Rust. What I like about this, rather than say the Rust book, is you've given examples that map closely to other language paradigms without ignoring the fact that rust handles them differently.

What is the reason for using different syntax for Move and TakePhoto from the Operation enum in the penultimate example? Is there a name for each of those enum member types?

Enums have 3 different variant types:

    enum Foo {
        StructVariant { named: i32, fields: String, with: (), types: usize },
        TupleVariant(i32, String, usize), // only types and position information
        UnitVariant, // just a tag, no other days being carried
    }

The struct variant is useful when you are carrying enough different fields that you want to name them.

Great stuff. Thank you :)

JavaScript and Rust are so opposite in their goals and design that I wonder why something like this is written...

I have professionally come across many website backends that were developed in Javascript. Over time we are seeing projects that add more robustness, performance, and ability to support in the form of Typescript and now Deno.

With WASM becoming mainstream and the growing maturity of the Rust ecosystem, it appears to be a solid choice for backend systems and potentially emerging frontend use cases.

If they wanted something better for the backend (for robustness of performance reasons) there are other alternatives much better suited for the vast majority of backends. For instance, C#.

WASM is not needed for the backend, so that is not the reason either. For frontend, I can see it for some pages (I used it myself), yes, but there are also other languages that can target WASM.

You also mention maturity of Rust, but other languages in its domain are more mature, so that cannot be it either.

WASM is not required, but having a sandbox around native (e.g. Rust) code seems like a good idea, and it allows you to mix languages and opens other ecosystems.

No, it is not a good idea for backend where your processes and code are trusted because it does not come for free (it comes with a big performance penalty).

Mixing languages does not require WASM in native code either. It has always been done through the C ABI.

No process nor code is trusted in my company. The performance penalty is negligible compared to the multi-million risks, and WASM execution can be optimized a lot as it goes past MVP.

I don't know how to import a ES6 module using C ABI.

> No process nor code is trusted in my company.

Then you cannot use any modern computer in your company. Much less hypercomplex systems like Node, v8 and JITs in general.

> compared to the multi-million risks

What "risks"? What standards are you following? Insurance?

> WASM execution can be optimized a lot as it goes past MVP.

Citation needed. There is no magic.

> I don't know how to import a ES6 module using C ABI.

So that is why you want WASM?

V8, JVM, CLR and others are well proven runtimes providing a well functioning sandbox, but of course we test a lot. I am sure you know what the risks are with native code - you're a native code proponent after all; any code running in a medium and bigger company that could misplace money is a huge liability.

I don't know what magic you're talking about - we're simply waiting for WASM to standardize GC, SIMD, tail calls etc, you can read more here[0]; and to get the execution optimized like V8 did over the past 10 years[1] (WASM on V8 gets most of these gains by default as a bonus).

I already said why I want WASM: because it provides a good sandbox and opens most ecosystems through safe and typed interop. I don't care about ES modules in particular - I care about ES modules, Ruby modules, Python modules, Prolog modules, Fortran modules, C modules, Rust modules, AssemblyScript modules, COBOL modules, Erlang modules, F# modules, Java modules, and so on - at once. Finally a future where I can use and trust any library and not get limited to the language I use (if I don't want to spend most of my time fighting and securing C ABI interop, if at all humanly possible) is there.

[0] https://github.com/WebAssembly/proposals

[1] https://v8.dev/blog/10-years#performance-ups-and-downs

Wouldn’t that imply that it’s needed more? The more different two things are, the harder it is to get from A to B, and the more a map is useful.

The question is whether enough people need such a map.

In that case: yes, we have a lot of JS people coming to Rust. The difference helps explain that too: if you’re going to learn something, learning something that’s very different from what you’re used to means you get more out of it than learning something similar.

When I decided I wanted to move away from Ruby, I didn’t even consider getting deeper into Python. Why would I? They’re so similar that the cost/benefit ratio is off, at least for me back then.

YMMV of course.

Oh and finally, not every bit of writing has to be extremely useful or hit a wide audience. Sometimes people write things because they want to.

Good points, but when I have done that, I always preferred to start fresh, rather than take a "shortcut" map from A to B.

The reason is that otherwise you always try to map things back to your previous knowledge and expertise, which can make you biased or give you tunnel vision.

"If you have a hammer..."

Yeah, I totally get that. Different people learn differently. :)

I think you just answered your own question.

Reply on news.ycombinator.com