Implications of Rewriting a Browser Component in Rust | Hacker News Reader

Implications of Rewriting a Browser Component in Rust

hacks.mozilla.org

464 pointszwliew268 comments

This is in tune with my own experience using Rust in production: it can stop you from doing certain classes of mistakes, but it won't stop you from doing stupid things.

But the idea that I don't have to think about certain classes of problems allows me to give these stupid things more focus, which is surprisingly refreshing.

The predictable nature of Rust was so refreshing for me that I ended up using it even for smaller reusable scripts where I would happily have used Python before but soon got annoyed with obvious errors that would only show up once you run a program.

If you e.g. have a `print foo` in some obscure branch that rarely happens, that print will ruin your day if you use Python 3. If python would be a little like Rust you would get on save (or at least on compile) a hint or error, that the print should look like this: `print(foo)` for Python 3. You can be incredibly careful and rust will still catch things now and then, that would have gone unnoticed into production unless you have immense test coverage.

I like Rust for the experience I had with it. It definitly changed how I approach certain problems in a very good and productive way, even when I don't use it.

Python will actually give you an error for `print foo` as soon as it parses the file. But there definitely are other scenarios where you'll only get the error in the middle of execution (such as passing the wrong type of thing as an argument to a function)

Agreed but Python lazy-loads & parses files as packages are imported. By convention these are all at file scope and at the top of the files, so it's often confined to an initialization phase. But...sometimes developers use clever fallback behavior by catching ImportError.

So the scenario described is possible to escape simple tests, I suppose.

Doesn’t the print raise a SyntaxError not ImportError?

Yes, sorry if that was confusing -- perhaps I shouldn't have brought up ImportError. The point was that uncovering SyntaxError could possibly be as challenging as other feature tests.

    try:
        import collections
    except ImportError:
        # fallback impl of collections:
        import something_that_uses_print_statement

A linter would catch that.

Agree completely. For a bit of my own story, a year+ ago I had the option to write a project in Rust and evaluated it vs Go. Long story short, I tried rust, and it was a massive headache and I failed. We used Go (as I had been for ~4 years).

Fast forward to ~2 months ago, a work project dictated tight control over memory which, while possible in Go, had me looking at alternatives. I decided to give Rust another try. This time it wasn't just an evaluation, it was needed to work so I bought and Rust book and spent some after hours time learning/etc.

This time, Rust has been an absolute joy. I have no understanding why last time was so painful, and this time it's been so amazing. Maybe it was the book[1]? Maybe it was just a 2nd round of learning based on my previous experience? Regardless, it's been great.

There's just so many mental overheads like what is concurrent safe, what is non-null, etc that are just great to not think about anymore. On top of that, the formatter and LSP are just great. It highlights in my text editor (Kakoune) what variable caused an error, where it gets moved incorrectly, etc. So much just works, it's great.

My only complaint these days is:

1. I find it odd that some things like slice reads can still panic by default. Yes, I can use `foo.get(1)` to avoid panics, but still - it's a bit odd to me. 2. I'm anxiously awaiting async/await. It's quite difficult to be patient.

[1]: Programming Rust: Fast, Safe Systems Development

It’s probably a combination of both things, but “I tried rust, struggled, quit, came back months or years later and now I have no idea why I thought it was so hard” is certainly a recurring pattern we’ve seen. Glad it’s working for you now!

Because people underestimate the amount of effort needed to learn a new language at first?

I'm not sure it's that simple. I've never heard anyone say this about Go, for example.

To be fair, Go is uniquely designed to be easy to learn and become productive with quickly. It was practically a design goal of the language.

Sure. I think it's one of Go's strongest points. But it's still a counter-example to the parent's point. Learning a new language does not inherently require a lot of effort, and so expecting it to be reasonably easy, especially when you're an experienced developer, doesn't seem unreasonable to me.

Go does still suffer from people bringing idioms and assumptions from other languages, and that has leaked somewhat into common libraries. I've been away from it for a while (mostly doing very low-level C on microcontrollers these days), but there was a lot of Go code I came across that just screamed "This was written by a Java developer who is learning Go!"

I had a great time writing Go web services, and I definitely agree that it was easy to pick up, both for me and other teammates. Figuring out the idiomatic way to do things wasn't always straightforward though, although getting up and running was easy.

Have not used rust but I'm assuming it's because when you're down a rat hole rust stops you cold at every turn. You need to back up and rethink what you're trying to do. But when you don't have a good handle on what rust is complaining about you can't see that.

The books are incredibly well written. When people are “stopped cold at every turn” they are usually programmers who come to Rust with a metric ton of existing assumptions how to solve certain problems, without really thinking about the concepts of the language itself.

When a C++ programmer starts with python they will write incedibly “unpythonic” code, when they do the same thing in Rust, it just won’t compile.

IMO Rust is very straightforward and the std library is incredibly good. But you have to really understand certain core concepts and what they mean in terms of structuring your code.

Just like that C++ programmer needed to learn what is ideomatic python looks like and why it makes a lot more sense to write python that way, you need to find the rust way of things. But this is the same for every progtamming language out there..

You've restated the point I was trying to make much better than I did.

I agree entirely.

One of the reasons why when I learn a new language I deliberately research the idiomatic way to write that language, it saves a bunch of trying to hammer a square peg into a round hole time.

Also some languages purely because of the domain they are in can do things in ways that others can't (trivially) do so if you treat each language as "foo" with different syntax you end up hobbling yourself to the lowest common denominator of "foo"'s features across every language you use.

There's plenty of languages which take limited / little efforts to learn. So I'd say more that the unfamiliar concepts need to stew or sink in.

One way is to just keep at it until it clicks. An other way is to not do that, but once you've got the words for something you start seeing the issue everywhere, and next time around it makes a lot more sense.

For real Beginners to programming Rust can be hard, because concepts like pointers are hard in most languages that use them.

For Programmers that try Rust out Rust can be hard because they are stubornly trying to program Rust as if it were Python/Java/C/C++/Foo – but it isn't.

Rust as a language makes certain types of approaches (or anti patterns) nearly impossible. In the beginning it happens quite often that after a day of struggle you will erase a Rube-Goldberg-Machine that does something simple and replace it with the right line of code that does exactly the same while beeing way more extensible.

If you find yourself building more and more elaborate structures just to get something very simple done, the problem is very likely your approach (e.g. trying to implement a OOP structure instead of solving the problem) – at least that is what happend to me a lot.

Rust's borrowing and ownership concepts, as well as traits and generics make it extremely powerful, but these concepts lend themselves better to certain ways of structuring code than to others. Learning something is in my opinion always about allowing something to change your perspective. If you are not allowing the thing you are learning to change you, chances are that you are not learning, but judging.

From my friends that have tried rust, creating a graph or doubly linked list without using some special structures is fairly painful or impossible. You can create a graph by using some sort of adjacency matrix or some other kind of structure that keeps ownership in some sort of tree without much pain, but that has it's own downsides.

This was also my experience. It's a fine language if you can't (or more usually "won't" for spurious reasons) use a garbage collector. However a lot of things are just easier to do using a GC, and I suspect in many, not all, of the use cases of Rust, a GC would be just fine.

(Also I have to rant here a bit: Yes I know the GC you used in Java or Emacs in 1997 was terrible, but modern GCs are very good indeed)

The garbage collector in 80s MS BASIC was incredibly inefficient. The routine was written on to minimize the amount of memory to perform the GC, something like 8 bytes.

It would simply iterate through every entry in the temp string stack and then all the entries in the symbol table to find the string with the lowest address above whatever was the previous lowest address. After that full sweep, it would move that string to its new home and adjust pointers to it, then set the new low bound to the string just identified. It would keep iterating until all strings had been visited.

As a result, it required (n2)/2 sweeps of memory (n=# of strings). Having BASIC lock up for 30 seconds every once in awhile was just how it went.

[ and in case someone is going to correct me, it might have swept from top of memory to bottom, I have forgotten the details ]

Graphs are OK in reference counting languages like swift / objective-c, it doesn't have to be done with a GC.

I think you could use Rc<Thing> in rust too if you want to solve it using rust, but then you have "swift with more boilerplate" as one friend said.

Ref counting is a poor kind of garbage collection. In particular it has a very large overhead: typical GC has an overhead of a couple of bits per allocation, but ref counting usually wants to allocate an extra word (eg. 32 or even 64 bits) per allocation.

Another actually more serious problem is that typical ref counting implementations mix the allocated data with the references, so that when you inc/dec the reference you dirty a cache line containing the data. There exist implementations which improve on this by storing the references in a separate part of memory, but they are not how it's commonly implemented.

...annnnd this is probably a good example of the frustration that comes from solving familiar problems in a new paradigm, before the lightbulb clicks on.

Disclaimers: I'm only lightly familiar with Rust, and linked-lists are perfectly reasonable solutions _in the proper context and paradigm_.

So the problem to be solved is that we want to a) have collection of items which have a very strict ordering; b) be able to directly access the first and last item in that order; c) given a particular item, find the item which is immediately before/after it; d) when removing or adding an item, the relative ordering of the other items is undisturbed. Oh yeah, and e) we want it simple and efficient.

Doubly-linked lists are, of course, a very common solution used for these kinds of problems. They are perfectly suitable _if our paradigm is_ "I hereby assert that somehow, external to the compiler, I've verified that all list manipulation is being done correctly in all circumstances." If, however, our paradigm is "I want the compiler to automatically guarentee lots of these invariants" we run into some problems:

1) The data structure itself (each item has two pointers, and there are two global pointers for "head" and "tail") makes very few guarentees. Just one, actually: "each pointer will either point to an object, or will be null (or its moral equivalent)". There just isn't much compile-time meat for the compiler to chew on.

2) Linked-lists typically have (relatively) persistent scope, and exist outside of the lexical scope of whatever code block is immediately operating upon them. Again, it doesn't give the compiler much to go on.

3) Without managed memory (GC), there's no way for the compiler to guarentee that a pointed-to object still exists.

4) There's no built-in guarentee that the "head" and "tail" pointers actually point to the first/last object.

5) There's no guarentee of overall ordering (if a.next == b, then b.prev == a).

6) There's no guarentee of even a consistent view of the items in the collection (head == a, a.next == b, but b.prev == null/end-of-list).

7) There's no way for the compiler to guarentee, when viewing the collection as a whole, that modifications are atomic or thread-safe.

Yes, it's possible to take care of these issues in a non-Rust paradigm by making the structure of the list itself opaque, and only exposing insert/remove functions which enforce all the invariants. (I think Rust itself can do this with unsafe code.) However, you're still left with difficulties:

8) If list items are opaque containers, how do you guarentee that the payload object continues to exist?

9) How do you guarentee payload object ownership, atomicity, mutability, or other properties?

10) How do you guarentee that these needed invariants are held transitively?

Now the Rust paradigm, of course, isn't the only way to deal with these issues. But whether you're using Rust, managed-memory, C++ smart pointers, immutability guarentees, etc., you're going to need to do things that are unnatural in the other paradigms. Rust has the benefit of automatically enforcing lots of these invariants without having to do copying, worrying about shallow vs. deep copying, transitivey, etc.

Both type inference and borrow checking have also received a whole bunch of small improvements over the past couple years. Subtle enough that you probably wouldn't notice them except in that the language just takes less effort to use.

If you use Rust 2018 now, non-lexical lifetimes are a bigger recent improvement: https://doc.rust-lang.org/nightly/edition-guide/rust-2018/ow...

Also error messages have had a lot of effort put into them, so an error that would have been confusing a few years ago, now may no longer be confusing.

This is true, but it also happened a lot even before NLL. That said, I do think it's an additional factor today, you're right.

> 1. I find it odd that some things like slice reads can still panic by default. Yes, I can use `foo.get(1)` to avoid panics, but still - it's a bit odd to me.

I wonder if this is similar to C++'s `[]` vs `at`. `at` does implicit bounds checking but, as an optimization, if you are already doing an explicit bounds check, you can elide the implicit check via `[]`.

It's somewhat similar, except C++'s [] leads to memory unsafety whereas Rust's [] on slices has bounds checks and will abort the program on failure. Your program will crash but it won't be an exploitable bug (except for a DOS). .get(), meanwhile, returns an Option so that code written using it can recover from OOB accesses.

Except Rust's [] behaves like C++'s at (checks and aborts). C++'s [] is called `get_unchecked`.

C++ at() doesn't check and abort, it checks and throws a specific documented exception that you can catch. So it is, in fact, closer to get(), just using a very inefficient way of reporting.

> C++ at() doesn't check and abort, it checks and throws a specific documented exception that you can catch. So it is, in fact, closer to get(), just using a very inefficient way of reporting.

You can catch a panic, and you can compile C++ with -fno-exception. at() is not closer to get() than to [].

If you do that, you will be invoking nasal daemons, as at() is required by ISO C++ to throw.

thankfully, ISO C++ is a language that no one actually programs in, everyone use `MSVC C++`, `g++ 8.2.1 -fwhatever`, etc

On real world where code portability actually matters, many do program against ISO C++, and have to deal with workarounds for lack of compliance.

Not doing so means ending up with situations like the Linux kernel, Windows or console games, which might be ok, when code portability doesn't matter to start with.

It's more about ergonomics. With `get` you get an `Option` which allows you to handle the out-of-bounds situation, but is unnecessarily noisy when you know that the index must be valid unless the program is buggy (and in that case you most likely want to abort anyway).

How do you like Kakoune? It's the first I've heard of it

Love it - was a Vim user for many years before that. I will say that while the Unix-y design is really cool, the plugin system written around Unix (Bash) is a bit of a pain. You technically don't need to use bash at all, but idiomatic plugins tend too.. so.. meh.

These days I'm wanting out of the Terminal - but wherever I go, I'll need to take Kakoune's style of modal navigation with me. I love it too much.

_(For reference, I'm hoping Xi editor implements the foundation of modal editing, and then I can make Kakoune style editing work in Xi Editor)_

> I find it odd that some things like slice reads can still panic by default. Yes, I can use `foo.get(1)` to avoid panics, but still - it's a bit odd to me.

Panicking is a perfectly good way to handle the situation where an invariant of the program is violated. That is, it is perfectly fine to index using [] in cases where there cannot possibly be an out-of-bound access unless the program has a logic bug. And if it does have a bug, there's usually not much that you can do except abort because you can't trust the program anymore.

On the other hand, if you know that your input might be OOB without violating a precondition, as a part of the normal execution of the program, and you can handle it gracefully, use `get` instead.

This distinction between "expected" and "unexpected" errors is an extremely valuable one, and one unnecessarily muddled by languages where all errors are signaled with exceptions. Rust gets it right.

I don't understand, tbh. Eg does Rust not make great effort to prevent null pointer exceptions? OOB feels quite similar to null pointer.

It's strange to me because a lot of what I value in Rust is writing code that handles all outcomes safely and with confidence. Yet that all seems to be out the window if you access array values - as it may or may not work.

The safety of [] slice/array access is entirely up to how the dev wrote the code.. which largely feels the same as any other language.

Am I thinking about something incorrectly?

The usage pattern is somewhat different. The primary problem with null references in languages such as Java is that they are not opt-in, and they're not even opt-out! That is to say, you don't even have an option (heh!) to encode in the type system the fact that a reference is never null. What's worse, references tend to get passed around a lot, and stored inside objects, and programmers are lazy and don't assert or even think about preconditions, so the root cause of an unexpected NPE can be far removed from the code that finally triggers it.

With code that indexes slices, the index computation is usually much more proximate to the actual indexing operation. If ensuring the validity of the index is about upholding an invariant internal to the module, it makes no sense to return an "sorry, I have a bug" error to the caller if that invariant fails to hold. What would the caller do about that?

I think the slightly odd or inconsistent thing is that rust has [] at all. I think of it as a compromise for people's familiarity with the operator. And if that's all it is, it makes sense that it behaves the way it does; I think a panic is more similar to what other languages with that operator do with an out of bound index than returning Option would be.

> I have no understanding why last time was so painful, and this time it's been so amazing.

It has been my experience that if I try to learn something hard and give up and then come back to it a few months later, I have a way easier time even with no apparent changes in process. I suspect some of it is giving your brain time to process fundamentals or something.

This seems true and unsurprising - it strikes me as being similar (or identical) to spaced repetition, which is a powerful learning technique.

I wonder if it has something to do with the nature of the problems. Rust can be very smooth or a real pain, depending on the data structures you need to model - basically, whenever ownership is unclear and sharing is common, it's going to be more painful.

Usually that just means you have to think much harder about the problem you are trying to solve within the new set of requirements (borrow checker, lifetimes).

Weirdly enough these things should also be a hard problem in other non-Garbagecollected languages, but you get away with a lot more when the compiler doesn’t force you.

From a purely practical level there is so much already that I had a hard time finding some data type that had no existing and tested solution for it. Looking at the code in the std library can be enlightening.

> Weirdly enough these things should also be a hard problem in other non-Garbagecollected languages, but you get away with a lot more when the compiler doesn’t force you.

It's the same as with static vs dynamic typing - some things are just easier to model with dynamic, because you don't have to fight with the type checker to prove that your program really is sound when you know that it is. But, of course, you might believe it to be sound when it really isn't...

> before but soon got annoyed with obvious errors that would only show up once you run a program.

From what I gather you are comparing static vs dynamic typing, and I agree. I do not use Python because I prefer a subset of errors caught at compile-time. However, it is silly to make it sound like that this is somehow limited to Rust. You could just as well have used Go or OCaml and feel "refreshed" because obvious errors would have been caught at compile-time.

> The predictable nature of Rust

Again, given the context, it is static vs dynamic typing. This is not something limited to Rust. In case you were not talking about static typing and its pros, could you please tell me what kind of official formal proof tools exist for Rust that makes it predictable?

I assumed borrowing and lifetimes were part of his experience as well.

Sure, and I find both not very hard as a concept.

The hard thing is not understanding the concepts but learning what this implies for your coding style, especially when you come from a garbage collected language like Python.

Both concept can at times give you hard to solve tasks, but it usually is because the problem you are trying to solve is hard and you cannot just naively hack away without thinking long and hard about it.

Other than that I learned liking the borrow checker and lifetimes because you will get a incredible narrow scope of which variables can matter when — which makes debugging much easier

And with something like Mypy, you can get most of the benefits of static typing even in Python, if you want to. But Rust has so much more to offer, such as memory ownership.

I didn’t know mypy, but I am going to check it out, thanks!

That’s one aspect of the errors that are avoided, but there are many others too - some of which aren’t solved by other languages. Go, for example, will happily nail through all of your obviously null pointers with gay abandon. The Rust ownership model prevents other classes of bugs.

It’s not a panacea—and to be blunt, I find Rust more work than it is worth for the sort of projects I typically work on. But any tools which can eliminate whole error categories are worth looking at for sure!

I was responding to the issue I quoted, which really is just a matter of using a programming language with static typing. :)

> some of which aren’t solved by other languages

You only mentioned null pointers, so I will go with that. In Ada, you can have access types (pointers) that are guaranteed to not be null, and accessibility rules of Ada prevent dangling references to declared objects or data that no longer exists, so this particular issue is solved by a language other than Rust. Please feel free to give me other examples of errors or issues that you may believe is not solved by languages other than Rust.

https://www.adaic.org/resources/add_content/standards/05rat/...

> But any tools which can eliminate whole error categories are worth looking at for sure!

I agree. That is why I think Ada/SPARK is awesome! :P

Ada is awesome, and if compiler writers had been more timely it might have taken hold. Rust solves the same problems without a runtime, and about 3 times the performance. Granted it looks like c++ and ocaml made a particularly ugly child.

Which problems are you referring to?

You can do everything that has been mentioned without Ada's RTS, you can also disable all run-time checks. You can use static analysis tools (there are many, and available for free), for example, you can formally verify the correctness of the program, no run-time checks required.

>> [...] solves the same problems without a runtime, and about 3 times the performance.

I think you should look more into Ada. The only "runtime" is for the exception handling and bounds checking, both of which can be turned off if needed.

And I don't know where you got that "3 times" figure from? Do you have an example you are referring to?

I knew you could turn off the runtime but not about the formal verification lint tools that make it safe to do so. My reference for performance is the benchmark game. If you or someone else can mod those programs to beat rust I'd be thrilled. I like Ada.

Many planes, trains, rockets and medical devices run on Ada.

There is more to commercial compilers than winning the benchmarks game.

Those things don't need to be particularly fast. What they need is consistent and predictable performance and resource usage.

Planes trains and rockets have more sensors than ever and speed is very important.

Most of my experience is embedded c++ but I have written ada rust and a fair bit of jovial73. Rust is a perfect fit for those same domains. Most o the things that ran Ada in the US have now switched to c++. Do178b certified safety critical c++ is a horrible thing to develop. I'm hoping rust comes in to fix that.

Sure, safe system programming languages should be seen as allies to the common goal of improving the quality of our IT stacks.

For sure.

The benchmarks game is just an easily available source of examples.

Which, for instance, may show an Ada program with much the same measured time as a Rust program —

https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

I believe the performance differences on The Computer Language Benchmarks Game is largely due to the implementations themselves. For one, pidigits have been translated from Pascal to Ada using p2ada. I do not think that this is a fair comparison at all.

I agree on that one, I'm rusty at ada, bad pun, but someone should profile and fix it. The n body one is half the speed of rust and it is hand tweaked using simd instructions. I wouldn't mind at all goin back to Ada. Most of the people that switched to c++ did so in order to have a larger hiring pool or to seem more modern. Some of them are switching to safety critical Java now for the same reasons. I suspect safety critical python is on the horizon. If airliners have to start worrying about python 2/3 headaches I'll know it is time to retire.

Yeah, I wish we had Ada experts fixing and optimizing them. That would be neat! It matters a lot to people, and Rust versions have been optimized to oblivion by many people. It should be done with Ada, too, to have a fair comparison. :)

And yeah, it is a valid and very serious issue. If we had more Ada programmers, we would not have to sit in trains or airplanes "powered" by C++, haha.

Even being rusty at Ada, you may be able to fix this regex program:

https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

https://salsa.debian.org/benchmarksgame-team/benchmarksgame/...

The other Ada pidigits program currently fails because output checking was made more restrictive:

https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

It would be great if you would contribute a program that fixes this trivial problem:

https://salsa.debian.org/benchmarksgame-team/benchmarksgame/...

I tested the code. It compiles and runs just fine. I compared the checksum of the output from both Rust and this version, they are identical:

md5 5b185f9a67a426baf78aa3bbb5baf8df out_rust

md5 5b185f9a67a426baf78aa3bbb5baf8df out_ada

On top of that, I got this:

Rust:

real 0m0.702s

user 0m0.693s

sys 0m0.007s

Ada:

real 0m0.708s

user 0m0.706s

sys 0m0.000s

Why don't you use this version instead? The output is correct and identical to Rust's, and the performance is significantly better: just as fast as C and Rust, as expected.

Thanks for letting us know about its existence!

> I compared the checksum

At what N?

> The output is correct

Nope. diff

    3c3
    < 6264338
    \ No newline at end of file
    ---
    > 6264338    :27

    PROGRAM OUTPUT:
    3141592653 :10
    5897932384 :20
    6264338

Here is the code that should solve your output issue: https://slexy.org/view/s21DoMQaHF

Thanks, done.

No worries. I got rid of the unnecessary `range' in the last version, but it does not matter much anyway.

My guess is that the Ada regex-redux #6 program is also easily fixable:

https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

:and there's a working Ada regex-redux #5 program:

https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

I tested it with N = 10000. I see the issue now though.

Edit: I do not understand your reaction to me thanking you for bringing my (and possibly other people's) attention to the other implementation. It was genuine.

> You're reading-between-the-lines something that isn't there.

Okay, my mistake then. :)

> Thanks for letting us know about its existence!

On the benchmarks game website anything underlined is a URL.

On the faster/ada.html page there's a link "all other Ada 2012 GNAT programs & measurements".

On the performance/pidigits.html there's a link "Ada 2012 GNAT #2".

Okay, thank you for the steps. I have to admit I completely missed it at first.

I will give you a few examples. My knowledge of Ada is insufficient, so I'll let you tell me whether Ada can solve that without using an external prover (note: being able to use an external prover is great and I haven't seen this done in Rust yet :) – but that's not the topic at hand).

1/ Consider a file `f` (or a socket, etc.). Using the standard library, Rust will statically ensure that, once the file is closed, you cannot attempt to, say, read from it. This is nothing special to files, just an aspect of the borrow checker.

2/ Consider a communication protocol. You need to send a message `HLO`, expect a message `ACK`, then send something else, etc. It is pretty easy to design your objects such that the operations of sending the message, receiving the message, etc. will change the type of your protocol object, ensuring statically that you never send/expect a message that you're not supposed to send in the current state.

If you're curious, I wrote a blurb last year on the topic: https://yoric.github.io/post/rust-typestate/

3/ I quickly googled "Ada spark phantom types" and didn't find anything. Does Ada support phantom types?

SPARK is a subset of Ada 2012.

Your examples are possible with contracts.

I'm interested, do you have examples somewhere on how to implement that kind of properties with contracts?

You can look at this blog post where I used a ghost global variable to hold the current state of the game (see section "Proving Functional Properties of Tetris Code"): https://blog.adacore.com/tetris-in-spark-on-arm-cortex-m4

You can similarly express ghost properties of your types, even though we don't have ghost fields in SPARK. For more on ghost code in SPARK, you can look at this presentation last year from my colleague Claire Dross: https://www.adacore.com/uploads/products/SSAS-Presentations/...

As a more extensive example of a useful library with this kind of contracts for proof, Joffrey Huguet added rich contracts of this kind to the Ada.Text_IO standard library just two weeks ago, as part of his current internship with us. This should be in the FSF trunk in the coming weeks. For example, here are some contracts he added:

   procedure Open
     (File : in out File_Type;
      Mode : File_Mode;
      Name : String;
      Form : String := "")
   with
     Pre    => not Is_Open (File),
     Post   =>
      Is_Open (File)
      and then Ada.Text_IO.Mode (File) = Mode
      and then (if Mode /= In_File
                  then (Line_Length (File) = 0
                        and then Page_Length (File) = 0)),
     Global => (In_Out => File_System);

   procedure Put (File : File_Type; Item : Character) with
     Pre    => Is_Open (File) and then Mode (File) /= In_File,
     Post   =>
       Line_Length (File)'Old = Line_Length (File)
       and Page_Length (File)'Old = Page_Length (File),
     Global => (In_Out => File_System);

   procedure Close  (File : in out File_Type) with
     Pre    => Is_Open (File),
     Post   => not Is_Open (File),
     Global => (In_Out => File_System);

>could you please tell me what kind of official formal proof tools exist for Rust that makes it predictable?

I feel like you aren't really asking a question, that your intent was really to lecture someone, but in case you really are:

* a borrow checker * option types

So OCaml would catch a similar set of obvious errors, though Rust would also catch all data races at compile time, as the borrow checker does that as well as prevent memory mistakes.

No, I was really curious. My bad if it looked like I was trying to lecture. I do not think that I have the necessary knowledge in this topic to do that. :)

> a borrow checker

What exactly do you mean? How does it differ from any other language's type system?

> would also catch all data races at compile time

In Ada/SPARK, you can formally verify tasks, too. Please take a look at https://docs.adacore.com/spark2014-docs/html/ug/en/source/co... if you have some time!

> prevent memory mistakes.

Which mistakes are you referring to specifically? I need to know so I can have a meaningful response to it, but all I can say right now is that Ada/SPARK does the same.

> What exactly do you mean? How does it differ from any other language's type system?

If you're familiar with affine types, this is basically what Rust's borrow checker implements.

If you're not familiar, it's a bit complicated to summarize on Hacker News. You should try it out, because it lets you guarantee statically entire classes of properties that non-academic languages struggle with.

> In Ada/SPARK, you can formally verify tasks, too. Please take a look at https://docs.adacore.com/spark2014-docs/html/ug/en/source/co.... if you have some time!

I'm not an expert in Ada, but yes, that's pretty similar to some of the properties Rust will let you check out of the box.

edit My memory of Ada was incorrect.

It's more than just affine types, it's region typing, which is far less common.

Fair enough.

> > a borrow checker

> What exactly do you mean? How does it differ from any other language's type system?

Part of rust's type system is sub-structural: by default, Rust's types are affine, which means you can only use them once (at most, not exactly).

Now this is not super convenient to use and could have efficiency issues (e.g. any time you want to check a value in a structure you'd have to return the structure so the caller can get it back), so to complement this you can borrow (create a reference). The borrow checker is the bit which checks that borrows satisfy a bunch of safety rules, mostly that a borrow can't outlive its source, and that you can have a single mutable borrow or any number of immutable borrows concurrently.

The borrow checker provides for memory-safe pointers to or into a structure you're not the owner of, with no runtime cost.

Can you recommend a good book on SPARK in general? I've read one on Ada 2012 itself, and what it had to say about Ravenscar and SPARK got me interested.

"Building High Integrity Applications with SPARK"

There is also "High Integrity Software: The SPARK Approach to Safety and Security", I never read it, but it is written by a well known author in the Ada community.

Yes, and also: https://learn.adacore.com/courses/intro-to-spark/index.html

Sorry if I angered you. I never claimed that Rust was the only language that has these properties.

My experience was is: I have never seen a language/environment where these rules were so well thought out, so well communicated and so well enforced. The package managment works like a charm, the module system (since edition 2018) is the best I have seen etc.

These are all subjective – I am a film/philosophy student who codes and I never received any formal education on any CS related topic. I was speaking about why Rust feels sound to me. Good chances Haskell will look sound to me too, very good chance I might check out more CS topics. No hard feelings please.

It is fine, and no it didn't anger me, so no need to apologize, really. :)

> I never claimed that Rust was the only language that has these properties.

Yeah, that is my mistake. Sorry for misunderstanding you!

> I never received any formal education on any CS related topic

Same here actually! :P

> No hard feelings please.

Not at all!

I understand that you like coding in Rust, and that is completely okay with me, and even if it was not, that should not deter you from continuing what you are doing.

This is not only static vs dynamic typing. Rust provides a set of abstractions seldom found elsewhere. The borrow checker, for instance.

It is more a question of semantics.

> If you e.g. have a `print foo` in some obscure branch that rarely happens, that print will ruin your day if you use Python 3

You can use any compiled language for that though, and even some uncompiled ones (PHP will refuse to run the file)

I recommend using mypy for python, type safe python has came a long way. The type annotation syntax has been in the language spec for years

Every software allows you to do stupid things. Its your job to avoid them.

Use a linter. Most IDEs will do it for you natively, and even if you're dealing with a terminal based editor like vim, it's really easy to get the output of pylint, flake8 etc. running in it. It's extremely rare I ever get a syntax error make it through to code review, let alone build/deploy.

Linters catch the easy cases but when the problem spans multiple files they have no clue what is going on.

The mypy type linter performs type checking across files, in many cases even without explicit annotations.

It's nice to see a balanced, real world, case study including 'these things are fixed by Rust', 'these are problems that don't occur in idiomatic Rust', and 'these are problems that Rust can't help you with'.

I'm a big fan of Rust, but the one sided 'Rust makes all the problems go away' articles don't provide any value.

It also highlights an example of a security bug introduced during rewriting; highlighting that rewriting any significantly large piece of software is bound to introduce bugs.

Not just introducing new bugs but reintroducing old bugs which were publicly documented and/or previously exploited. Which you could argue are worse as it’s a lower barrier for detection by attackers, but also on the otherhand by the team/community.

Also of note was that there was already an automated test for one of the high priority bugs that got reintroduced but the that particular tests was turned off.

What confuses me about this is the tests were turned off because they were taking too long. But wouldn't the appropriate behavior there be "run a subset of the tests normally, but run the full test suite occasionally" rather than just disabling the tests completely?

Or turn off some during development but run the whole suite before release?

I have a feeling the a bunch of the tests in that particular category needed to be updated, so it wasn't simply just too long.

Pretty clear that the QA process is at fault here, which is sadly common in the software industry.

Yeah I keep hearing / reading about rust, even seen some demos but the demos all end with "oh no I'm not using this for anything". Still cool but ... want to see someone doing something in production / get their thoughts on that.

Edit: To be clear I'm not saying anyone isn't using it, this is just more of a comment about the impression I can get when I hear about X tech is so cool, but that's most of what I hear and at some point I want to see those same articles about real world use / experiences. I'm aware Mozilla and others are using it.

Firefox?

I'm not saying nobody is using it, just the volume of "hey this is so cool" gets to be a bit much and I want to see more "here it is solving problems in production for a real thing".

The explorations and small little demos and projects in a language is almost always higher than the number of production uses, it's just that in popular languages nobody cares to read much about it (or at least share is so it spreads), because it's already known. Rust is still fairly uncommon, so you're bound to see a lot of people announcing their experiences while playing around with it, and that can skew the perceived ratio of things.

I think the problem is solved by paying less attention to the stuff you care about less, and/or seeking out more production(y) uses if desired by looking in places where it's already common, such as /r/rust, where the content ratio will naturally fall more favorably towards production uses, libraries and advanced discussions since a minimum level of experience (or at least interest) can be assumed.

We're doing a low level core functionality lib in Rust that is shared between Windows desktop, iOS and Android apps.

It works really well.

If we were to do it again, the core Rust lib would do much more and the platform native code would only be things that need to be like UI and notifications.

I'm using it for an not-yet released product. Roughly 30k lines of code running on Windows, macOS, and Linux. Mostly high volume event processing and parsing.

Been a lot of fun to work with. Having spent 10 years doing C++ professionally before, I can't see myself ever looking back.

Once product is public later this year, I may see if I can talk more publicly about it.

That sounds like it would be very interesting.

>Yeah I keep hearing / reading about rust, even seen some demos but the demos all end with "oh no I'm not using this for anything". Still cool but ... want to see someone doing something in production / get their thoughts on that.

Huh? Besides Mozilla itself using it in the browser in several backends, there are tons of places where its used in production (and several articles on HN on the topic).

Where do you see all those demos who say "I'm not using this for anything"?

I've seen articles on Mozilla using Rust, Dropbox using Rust, MS using Rust (rustgrep used in VSC but also something in Azure IIRC), Google using Rust (Google Fuchsia), Facebook using Rust (Mononoke, etc), and so on.

NPM is using Rust on the backend

I believe Discord is using Rust for their games store.

Amazon is using Rust - Firecracker powers Lambda and Fargate: https://aws.amazon.com/blogs/aws/firecracker-lightweight-vir...

MS security devision is also proposing a trio of C#, Rust and constrained C++ (meaning no Cisms) for their software going forward.

Dropbox is using Rust to decompress Brotli: https://blogs.dropbox.com/tech/2016/06/lossless-compression-...

Thank you.

Our product is basically written in Rust (~40k LoC) [1]. When I say basically: there are a few backend components that are written in Java (i.e, ActiveMQ) and the frontend is TypeScript/React, but it is mostly Rust.

[1]: https://www.schoolbench.com.au/

These are all ways that Rust is neutral or better. Were there any ways that Rust was worse? For example, did the previous code use value-type templates? If so how was their absence worked around in the new code?

Chucklefish made some buzz few month ago saying they were writting there new game in Rust, they now stopped and went back to C++.

https://www.reddit.com/r/programming/comments/atyzz4/halley_...

They're still using Rust for some stuff, just not for the game itself. Wargroove's matchmaking server is still in Rust. That said, yes, this is a good example of what your parent is asking for.

https://www.reddit.com/r/programming/comments/atyzz4/halley_... has some more context too

Even more context: https://www.reddit.com/r/rust/comments/avwxq1/comment/ehijdw...

Compile time. For my 9k lines project it takes 20 sec to compile in debug mode and 1 minute in release. Wich is insane because I have to do a lot of runs to test things(where cargo check can't help).

In spite of this i'm in love with Rust

If you modify a single line and rebuild, how much time does it takes, on average ?

I tried to change some constants in random places(in 1 line each time) and on average it's ~10 seconds Also compilation unit in rust is a crate. I guess if your app can be separated in different crates it can ease the pain.

cargo check is 3.11s

Overall, it did a decent job of being balanced, but I don’t buy the memory overflow example at all.

For one thing, idiomatic C++ bounds checks by default. You need to use at(). If you don’t like typing at(), you can implement an array type that always bounds checks fairly easily. On that note, the vulnerable c++ code should be using accessors, not indexing to access the oddly packed and laid out array. Even the fixed version wouldn’t pass a code review from me. You could write equivalently bad code in any language that supports array types, and get similarly broken results.

For another thing, there’s no evidence that you couldn’t achieve the same improved data structures in C++ using its type system (which is turing complete...)

The “thread safe by default” property sounds interesting; I’d be interested in reading more about that.

> For one thing, idiomatic C++ bounds checks by default. You need to use at().

Sounds like it doesn't check by default then. It checks if you remember to check using the more verbose bounds-checking method. Not unlike the issues with subscripting std::map.

This is like saying the Rust doesn't have memory safety because you can use "unsafe". C++ offers you a safe Api and an unsafe API if you want to go that way. If you decide to use the unsafe api the consequences are on you, the same way that it is on you if you decide to use "unsafe" in Rust.

The difference is in the defaults. In Rust, the easiest and more common thing ([]) is checked, and the more verbose and uncommon thing (get_unchecked) is unchecked. In C++, it's the reverse.

> This is like saying the Rust doesn't have memory safety because you can use "unsafe".

It's the exact opposite which is the point. Rust requires extra work and using the non-default and less convenient way for unsafety, C++ requires extra work and using the non-default method and less convenient way for safety.

I have some experience in C++, and I am familiar enough with the standard library to remember that operator[] doesn't check bounds while the at member function does. I would assume that the Firefox C++ programmers know this as well. However, maybe I'm wrong or have too much faith?

Or, maybe, the programmer was aware that operator[] didn't perform bounds checking, but opted to use it for some reason? A good way to dissuade people from making unidiomatic choices is to make them more verbose. IMO calling the at function isn't particularly verbose, but if the member function that didn't check bounds were called something like "at_unchecked," perhaps people would be less inclined to use it.

Also, from the snippet in the blog post, note that you can't tell whether the Firefox code used std::vector, C-style arrays, or some non-STL container type. Projects may use their own container types, but your criticism only applies if the programmers were using the C++ standard library.

> I have some experience in C++, and I am familiar enough with the standard library to remember that operator[] doesn't check bounds while the at member function does.

Everybody knows you're supposed to check pointers for being null, and yet time and time again developers fail.

As long as you rely on human nature and provide one API which is simple, convenient, obvious and dangerous and one which is complex, inconvenient, non-obvious and safe, you will just drive users towards the former.

> I would assume that the Firefox C++ programmers know this as well. However, maybe I'm wrong or have too much faith?

Just because they know when quizzed doesn't mean they'll always remember when actually doing. Even less so when subscripting is safe in pretty much every other language which provides array subscripting, and ::at… only exists in C++?

> IMO calling the at function isn't particularly verbose

No, but it's still more verbose and less intuitive than [], especially given the above (that tons of languages use [], and very few have an at method)

> A good way to dissuade people from making unidiomatic choices is to make them more verbose.

Indeed.

this is so true

and has been my experience with unwrap

they dont want people to use it but the alternative is so verbose and clunky

The alternative tends to be to propagate the error upwards using the `?` operator, up to some point where it makes sense to handle errors

Aka exceptions.

Yes, chaps, that Result<T,E> type is all but isomorphic with checked exceptions, Java-style.

The "but" is where all the difference lies though, Result (or Either or whatever you want to call it) is the reification of the sum of a return value and an error, and as such manipulable without having to add dedicated tooling… (which java didn't have either, and still does not).

Amongst other issues it's possible to pipe one through a generic wrapper without that wrapper having to care about it.

e.g. let's say you have an input collection, you map() over it, and the map callback can fail.

In Rust or Haskell you… just do that. And the caller deals with a collection of results however it wants.

In Swift, you need map to be specifically annotated in `rethrow` so it can be transparent to failure (aka can't fail if its callback can't, but can if its callback can).

In Java, you're shit out of luck and jolly well fucked, your generic map can't be generic over generic exceptions, so either it callback can't fail or you need to wrap said callback to convert the checked exception into an unchecked one, and possibly back again outside the map.

So… yeah, they're "all but isomorphic" because they're both implementations of the concept of statically checked fallibility. It's just that java's checked exceptions[0] are a bad implementation of the concept.

Put an other way, a 2018 fiesta or yaris are "all but isomorphic with" a 1960 corvair or a pinto, but you couldn't pay me to take a road trip in a corvair or a pinto.

[0] java's because someone might come up with better ones, though the well's been pretty tainted at this point

Lack of parameterisation of exception signature is not the reason why checked exceptions are a bad idea, though. Failure is a function of implementation details; runtime errors are fundamentally an abstraction violation. There's no escape.

> Lack of parameterisation of exception signature is not the reason why checked exceptions are a bad idea

It absolutely is one of the reasons why they are a bad idea, and why reified results are so much more useable and useful.

> Failure is a function of implementation details

In the original intention, that's what unchecked exceptions were for, with checked exceptions for the reporting of errors rather than failure. That's why java didn't have only checked exceptions.

Unchecked exceptions were for programmer errors, errors that could be avoided with a sufficiently careful programmer: null pointer check, a divisor not zero check, a list bounds check, etc.

Checked exceptions were for unavoidable errors. Errors that are fundamental to the operation being attempted. They usually occur because the operation interacts with the world outside the program, which means the program is subject to violations of expectations. Network errors. File not found when you try to open it - you cannot test for file existence first without a race.

The reality is that the latter kind of errors are better off as overloaded call signatures: one call variant when you care about the error and want to catch it, another variant when you don't care about specifics of the error and want the whole stack to unwind when the expectation is violated.

Neither of these approaches require checked exception signatures throughout the stack (or Result types for that matter - you can assume I also mean those, due to the isomorphism).

There's a reason .net modeled number parsing with Parse() and TryParse() instead of throwing NumberFormatException like parseInt(). It's because sometimes you care - input came from user and you need to handle it - and sometimes you don't - input came from configuration file and the stack needs to be torn down if you can't parse it.

Picture a stack that looks like this:

    0: <operation that may throw exception of the checked variety>
    1: <code that may be interested in handling error>
    2: <code that doesn't know about implementation details>
       .... could be 10, 100, 200+ methods in this stack dump
    N-1: <code that doesn't know about implementation details>
    N: <request handler or event loop that catches all exceptions>

The great problem with checked exceptions is the methods for stack frames 2 to N-1. Either exceptions are handled at stack entry 1, or at stack entry N. The only job of all the code between is to pass exceptions unmolested back to N.

Those calls may be dynamically bound (whether via vtables or function pointers, objects or closures) and / or dynamically linked (so unavailable to a type system at compile time). In large production programs, control flow will be dynamically determined. It's a fact of life; if it's not for testing purposes, it'll be for deployment flexibility.

I think there's value in the IO monad; in marking functions as the kind of functions that may interact with the outside world. And checked exceptions can work this way. But not unless the error type has a polymorphic storage location, and unwinding the stack is syntactically weightless. I don't ever want to have to change the signature on a dozens of methods just because there's a new implementation detail deep inside a dynamically linked abstraction.

And exception / error wrapping isn't the answer either - it's almost always a bad idea.

Semantically, returning a sum type like Result<T,E> absolutely is the same thing as exceptions (ie, it's the exception monad).

However, there's one very important language design difference between this and Java-styled checked exceptions: using sum type gives you effect polymorphism for free. This means you can write (say) a map function which says it has precisely the same exception type of its argument function, using the same machinery you're using for generics everywhere else.

This is a big from the usability side, since AFAICT it was the lack of effect polymorphism that turned people off of Java-style checked exceptions. From a language implementation standpoint, it's also nice not to have to implement type inference twice, once for return values and once for exceptions. :)

No, lack of static parameterization of the representation of failure is not the problem with checked exceptions. In fact the prison of being forced to construct a tunnel in the (static) type system to carry your error information, irrespective of how dynamic it becomes, is the torture that kills the idea. Errors are fundamentally the leaks in abstractions. They betray the implementation.

The network error that makes a mockery of pretending something is local. The dynamic link error that discloses a pluggable implementation. And all the steps between giving lie to the idea that errors should be caught and handled anywhere near their occurrence - when the only code that knows about the actual final composition is at the top of the stack, far far away, a perilous journey for a fragment of error info - unless such transport is automated, and not gated at customs, as so many checked exceptions boundaries end up being.

I'm in favour of error values for functions that return errors in normal circumstances. But they should be exceedingly easy to dispatch into an unwind mechanism, if it turns out you're not interested. Almost all of the time, the code in the middle doesn't care. Why should it pay the tariff for the transport?

> The network error that makes a mockery of pretending something is local. […] I'm in favour of error values for functions that return errors in normal circumstances.

Checked exceptions were intended for the latter, and unchecked exceptions for the latter.

Most debug builds across major C++ compilers do support it though.

The relevant code in fact uses a non-STL array container. Unfortunately, the performance of the STL containers is fairly unreliable across C++ standard library implementations and can be very poor in some cases. That makes them harder than it should be to use in code where you need to understand the performance characteristics of your data structures.

>Sounds like it doesn't check by default then.

Yeah, in the phrase "idiomatic C++ bounds checks by default", the term "idiomatic" is a "weasel word" that disqualifies "by default".

It is implementation dependent.

operator[]() does not require bounds checking by ISO C++, however most compilers do actually enable bounds checking in debug builds.

Visual C++ certainly does it for example.

Huh. According to https://en.cppreference.com/w/cpp/container/vector/operator_..., “no bounds checking is performed.” I find cppreference.com generally trustworthy, but maybe it’s wrong here? Or, maybe “no bounds checking” actually means “no guaranteed bounds checking?”

It means “no guaranteed bounds checking”, the standard only requires at() to throw if out of bounds (§ 26.2.4.1, note 15), but leaves unspecified how operator[]() should behave in invalid accesses, only that it isn't allowed to throw.

http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2017/n471...

Here is the Visual C++ documentation for bounds checking in debug builds.

https://docs.microsoft.com/en-us/cpp/standard-library/checke...

You need to look at the text of the Standard itself when it comes to this level of language lawyering. The short answer is that operator[] has undefined behavior if the index is out of range. Performing a check and terminating the program with some kind of runtime error is a legal subset of "undefined behavior", and so it's commonly done in debug builds, but you cannot rely on it in any sense.

> You could write equivalently bad code in any language that supports array types, and get similarly broken results.

You wouldn't get "similarly broken results." The results for doing this in C/C++ are far more serious, which was a point the article made.

If you do this in Rust/C#/Java/etc they will safely crash. If you do this in C/C++ it is undefined behavior, it may crash, but it also could allow remote code execution.

The Rust version of this bug is Security-Low (crash), the C++ version is Security-Critical (potential RCE).

If you want to prevent this class a bugs, you don’t need to switch languages is the point. Run a linter that prevents that syntax.

Right, but all you have to do in C++ is switch the array implementation to bounds check by default. This is not rocket science, and is certainly easier than rewriting large code bases from scratch.

Similarly, I could complain that rust arrays are too slow, and produce an array implementation that uses unsafe under the covers.

Dropbox did this in their Rust code: https://github.com/dropbox/rust-brotli-decompressor/blob/mas...

https://github.com/dropbox/rust-brotli-decompressor/blob/mas...

Toggle the feature, remove the bounds checks unconditionally.

> You need to use at()

That's not "by default". std::array_view still hasn't landed so I can't even wrap ptr+size s provided by third party libraries or across standardized C ABIs without going beyond the standard library (really not by default now)... and I believe I have yet to see a single solitary use of at() in a production C++ codebase. Some "default".

For code where overflows are expected, C++ exceptions are way too heavyweight and some other kind of "attempt dereferencing" pattern is used (if only checking the index manually before invoking operator[]).

For code where overflows are unexpected, uncatchable assert-style checks are used.

I've heard these "idiomatic C++" and "improved data structures" arguments a few times and as a C++ developer, I'm a bit skeptical. Does anybody know of any non-toy C++ projects that actually demonstrate such a high-level of reliable use of C++?

If so, how much do they rely on the developers being ideal programmers, who know intimately the intricacies of C++ and how to avoid ending up in UB-land?

I have a codebase that amounts to several kLOC of C++, available here: https://git.sr.ht/~maelkum/viuavm While I openly admit that parts of it are shoddily written, I also try to always use the "reliable subset" of C++.

From my experience it is not hard to make it a habit (e.g. using `::at()` and declaring variables as `auto const x = ...;` is muscle memory at this point) but the code becomes very verbose and looks like programmed defensively to a sometimes ridiculous extent.

So I understand your skepticism. To address the point of relying on developers being "ideal": just turn the compiler warning flags up to eleven, make all warnings errors, and run your tests two times - one time with sanitisers, and the other under Valgrind. This won't catch all errors, sure, but will still make you more confident in your code's reliability.

> The “thread safe by default” property sounds interesting; I’d be interested in reading more about that.

This property is simply a corollary to Rust's ownership invariants enforced by the borrow checker. There can only be a single live mutable reference to any object, therefore two threads can never hold mutable references to the same object simultaneously. Similarly, all objects must have a lifetime at least as long as the objects which reference them. Therefore no thread can ever hold a stale reference (mutable or immutable) to an object. If code passes Rust's borrow checker it must necessarily be thread-safe.

The borrow checker doesn't need any special knowledge of threading, though AFAIU there are traits that permit the compiler to check that you're using the correct boxing type when passing objects to threads. Objects which implement these and other traits are responsible for maintaining ownership invariants using unsafe code.

TL;DR: Threads cannot share mutable references because no code can share mutable references in Rust. It follows that any Rust code is thread-safe, ignoring bugs within or induced by unsafe code.

This is also the normative approach to writing "thread-safe" code in Unix programming more generally. Most libraries that grew from the Unix culture are only written to be re-entrant--they never hold references to objects shared outside their encapsulation or execution scope. Therefore most such libraries claim thread-safety provided that callers maintain the same re-entrancy invariants for library-defined objects. Without needing to use mutexes such code is thread-safe, you just don't normally get compiler verification. Contrast that with Windows programming or, especially, Java, where the expectation is that objects are shareable and guarantee thread-safety internally. Depending on which programming culture you grew up in, Rust's thread-safety is either obvious ("oh, it's just enforcing re-entrant-safe APIs") or magical ("how does it insert locks in all the right places?").

Caveat lector: I've never written any Rust code.

> Contrast that with Windows programming or, especially, Java, where the expectation is that objects are shareable and guarantee thread-safety internally.

That's not an expectation on either. For example, the single most common phrase you can see on MSDN in class docs is: "Public static members of this type are thread safe. Any instance members are not guaranteed to be thread safe."

There was a period when it was different for some things. In particular, both Java and .NET had thread-safe standard collection classes initially - e.g. Vector in Java, ArrayList in .NET. This has proven to be a bad trade-off in both cases - it's a massive perf hit for something that's not even all that useful even to threaded code, because in practice you often need to perform multiple operations on the collection atomically, and then you still need your own lock.

So they have since been obsoleted by new collections that do not attempt to do any thread synchronization; the old collections remain for backwards compatibility purposes. Modern idiomatic Java or C# code doesn't do any kind of synchronization or locking to protect the caller, unless that is specifically the purpose of this class or function to provide such things.

So I wouldn't say there's significant cultural differences between Windows, Unix and Java in that regard.

I'm pretty sure OP meant to write "idiomatic C++ does not bounds check". That's certainly true - the only place I've ever seen at() in use is in textbooks, never in production code. It doesn't really make much sense, because the moment you start using iterators, it's all unchecked anyway, so why bother with at() specifically?

I primarily use Java for my job. Security and memory related features of Rust are not an advantage compared to Java. But I like rust because it feels modern and produces efficient standalone binaries. Most of my hobby projects are in Rust now. But I would not rewrite any of my work projects in Rust even though they require ultimate performance. That would be a maintenance burden.

It is great to see people rewriting in Rust where it makes sense.

Nullpointer exceptions are a huge thing in Java.

I haven't written any Rust in a few years, but last I knew, it was very possible for a Rust program to crash. The guarantee is that it won't corrupt memory while doing so.

That's true, but 'null pointer exceptions' (which I suppose means expecting a Option<&T> to contain a &T when it is instead empty) are much rarer since nullability is made explicit.

Rust goals in life towards bugs are basically

- Isolate memory bugs to unsafe code which you rarely write.

- Make as many classes as bugs reasonably possible rarer by encoding as much as is reasonable in the type system and encouraging the programmer to think about all possible cases.

The first gets all the attention because it is the one you can make guarantees about, but the second is really just as important.

There are different reasons why a program can crash. Nullpointer exceptions are one of them. Out-of-memory situations are another. Rust protects you against the former, but not the latter. But at least in my experience I only encountered crashes when I disabled safety checks by calling .unwrap().

unwrap() does not disable safety checks, it just means you ignore/disregard error handling. It will panic and abort the program, but not cause a safety issue.

/pedantic

This is actually an interesting distinction. The way that Rust defines "safety" is a little jargon-ish. It's probably more useful than the (highly impractical) colloquial meaning, but still different. In a rust context, "safety" means that the program is protected from a very specific set of things.

One might expect that set to contain things like crashing (panic! and friends) and leaking memory (forget). It does not, and is not intended to.

But the parent's point remains: a Java NPE and a panic because of unwrap are really equivalent. Not really unsafe in the C sense, but equally bad as unhandled crashes.

Assuming that nobody catches the NPE by accident or error (according to my experience in Java, it's a pretty big assumption), yes, they pretty much are.

The big difference is that, in Java, you typically get your NPE because you didn't know/didn't check that your pointer could be `null` – in other words, the default behavior of the language is to NPE. In Rust, the default behavior of the language is to inform the developer that they need to check. They may decide to explicitly assert that the pointer is not null, causing a crash if it is, but that's a conscious choice.

For instance, in my Rust code, pretty much every occurrence of `unwrap()` or `expect()` contains a comment explaining which invariant guarantees that the call will succeed. I don't think I have ever seen any comments associated to a member access in Java.

There's one really major distinction - you can `grep` for `unwrap`. You could grep for null, but that won't tell you if an NPE elsewhere is possible, and it won't hit every single case (like 3rd party libs returning null).

Making bugs grep'able is huge.

Greppable also means it is trivially lintable. And the standard linter Clippy defaults since two years to disallow unwrap()

I think safety is about risk managment. Having a seatbelt on in a car is safer than not having one. This is despite the fact that most regular people really don’t desire crashing their car. By putting on a seatbelt you pay a small prize to weaken the impact of a existing risk (crashing your car).

Rust’s idea of safety is similar: it tries to reduce the impact of common risks that happen during programming. It won’t save you from logical errors — just like a seatbelt won’t save you when you decide to maneuver your vehicle into a flaming pit of lava. The only thing that would save you in that situation is not maneuvering into flaming lava pits.

That beeing said rust has a very straightforward implementation for Unit Tests and Integration tests which could help you dealing with remaining risks.

Of course you can always live on the edge, ignore seatbelts and do things like calling unwrap() on results that might not retun Ok(value) at all times. But that will bite you sooner than later.

What Rust is really good at IMO, is to make risks clear at any givrn moment in time. The routes your program can take are very well represented, especially because of the ownership model. Once you understood it you can make very strong assumptions about which part of the software is manipulating which data. These strong assumptions also help mitigating risk. It is harder in Rust to do things just wrong than in most other languages and it is still fast

Not exactly "disabled safety check": `unwrap()` is an assertion check (that may sometimes be optimized away by the compiler). You're still checking safety, but you're doing it in a much more terminal manner.

It is possible, but the causes and frequencies are quite different.

In Rust runtime panics are certainly possible (array out of bounds, out of memory, etc), but the ones analogous to Java's NullPointerException have to be explicit opted into (unwrapping optionals) and do not happen implicitly.

Rust lets you handle optionals in nicer ways which lets you be sure you covered the "null" cases at compile time with no runtime panics possible, if you like.

I think that is somewhat related to the article. There a class of bugs not possible in Rust, they are also not possible in my GCed langs. unwrapping an `Option` is similar to NPE. but that code does not feel idiomatic Rust. IMO there is higher chance of making NPE mistake in Java than unwrapping None in Rust for similar logic.

I write Java daily, I haven't seen one in a production environment in... at least recent memory. Most common NPEs I've seen can be resolved by doing two things: 1) for returning a single, nullable value, wrap it in Optional. 2) Never return null collections, always default to an empty collection instead.

More nuanced ones can often be caught by using all args constructors and requiring the constructor values with Objects.requiresNonNull() or similar. Using spring? Don't use field or setter injection, always constructor so the above applies as well. Making the state of your objects largely immutable means their state is more consistent and what is null and when becomes a lot less surprising.

Lastly, write good tests. If you're exhausting the behavior of your system with tests, these things are much less likely to surprise you later.

NPEs are definitely a problem with Java, but they're a very avoidable one as well.

Edit: I don't understand the downvotes. The parent said they're a huge deal and I'm disagreeing because I think they're relatively simple to avoid?

Rust basically just enforces the things you mention

With the cost of catching up to 25 years of tooling and libraries.

We’re getting there. I dislike this argument, I’m sure the same was made in favor of C when Java came out.

I agree with you, it is just a call to reality to those "lets rewrite everything in language X", without consideration that there is more to it than just grammar and semantics.

Also there are plenty of domains where Java or C++ aren't even taken into consideration, in spite of decades of effort.

Just to cite Microsoft's recent security recommendation out of Blue Hat conference, regarding their own software, "use a mix of C#, Rust and constrained C++".

I suppose that you are being downvoted because, though your arguments are correct, this thread is not about "how to avoid NPEs in Java" and is therefore OT.

That said, "write good tests" is impossible. You can write as many tests as you want, you can make sure those loops loop or don't, and those ifs if or don't, but you cannot write "good" tests. Tests will always miss some corner case, some input, some unexpected locale variable.

Can also use checkerframework to statically analyze your code for NPEs. It effectively augments the type system to require you to handle nulls where appropriate. It doesn't recognize I/O boundaries for you, you have to make sure those are annotated properly, but after that with sensible defaults, you can be more confident of the NPE-free-ness of your code.

> I think they're relatively simple to avoid?

Most issues in software fall into this category, though. The issue with Java’s null is that it’s not type safe. The language is generally a strongly typed language, except in the case of null. And until Value Types land, there is no way to express nullability to the compiler.

The arguments you make aren’t all that different than the undefined behavior arguments with C. “Just follow these simple rules and you’ll avoid all issues”, a compiler should do that for you IMO.

I can’t wait for Value Types in Java, at that point we can have real Option types.

Right.

No downvote, but GP post conflates "how code SHOULD be written" with "how existing code was written" in my opinion. Same with one of the earlier comments about "idiomatic C++ does bounds checking." I've seen plenty of NPEs in real, working Java.

If Mozilla isn't managing to write idiomatic C++, I don't hold out much hope for myself either.

> “Just follow these simple rules and you’ll avoid all issues”, a compiler should do that for you IMO.

This was one thing I really liked about Kotlin vs Java. It's nice that my IDE can do the work of generating getters/setters on a POJO for me, but it's not nice that the language requires me to fill my source code with that noise. To a first approximation, any task rote enough the computer could do it for me, it should not ask me to do.

See also: static languages that don't offer type inference.

this is just an honest question, and i’m genuinely curious, but why couldn’t Optional<T> be done as an immutable reference type... why does it need value types to be implemented first?

do you mean like a final field in an class?

i was thinking something like that, yes

It’s still not a “guarantee” that it’s not null, but you can be somewhat more confident that it isn’t if you assign it in all constructors of the type.

For params and return values you still can’t know for sure, and in all these cases it -could- be null. But yes, making all fields and variables that are Optional would help as a coding practice, except of course where you need to reassign the value in the Optional, as there is no “set” on the type.

NPEs literally The Billion Dollar Mistake, and you don't think they are a big deal?

https://www.infoq.com/presentations/Null-References-The-Bill...

In my experience null pointers and unassigned variables were a big problem in C an C++. But in java? They were never something that really caused a major issue. Most of them turn up in tests and are really easy to fix because you get an exception stack. Very few turn up in production and often it is because something else failed at runtime to cause it. If the null pointer wasn't there you would have had to deal with that other failure anyway.

Wouldn't using Koltin or Scala, which benefit from the engineering effort of JVM and arguably its ecosystem solve this problem ? (Since NULL is not idiomatic in either language)

It's not idiomatic, but it's still possible. I've got NullPointerExceptions while writing fairly complex, modern Scala, so it's about as possible as with Java in my experience.

Yes and no : I have been using kotlin for a while for Android. Currently our codebase is 75% kotlin.

Issues arose at the interface between java and kotlin. Unless there are @Nullable @NonNull annotations (and they need to be truthful), the kotlin compiler cannot know the nullability of something coming from a java method.

It can be pretty pernicious : if you use some java written libraries like Moshi (json parsing), it can also lead to crashes : IIRC if you declare a moshi generated property to be non null but it is absent from the json, it will generate an object with null, creating a crash.

Still, null is now an anecdotal issue in Kotlin. The Android framework team is working on annotating all their APIs with the corresponding nullability annotations and more and more JVM libs are also working on handling it gracefully.

It was never a huge issue to begin with even in java, just pretty cumbersome to have to add annotations everywhere and have to add some policies like 'no null collections, only empty' in a codebase to have sane handling.

For me, also primarily a Java person before Rust, it’s datarace safety that captured me.

Java’s executors improve the world, but I can’t count the number of concurrency bugs that I had in Java that are not possible in safe Rust. This isn’t just my code, btw, but large teams where it’s hard to disseminate good practices when building threadsafe code. If it had been in Rust, those issues wouldn’t have happened.

Other things that I appreciate about Rust over Java is the error handling combined with RAII, doing away with nasty bugs around try’s lacking proper finally statements for closing file handles, etc.

Java has its warts, not everything is just about memory safety.

about RAII there is quite a bit of support in recent Java versions not quite as good as Rust but there is support.

Data race is one other thing, I do have data race issues but that is very rare. Some of the static analysis tools even catch these anti patterns.

Curious about memory usage too

I think the best way to look at it is that memory usage becomes predictable and GC pause free. An application of similar scale and implementation between Java and Rust won't necessarily use less memory when in Rust.

Naively I thought Java data model was inherently more demanding than Rust but .. I never read about Rust memory layouts.

I use Java constantly in my job and recently tried rewriting a math & memory heavy component in rust to see what performance gains there might be. Surprisingly (to me) the naive rust version was ~15% slower than Java. There’s probably room for more rust optimization but it was interesting that “efficient standalone binaries” doesn’t automatically mean faster too when competing with HotSpot.

Google had a fast math library for Android, which got outperformed by ART JIT compiler and was eventually deprecated.

https://developer.android.com/reference/android/util/FloatMa...

I find where rust usually shines the most is if you do text processing. String allocations take time. In rust you can often avoid them and use things like cow to only allocate when you change something. That way often my text processing heavy scripts go twice the speed of a C++ version, and they're easier to write with Cargo, too.

Compared to highly optimized Java and C# I could often get a quite naive rust implementation to be 10x faster.

Naive rust means I didn't spend much time optimizing but I do use appropriate algorithms and to avoid unnecessary allocations.

My experience working on writing image processing code in Java is that rewriting the slowest bits in C++ only gave about a 10% performance boost, and this was over a decade ago. Moving stuff to the GPU was vastly more effective than doing faster CPU work.

When comparing speed on the JVM with speed with native languages, the only positive side you get from native binaries is cold start nowadays.

For a webapp this doesn't matter, but as we're moving towards more cloud functions it start to make a lot of sense.

That needs to be ponderated by the fact any real life application will have to access the network at bootstrap to load configuration and therefore your bottleneck will most likely be I/O.

> "ponderated"

Huh! I have a fairly strong vocabulary and thought this might have been a made-up word -- but apparently it means "to weigh down or give substance to", which aligns with your intended point here. TIL

Sorry that's directly brought from French.

English is my second language. I did a quick google search just to make sure it was correct but it's hard to gauge if a word that happen to be in the dictionary is also widely understood.

The main performance gain would be memory consumption.

I would except best performing Java code to be slower than best performing Rust code. But comparing general cases is not going to be simple.

I remember reading an article about Java performance many years ago. It had an example for Java being faster than C++ and it was maths on arrays / matrices. I guess that kind of code least trips up HotSpot optimizations, so it wins due to availability of runtime information and a really good implementation of certain parts.

Though I doubt that Java usually wins against the Eigen C++ library. Eigen uses some template tricks to fold operations together (which can avoid intermediate storage and unlocks more types of restructuring optimizations) and uses SIMD extensively.

Probably not, however Hotspot got some Intel help regarding auto-vectorization, including AVX support.

Rust is often sold feature by feature; the borrow check offers proof like safety over fuzzing, cargo provides real versioned package management over makefiles or git commits...

I choose Rust because taken as a whole, Rust changed the way I approached laying out my memory and how I composed my code. I think this more than anything leads to less issues than the equivalent C++. The article points out that a Rust vs C++ solution to any given problem are going to be completely different.

My only desire for Rust is to see compile times speed up and the C++ interop to improve.

Yes compilation times are a big pain point. I heard that there is work being done in this area.

One of the major hurdles in rewriting parts of C++ projects in Rust is that the interop surface between both languages is C. The necessary interface layer has created more bugs and work than the conversion saved. I'd really like to see more high-level interoperability between the two languages in the future, although C++ is a pretty fast-moving target at this point, with all the changes in C++20.

> The necessary interface layer has created more bugs and work than the conversion saved.

Is this from a project you did? This is weakness, for sure, but I'd be interested in hearing more about why it failed for you! We have some people working on this.

That's a proportional problem at least right? As months pass the more of your libraries converted to Rust the lesser the problem.

The potential party spoiler being third party code that's not practical to replace. In some cases even decades won't change it's nature, as a few examples have shown.

At some mid point you would have maximum C glue code in place, and then things get better after that.

Honestly, I wish a few different major languages would get together and start developing system ABIs that move beyond C as the interchange language.

But what would you put in such an ABI beyond what's in the C ABI? Beyond basic data types, struct defintions, and function definitions, languages begin to wildly diverge almost immediately.

Multiple return values. Ownership annotations. SIMD type support. String charset information.

More challenging would be to include stuff like allocation management (how to free a pointer when it's done), GC integration, function boxing, iterator/generator support. Vtables and cross-language inheritance is interesting but difficult.

One point of clarity: this would be an FFI ABI, not necessarily expecting that most datatypes would be laid out according to the ABI.

> Vtables and cross-language inheritance is interesting but difficult.

Windows' COM "solved" that problem. For example see C++/WinRT[0] for a modern C++ interface to COM. Or the C# .NET equivalents.

[0] https://github.com/Microsoft/cppwinrt

Take a look at this.

https://github.com/Microsoft/xlang

This was my experience when I was at Mozilla converting a part of Firefox to Rust. It also feels like a lot of Rust's safety guarantees go out the window when you are using the C FFI to talk between C++ and Rust. I'm not sure how bad it actually is in practice (vs just writing the whole thing in C++).

Does Rust allow for taint analysis too like Perl has for long time? If not I'd say it's missed opportunity.

(It marks all untrusted input as tainted and programmer must explicitly parse the data or mark them untainted to pass them further.)

There's not a language feature to do so, but you can do it through the type system if you wish.

In Rust, or any statically typed language such as C++ or Java, the idiomatic way to handle untrusted input is to treat it as a "bag of bytes" before you access it. Then either parse it into a strongly typed object or bail out of parsing. The strongly typed object is safe to use. Bailing out (throwing an exception or returning an error type) does not allow the program to continue assuming that the (malformed) input was correct.

More to the point, you should put untrusted input into a different type from trusted input. As much as I admire the design of the servlet API I think the biggest mistake is that everything is transmitted as Strings. The input characters should have had a different type than the output characters.

That's a feature that might work well with some plausible ways of guaranteeing memory safety in a type system because object reachability is a form of taintedness.

Recently some colleagues started using the type annotations in the latest python3. Really excited for this feature! It’s going to make a lot of our production systems safer to work with.

And of course Rust is a great technology, etc.

Can you build in some kind of "unsafe release" mode, so that every array bound check that were asked in the code are skipped ? If not, would it be an interesting feature ?

No. Such a thing could only remove some kinds of checks; for example, if you see that code sample later in the thread with a manual check, it wouldn't know that's what you're doing.

In general, we don't want to make it easy to turn checks off. They get removed if the compiler can prove they're not needed; if they're there, they're almost always for good reason.

This is trivial to implement, but it will never be accepted by Rust upstream. There will be a fork if someone really wants this.

You could do something with rusts feature system and macros. In essence you'd have a macro that would run a different line of code if your feature is enabled versus disabled, so you could use the unbounded lookup on the array.

That said, this would be a user implementation and wouldn't be likely to be provided by the standard Library

Meh. The whole thing seems weird to me.

We totally tried to write this twice then we switched to language x and everything is great. Feels like something a language zealot would say. I would scoff if someone at my company rewrote a core section in a different language. It is their language so maybe they just told them to do it that way ha.

> could have been caught by a run time bounds check

And here I thought rust was all about zero cost abstractions.

It's not, that's C++.

Rust requires a lot of runtime checks, but that's the price one has to pay for memory safety.

Rust is about zero cost abstraction. Bound checks can be disabled.

And then it's no longer memory safe, which makes the whole exercise pointless...

Even in that case it's not pointless. You can do a lot of testing and fuzzing with checks enabled to get some confidence that you don't have bugs. Then disable the checks for performance. That's just one option. I think it's nice to have options even if I choose not to use them.

Even better is to use iterators and other abstractions that don't require bounds checking at all. Rust has lots of good tools to build efficient code.

If your use iterators instead of indexes, you have no bound-checks because they are optimized away. I've been working full time with rust for a year and I'd say I need indexes less than 10 of the time, most of which are for fixed-size array with constant indexes, for whom the bound checks are also optimized away. So I'd say they aren't really a problem in practice 95% of the time. If you really need to go unsafe for the last 5%, well you're still 95% safer than C++ :)

Your choices are exactly the same as in C++ - memory-safe with runtime checks, or memory-unsafe without - except that the default is different.

But the real benefit of Rust is ownership tracking - and that one is zero overhead.

"Abstraction without overhead" is explicitly a goal of Rust.

https://blog.rust-lang.org/2015/05/11/traits.html

From your link:

"C++ implementations obey the zero-overhead principle: What you don't use, you don't pay for".

In Rust you pay for e.g. bounds checking or integer overflow handling or optionals. But I don't understand why everyone's getting so defensive about this, since it's the only way (static verification and proofs aside) to get the desired safety characteristics...

You only pay for bounds checks if you use them, same with the overflow stuff, or option types. That's not in contradiction to this principle.

In Rust release builds the integer overflow of checks are disabled. They're not zero cost and there's no magical way to do it, so it's only done in debug builds.

You can disable them in debug builds as well with wrapping integers. So this is again a practice of making the safer option the default, but won't incur runtime costs if it bothers you.

If there is a way to do it at compile time that will usually be used in rust.

What checks are you thinking about? There are some, but the vast majority of Rust's checks are at compile time, not runtime.

> Rust requires a lot of runtime checks, but that's the price one has to pay for memory safety.

It requires a lot of runtime checks on the boundary between rust and C/C++ code. You cannot trust C/C++ code, so you are forced to check everything. C-function returns a enum value? Check that this value is in the enum, before converting it to the rust enum. Check that there are no situations like

    enum MyEnum {
        val1, val2=5
    };
    enum MyEnum foo() {
        return 42;
    }

But after you've done all that checks you need to check almost nothing, because if you have a enum value in Rust, you know for sure that it is a valid value. And compiler knows that the value is valid and optimizes accordingly. If you have an &str, you know that it is a valid utf8, you need not to check it on each access, it have been checked already. You got a valid pointer from C? Wrap it into NonNull, so neither you, nor compiler would need to check validity of a pointer once more.

It is unclear thing, who needs more checks -- safe code or unsafe code. I'm gravitating to a belief that unsafe code needs much more runtime checks.

This comment is confusing two completely separate concepts: zero cost abstractions and run time bounds checks. Rust does provide zero cost abstractions: if you do something explicitly/manually instead of using the abstraction you get the same generated code.

If you want to combine the two concepts, I guess you could go for an example like this:

    if index >= vec.len() {
        panic!("out of bounds");
    }
    let value = unsafe { vec.get_unchecked(index) };

which can be rewritten using higher level abstractions as:

    let value = vec[index];

The second is more abstract, and that abstraction is zero cost, you do not pay for using the abstraction more than the explicit code above it.

What's an example of a language where the second would be more expensive than the first?

There are some languages where you cannot write the first yourself at all. Including leaving off the assert if you know better than the compiler.

For that specific example it's hard to be expensive but here's a loop from one of my crates that compiles down to efficient code:

    pub fn decode_12be(buf: &[u8], width: usize, height: usize) -> Vec<u16> {
      decode_threaded(width, height, &(|out: &mut [u16], row| {
        let inb = &buf[(row*width*12/8)..];

        for (o, i) in out.chunks_exact_mut(2).zip(inb.chunks_exact(3)) {
          let g1: u16 = i[0] as u16;
          let g2: u16 = i[1] as u16;
          let g3: u16 = i[2] as u16;

          o[0] = (g1 << 4) | (g2 >> 4);
          o[1] = ((g2 & 0x0f) << 8) | g3;
        }
      }))
    }

The "decode_threaded()" call is a function call that passes in a closure with the inner loop to a generic function that is used to multithread a bunch of similar decoders, instead of repeating that code. And the for loop is actually describing what I want (process every 3 bytes into 2 output values) instead of having me manage some iteration variables and have off-by-one bugs. "chunks_exact" and "chunks_exact_mut" are recent additions that allow me to say that I only want to receive exactly 3 bytes and output exactly 2 values, so if the array is improperly sized the extra at the end just gets skipped. This not only matches the intention of this code (I only ever process 2 pixels from 3 bytes and nothing else will work) but also gives the compiler a better way to lift the bounds checks out of the inner loop and make the code significantly faster (by 2x in some decoders).

Now let's see "decode_threaded":

    pub fn decode_threaded<F>(width: usize, height: usize, closure: &F) -> Vec<u16>
      where F : Fn(&mut [u16], usize)+Sync {

      let mut out: Vec<u16> = alloc_image!(width, height);
      out.par_chunks_mut(width).enumerate().for_each(|(row, line)| {
        closure(line, row);
      });
      out
    }

Besides allocating the image it uses "par_chunks_mut" to make the code threaded. That's a function that's provided by the rayon crate which manages a threadpool and it's scheduling for me. So I'm even using code written by other people, to build something that's apparently deeply nested even though it needs to be fast. And yet after all this indirection and syntax goodies the end result is efficient machine code that matches or is even better than the original C++ code and not the dynamic runtime like behavior of Ruby/Python that you'd expect from such code.

That's what zero cost abstractions are. After memory and data race safety I think rust shines because it gives me a lot of the ergonomics of Ruby with the speed of C.

Speculative execution and branch prediction make bound-checks much less of a burden. I believe this is actually a whole category of Spectre/Meltdown.

It is. As you say, there's a category of Spectre bugs where the CPU speculatively assumes a bounds check succeeds, then goes on to execute attacker-provided code.

And there's your wide execution width making it possible to do the checks in parallel with the access. You are using up entries in your branch prediction structure when you have something like that, as well as instruction bandwidth, but I'd doubt the function will run any slower once past decode.

You can't know everything at compile time. Runtime checks are witnesses that run at program execution instead of compile time.

Right, and this is equally true in Rust and C++ (and any other systems language). Zero-cost abstractions doesn't mean that the compiler is imbued with magical properties.

normally you can use iterators to access arrays and when you do, array bounds checks are elided.

if you can't use iterators, and you need to omit bounds checks for perf, you can do so with unsafe blocks.

The article is arguing that Rust somehow has better capabilities than C++ to fight memory-related bugs, but the example vulnerability given is not something Rust can solve nor is more powerful than C++ in its “bug catching” capabilities regarding this kind of bug.

Concretely, the article claims that in Rust the vulnerability doesn’t become a bigger problem because it simply crashes at run-time due to built-in bounds checking. True, but that is alao the case as well with C++ if you were using the equivalent Vec type with mandatory bounds checking - which many projects do (and, critically, enforce).

Personally, I like what Rust brought to the compiler/language world. However, some people is definitely overstating the case. Most non-trivial memory-safety errors and vulnerabilities are related to runtime problems like the example shown. In these, no language can help in the general case - we are not solving the Halting Problem. Therefore, saying Rust is immune to memory-related problems is not true. It is true, however, that those bugs will not trigger anything worse than a crash if there is no unsafe blocks. The same way that many other common languages out there do (Java, C# and many others).

The same way, I have seen people (and even the linked blog) to claim Rust is free of race conditions or thread-safety issues (even if it introduced great ideas to write correct code).

Giving a false sense of security is the worst thing we can do.

(Continued...)

It is not realistic either to ask everyone and every company to rewrite all their C/C++ code in Rust. Even if it were financially doable and a rewrite were to happen, in many cases it would simply be best to move to a language like C# anyway, not Rust; for productivity reasons. Where performance allows, of course.

In my opinion, the realistic and pragmatic solution is, instead, to strive to make all languages (in particular C and C++) embrace security-first approaches/types/mechanisms like Rust does. The compiler tecnology is already written - now retrofit as much as possible into C++ (even to the point of introducing a “safe” scope if needed) and allow companies to embrace it at minimal cost and progressively.

A system crash is a bug. Period. In many cases it could lead to Denial of Service. An insulin pump can stop working.

I remember when C# came out almost 20 years ago. People said "I can forget about managing memory so I can focus on the logic". Programs kept crashing, memory problems were still there.

The article goes with "...remove the burden of memory safety from our shoulders, allowing us to focus on logical correctness and soundness instead...". More or less the same, and admitting that said problems won't go away.

But here we are, it's 2019 and we're still using C/C++ as if nothing happened.

A crash is a bug but not a security problem.

Such affirmation requires that a crash will never produce a security problem. But for example...

"Families are LOCKED OUT of or INSIDE their homes as Yale 'smart' security app crashes leaving dozens stranded"

https://www.dailymail.co.uk/news/article-6268379/People-lock...

"Households up and down the UK were unable to lock or unlock their doors"

An unlocked door it's a security problem too...

That's still a bug. I feel like you're being intentionally obtuse about what's considered to be a security problem in code. Nobody ever suggested that rust code can never crash or have bugs. It's just about memory safety, which obviously has nothing to do with door locks.

Sorry if I gave that impression. It was not my intention.

> A crash is a bug but not a security problem.

I think that all bugs, the ones that produces crashes and security bugs should be all treated equally. A bug is a bug, whenever it has security implications or not.

To me, the article gives the impression that a system crash is not a security problem, because a Rust program will "terminate in a controlled fashion, preventing any illegal access". But one for example, can fingerprint a system by forcing it to crash.

And of course, nobody expects that Rust will prevent bugs from happening, but at the same time I don't get why the fixation of setting a difference between security bugs and bugs.

"security problems are just bugs" - Linus Torvalds. (http://lkml.iu.edu/hypermail/linux/kernel/1711.2/01701.html)

edit: Linus reference.

Security problems are bugs but not all bugs are security problems.

Reply on news.ycombinator.com