Fearless Concurrency in Firefox Quantum | Hacker News Reader

Fearless Concurrency in Firefox Quantum

blog.rust-lang.org

611 pointsahomescu1174 comments

I like this explanatory comment by Manishearth, a Servo dev, in the thread over on /r/rust:

"This blog post brought to you by the 'how many times can you say 'fearless concurrency' and keep a straight face' cabal.

"Seriously though, I now appreciate that term a lot more. One thing that cropped up in the review of this post was that I didn't have examples of bugs Rust prevented. Because I couldn't think of any concrete ones. Because Rust's safety doesn't work that way, it prevents your concurrency bugs before you realize you had them, by making sure you don't paint yourself into a corner. 'Fearless concurrency' really is the best way of putting this; the benefit was not that it prevented concrete bugs, but that it let us fearlessly and aggressively write code knowing that it would be concurrency bug free."

https://www.reddit.com/r/rust/comments/7cwpbq/fearless_concu...

I've built now several concurrent services with Rust. The language definitely gives confidence to try several things with different approaches to concurrency. None of my services crash (except once per 3-4 months when I deployed something "that will never crash" using `.unwrap()`). The crashes are always my own laziness, but if I follow the pattern of checking return values and unwraping only when the input is static and visible a few lines before, the resulting programs are fast, have a small footprint and basically never crash.

Oh and the tooling! I hope other projects take a serious example how cargo works. It's very hard to use any other build system.

Note that the panic you get by calling unwrap() where you shouldn't isn't a crash. It's a controlled program exit due to an unexpected condition. While yes, the panic will cause your program to stop, it will do it in a clean deterministic way (with a backtrace).

Actual crashes (due to segfaults) can happen a long way from the bug that actually caused the issue, can happen intermittently and generally be a nightmare to debug.

Thank you for correction. My native language is not English, but I meant a controlled exit.

It requires a bit of thinking how your library is structured when you need to avoid unwrap(). But in the end the result just works and I like how Rust forces a certain design to be safer.

I think the use of the term "crash" varies. While it's true for e.g. a C program that there are better or worse crashes, in most other modern langs with some more safety guarantees, the term "crash" is normally used for controlled program termination due to unhandled exceptions.

So coming from that type of language (where you can't segfault) I'd definitely call a panic a "crash" simply because it's the analog of an unhandled exception, which I always called a crash.

So this terminology probably varies between ecosystems

For yet another different, but related, use of the word, check out 'crash-only software': https://lwn.net/Articles/191059/ or https://en.wikipedia.org/wiki/Crash-only_software

It's a concept implemented in eg Erlang. Crash in their sense means basically, 'kill -9'.

It's interesting that the phrase "unhandled exception" has crept so pervasively into our terminology, since "exception" has connotations (e.g. first-class values representing errors, which we can construct, pass around and "throw") and "unhandled" implies that they could be "handled".

Haskell is a great example of how handling errors can be harmful, since it violates confluence, even though throwing errors is fine!

Confluence is the property that evaluation order doesn't change the meaning of a program, e.g. we can do:

    (1 + 2) * (3 + 4)
    3       * (3 + 4)
    3       * 7
    21

Or:

    (1 + 2) * (3 + 4)
    (1 + 2) * 7
    3       * 7
    21

We could inline some function calls if we like, thanks to referential transparency; we can even evaluate "under a lambda" (i.e. evaluate the body of a function before calling it, or evaluating the branches of an `if/then/else` before picking one); regardless of which way we evaluate, if we reach an answer (i.e. don't get stuck in a loop) then it will be the same answer:

    (1 + 2) * (3 + 4)
    (1 + 2) * (if 3 == 0 then 4 else pred 3 + inc 4)
    (1 + 2) * (if 3 == 0 then 4 else pred 3 + 5)
    (1 + 2) * (if 3 == 0 then 4 else 2      + 5)
    (1 + 2) * (if 3 == 0 then 4 else 7)
    (1 + 2) * (if False  then 4 else 7)
    if (1 + 2) == 0 then 0 else (if False then 4 else 7) + (pred (1 + 2) * (if False then 4 else 7))
    if (1 + 2) == 0 then 0 else 7                        + (pred (1 + 2) * (if False then 4 else 7))
    if 3       == 0 then 0 else 7                        + (pred (1 + 2) * (if False then 4 else 7))
    if 3       == 0 then 0 else 7                        + (pred (1 + 2) * 7)
    if False        then 0 else 7                        + (pred (1 + 2) * 7)
    if False        then 0 else 7                        + (pred 3       * 7)
    7                                                    + (pred 3       * 7)
    7                                                    + (2            * 7)
    7                                                    + 14
    21

Exception handlers break this, since we can write expressions like:

    try (head [42, Exception1, Exception2])
    catch Exception1 -> 1
          Exception2 -> 2

We can evaluate this one way:

    try (head [42, throw Exception1, throw Exception2])
    catch Exception1 -> 1
          Exception2 -> 2

    try 42
    catch Exception1 -> 1
          Exception2 -> 2

    42

Or another way:

    try (head [42, throw Exception1, throw Exception2])
    catch Exception1 -> 1
          Exception2 -> 2

    try throw Exception1
    catch Exception1 -> 1
          Exception2 -> 2

    1

Or another way:

    try (head [42, throw Exception1, throw Exception2])
    catch Exception1 -> 1
          Exception2 -> 2

    try throw Exception2
    catch Exception1 -> 1
          Exception2 -> 2

    2

This gives 3 different answers. Note that the throwing itself doesn't cause this problem, because we treat an "unhandled exception" as not getting an answer (equivalent to an infinite loop).

When I first grokked this it was quite enlightening: adding features to a language can make it less useful. It's not that certain features (like throwing or catching exceptions) are "good" or "bad", but that we must think of languages as a whole, and knowing that some things aren't possible (like observing evaluation order) can be just as useful as allowing more things. This contrasts strongly with the tendency of languages to accumulate features over time, especially when the major justification is often "we should have it because they do" :)

There's also a nice discussion on errors vs exceptions at https://wiki.haskell.org/Error_vs._Exception

The issue here seems to have little to do with exception handlers; see that

    head [0, throw E]

also produces such an "indeterminate" answer depending on order of evaluation, as long as you define things as you have here.

The solution in a lazy language like Haskell is to be specific about when things get forced, which outside of explicit overrides only happens in response to actual usage of that expression, eg. when the value is printed. At this point you're naturally forced to introduce either sequentialization or explicit parallelism; in the former there is no issue and in the latter you still need to explicitly sequence the results, of which the exceptions are a relevant part.

In a strict language like Rust, of course, you never aimed to have this property anyway.

> The issue here seems to have little to do with exception handlers; see that

> head [0, throw E]

> also produces such an "indeterminate" answer depending on order of evaluation, as long as you define things as you have here.

I disagree; note that I said:

> we treat an "unhandled exception" as not getting an answer (equivalent to an infinite loop)

More formally, throwing an exception/error results in _|_ (bottom) and all _|_ results are equivalent i.e. we can't tell "which error" we got. In particular, we can't tell the difference between _|_ due to an error being thrown, and _|_ due to an infinite loop.

This is important in a general recursive language, since we can't know (in general) whether evaluating some value (with whatever strategy) will eventually produce a result or not. Consider the following, where omega = (\x -> x x)(\x -> x x):

    head [0, omega]

Under a call-by-name strategy this will reduce to 0, under call-by-value it will loop forever, i.e. giving _|_. Should this count as breaking confluence?

Total languages like Agda and Coq would say this breaks confluence, due to general recursion being a side-effect, and hence it should be encoded as such rather than allowed willy-nilly.

Turing-complete languages like Haskell would say that such encoding is cumbersome, and hence that their notion of confluence should be weaker; specifically, evaluating an expression using any evaluation strategy to get a value other than _|_ will produce the same value.

It just so happens that lazy evaluation has the property that if some (perhaps unknown) strategy can reduce an expression to normal form, then non-strict evaluation can also reduce it to normal form. In other words, lazy evaluation avoids all 'avoidable' infinite loops, but it still gets stuck in 'unavoidable' ones. That's nice, but isn't important for confluence.

You're perfectly right that introducing sequencing like `seq` throws a spanner in the works :)

> throwing an exception/error results in _|_ (bottom) and all _|_ results are equivalent

My point is that they're not when you have `catch`, and that this distinction adds nothing that ⊥ doesn't already add with regards to order of evaluation.

I'll have to think about this some more, since I don't quite understand.

Let's say we have the following Haskell definitions:

    error msg = undefined  -- "throwing an error"
    undefined = undefined  -- infinite loop

In this setup, there's no way for us to tell, under any evaluation strategy, whether or not an expression evaluates to _|_. Any code we might write to "check" for this would have to run "after" an infinite loop, which is impossible. Hence it's impossible to write an expression which, under any evaluation strategy, normalises to two distinct non-_|_ values: either it always produces the same value, or it sometimes produces one value and sometimes _|_ (depending on the strategy), or it always produces _|_ regardless of strategy.

If we can distinguish between "different _|_s", e.g. catching some as exception values, then we can write an expression which reduces to different non-_|_ values depending on the evaluation strategy, and hence we lose confluence (the weaker form; we already lost the stronger Coq/Agda form by having _|_ in the first place).

This is fundamentally different to the value-or-_|_ uncertainty, since that's unobservable from within the language.

> This is fundamentally different to the value-or-_|_ uncertainty, since that's unobservable from within the language.

This seems to be a much weaker claim than I thought you to be making; of course being able to catch errors moves visibility of errors from externally visible to internally visible. That's the goal, after all.

I disagree, however, that this matters in the regards you raise, because a language which can arbitrarily decide to return ⊥ on the basis that it isn't internally visible is a broken language and any reasonable implementation needs to avoid that.

You're right that having some mechanism to act gracefully in the case of errors is almost always required. "Internally" this might be a "main loop" with an exception handler. "Externally" this might be a bash script which runs our binary in a loop, or a systemd service, etc.

The intriguing thing about Haskell's approach is that it shows us that such mechanisms aren't pareto-improvements: we have to give up something, like confluence.

Keep in mind that confluence isn't just academic, it's the thing which makes functional programming attractive for parallelism. Confluence solves the hard problem of taking all possible interleavings of concurrent execution into account, since they act like different evaluation orders, and hence can't mess up the result.

Servers are a scenario where these two features clash: we want concurrency and parallelism for scaling, but we need restart loops to prevent downtime.

The use of external restart loops reminds me of delimited continuations, where even "undelimited continuations" are still delimited by the OS (e.g. Scheme's current continuation doesn't include the state of other OS processes). Likewise, "unobservable errors" can still be observed by the OS (e.g. when our process dies, as in a bash or systemd loop).

I think a good compromise is to "stratify" error handling: we write our business logic (or whatever) in a provably confluent sub-set of our language, and execute that logic using non-confluent features like error handlers. Confluent expressions can take advantage of optimisations (e.g. speculative evaluation) which are invalid for the wrapper.

One thing I'm not sure about is nesting exception-handling code in pure code. Approaches like algebraic effect systems let us mark expressions as requiring effects like 'stdio', yet we can handle those effects in a pure way (e.g. using hard-coded strings during a test). I don't think this is enough to maintain confluence in the face of concurrency though; we'd probably have to pass in a deterministic scheduler, but that may parallelism gains of things like speculative evaluation, work-stealing, etc.

> You're right that having some mechanism to act gracefully in the case of errors is almost always required. "Internally" this might be a "main loop" with an exception handler.

This isn't (just) about exceptions that escape; you also need to guarantee that fst (0, ⊥) returns 0 rather than ⊥. Heck, you're practically required to do the same for

    fst (0, [0..(10 ^ 10)] !! (10 ^ 10))

and for that evaluation order isn't even visible at the denotational level.

> Keep in mind that confluence isn't just academic, it's the thing which makes functional programming attractive for parallelism.

Automatic parallelisation of functional languages is academic.

> you also need to guarantee that fst (0, ⊥) returns 0 rather than ⊥

Why guarantee 0 instead of ⊥? I'd say it's due to a general agreement that ⊥ has the lowest desirability: if we have the option of returning ⊥ or something else, we should pick that something else. Non-strict evaluation strategies are the most extreme choice, but most languages consider the desirability of strictness to be stronger than than the undesirability of ⊥.

I don't think exceptions are so simple though. Imagine a situation like this:

    getUserById :: [UserDetails] -> UserId -> User
    getUserById db id = MkUser id (getName details) (getDOB details)
      where details = head (filter ((== id) . getID) db)

On one hand, we may want this to fail fast: if `head` throws an `EmptyList` exception, we want to propagate that to the whole expression. Since getUserById might throw, we can wrap it in exception handlers and deal with missing users appropriately.

On the other hand, we may want to ignore exceptions in sub-expressions that we don't care about, e.g. having `fst (0, Exception)` reduce to 0. This seems trickier for `getUserById`, since we might do a bunch of processing which only needs the ID, and end up triggering the `EmptyList` exception far away, deep in the heart of a pure-looking function. I can think of three solutions to this:

- Wrap exception handlers around the subsequent steps. This smells funny, since those steps might be completely pure.

- Jump back to the original exception handler. Such non-local jumps may be very hard to understand, plus the handler would need to work in arbitrary contexts; all it can really do is return a different value of the same type (e.g. some predetermined default), or throw a different exception (which just defers the problem) or produce ⊥.

- Mark potentially-exceptional values somehow, so we can track their propagation through the program, and handle them if needed. That doesn't seem any different than `Maybe` or `Either`, perhaps modulo some lifting.

Of course, the situation becomes even more complicated if an expression contains many different exceptions!

> Automatic parallelisation of functional languages is academic.

Note I said "attractive", not "automatically solves all problems" ;) Even with "manual" parallelism, like `par`, map/reduce, etc. it's nice that these don't alter the semantics.

It also simplifies compiler optimisations, and helps programmers reason about when they will/won't fire.

> Why guarantee 0 instead of ⊥? I'd say it's due to a general agreement that ⊥ has the lowest desirability: if we have the option of returning ⊥ or something else, we should pick that something else.

Then `⊥ || True` would return True, but it doesn't. The reason Haskell specifies how it evaluates values is simply because any language which doesn't is broken. A language with no evaluation order is strictly less usable than one with (any) specified order of evaluation.

> Imagine a situation like this

I am really confused about what you're confused about. If head throws an exception E, and assuming getName and getDOB are nontrivial, getUserById reduces to `MkUser id E E` (this is not a language level statement). There is no jumping or impurity or any such thing happening.

> Note I said "attractive"

To academia, yes.

> Then `⊥ || True` would return True, but it doesn't

This claim is a little strong out of context. If you're just talking about Haskell, with the Prelude definition of `||` and no rewriting shenanigans, then you're right. That doesn't mean ⊥ is desirable though; it's just unavoidable in this case, due to constraints imposed by other desirable properties of the language.

Haskell's designers found the semantics of lambda calculus desirable enough to use as a base for Haskell, even though it removes their ability to define such a "parallel or" function.

This is similar to the desirability of strictness: most languages find it compelling, even though it removes the ability to avoid some ⊥ results like in `fst (0, ⊥)`.

> The reason Haskell specifies how it evaluates values is simply because any language which doesn't is broken.

Haskell only constrains evaluation order to be "non-strict". Implementations are free to use any non-strict evaluation order they like, although I agree that any "serious" language implementation should document what users should expect to happen. Note they should also specify what not to expect, e.g. it might say that the evaluation order of arguments is undefined, even if it just-so-happens to work in some predictable way at the moment!

In any case, in your `⊥ || True` example it's not the evaluation order that triggers the ⊥, but the data dependency in the definition of `||`:

    x || y = if x
                then True
                else y

If the language semantics allows something like Conal Elliot's `unamb` operator ( http://conal.net/blog/posts/functional-concurrency-with-unam... ) we could define `||` in a non-strict way but, as I said, Haskell's designers preferred to pick lambda calculus semantics over parallel-or semantics.

> If head throws an exception E, and assuming getName and getDOB are nontrivial, getUserById reduces to `MkUser id E E` (this is not a language level statement).

That's the first reduction step. The question is whether or not we should reduce it any further, to get `E`. Strict languages would do this, non-strict ones wouldn't.

If we do perform this (strict) reduction, we'd trigger some ⊥ and exception results unneccessarily, e.g. `getId (MkUser id E E)` would give `E` rather than `id` (and likewise for ⊥ instead of E).

If when we don't do this strict reduction that things get tricky, since we'll end up passing potentially-exceptional values around our program. This is just like Haskell passing around potentially-⊥ values.

The tricky part is handling these exceptions. If we define a handler at the point they're triggered, we'll have to put handlers all over our pure functions. For example the following is a pure function, but if we call it with the `MkUser id E E` value we got from `getUserById` we end up needing a handler for the EmptyList exception:

    isMillenial user = dob > 1989-12-31 && dob < 2010-01-01
      dob = try (getDOB user) (handle-exception-here)

Alternatively, we could define a handler at the point they're thrown, e.g.

    safeGetUserById db id default = try (getUserById db id) (exception-handler-goes-here)

Yet `getUserById` doesn't throw an exception (in our non-strict setting), so this handler won't be invoked; we'll just have `MkUser id E E` like before, with the exceptions potentially neing triggered elsewhere.

Alternatively, we could "attach" the handler to the result, so if the exceptions get triggered that handler will be invoked. That's the "jumping" I was talking about.

The other difficulty is where do we return to after handling an exception? If our handler's triggered during `isMillenial`, then it had better return a DOB; if it's triggered during `greetUser` then it had better return a UserName, etc.

We then have to consider what to do if a value contains multiple exceptional values, all with different handlers...

> > Note I said "attractive"

> To academia, yes.

Not sure what you're getting at here? "Attractive" doesn't mean "a solved problem which everyone would be mad to avoid", just a nice source of inspiration. Heck, Google's MapReduce is clearly inspired by functional programming ideas like confluence, and that's been out of academia for so long that it's become deprecated!

> This claim is a little strong out of context.

The context is whether exception handling restricts a language beyond how it is naturally restricted, not the particulars of any one language, so the claim is exactly as strong as it needs to be.

Haskell made its choice, yes, and there are other options like that offered by `unlamb`, sure, but the point is that both of those are a choice. Nobody leaves it up to chance. (You might mention C's lack of specified evaluation order in certain cases, but you should note that this doesn't break the guarantees you're looking for, since C never offered them, it just relaxes a different one.)

No doubt there are counterexamples with total languages, but we should keep that special case separate.

> Haskell only constrains evaluation order to be "non-strict".

This is an internal detail; the two claims (Haskell is lazy/non-strict) differ only in that the latter allows more implementations, not that there is any difference at the language level. Since one tends to assume the as-if rule, even that minutia goes away most of the time.

> That's the first reduction step.

No, in a lazy language it's most likely the last reduction step, if it ever happens at all. (In strict languages it would be first, but at the same time strict languages don't tend to perform reduction on programs, so there's never a `MkUser id E E` anyway.)

Importantly, this means it's senseless to talk about exceptions "being triggered". You either reduce to one or you don't. The most it makes sense to do is `deepseq` it, but as you'll note that is explicitly enforcing order of evaluation so that, not exception handling, is the thing that you should be complaining about!

> Not sure what you're getting at here?

I'm not trying to be subtle here: the advantages of pure, lazy evaluation with regards to automatic parallelisation of code are of interest only to academics.

> Google's MapReduce is clearly inspired by functional programming ideas like confluence

I don't agree remotely. They claim to take inspiration from the map and reduce functions from Lisp, which is strict and whose map and reduce functions are more correlated than intrinsically related to functional programming as a whole; even C++ has a particularly imperative one in its standard library.

> > This claim is a little strong out of context.

> The context is...

I was merely pointing out that you made the statement "X doesn't reduce to Y", but didn't specify which language you're talking about. I wrote the subsequent stuff under the assumption that you're talking specifically about (GHC) Haskell.

> Nobody leaves it up to chance.

Of course. I'm not saying that languages or implementations shouldn't pick (and document) an evaluation strategy. I'm saying that confluence is a useful feature, that exception handlers are a useful feature, that simplicity/predictability is a useful feature, but surprisingly (to me) we can only seem to pick two.

> No, in a lazy language...

Yes, in a lazy language. I'm talking more generally, about whether different (arbitrary) strategies would lead to different behaviour.

> Importantly, this means it's senseless to talk about exceptions "being triggered". You either reduce to one or you don't.

Whether you reduce to an exception or not depends on the evaluation order. Again, I'm not talking specifically about Haskell, but about confluence.

By "triggered", I meant attempting to evaluate an `E`. In the case of `fst (0, E)` a call-by-name strategy would produce 0 without "triggering" (attempting to evaluate) the exception; call-by-need would do the same; call-by-value would trigger it; etc.

Note that I'm assuming E behaves like _|_: if we "trigger" an E in any sub-expression, then the whole expression becomes E, and this propagates upwards until either the whole program is E, or we reach a `try x y` expression. In tht case an exceptional x would cause this expression to reduce to y.

That's why we break confluence: `try (fst (0, E)) 1` would reduce to 0 under call-by-name and 1 under call-by-value.

> the advantages of pure, lazy evaluation with regards to automatic parallelisation of code are of interest only to academics.

You're the one who's talking about lazy evalution and automatic parallelisation :)

I'm talking about confluence, which is useful whether a language is strict, lazy, serial, parallel, concurrent, etc. I'm saying it's especially useful for concurrency (and concurrency is a prerequisite for parallelism).

"Useful"(/"attractive") doesn't mean "automatic parallelisation": it's a whole bag of stuff, including easier to understand code, having the same semantics for both serial and concurrent uses (e.g. no need for the "thread safe"/"not thread safe" notes which litter Java's class reference), etc. Yes, automatic parallelisation is in that bag, but if you think it's only of academic interest then why bother mentioning it at all?

> > Google's MapReduce is clearly inspired by functional programming ideas like confluence

> I don't agree remotely. They claim to take inspiration from the map and reduce functions from Lisp, which is strict and whose map and reduce functions are more correlated than intrinsically related to functional programming as a whole; even C++ has a particularly imperative one in its standard library.

Lisp is a direct descendent of lambda calculus (e.g. McCarthy's "Recursive functions of symbolic expressions and their computation by machine, Part I" uses lambda notation, famously calling the s-expression alternative 'rather formidable for humans to read' ;) ). That makes it pretty functional in my view.

So what if Lisp is strict? It also has dynamic scope. That has nothing to do with whether or not it's functional. A more prudent argument against Lisp being functional would be the inclusion of impure, side-effecting operations, which makes programming in e.g. Common Lisp not much different than other impure, non-functional, imperative languages (except for the macros).

Still, map and reduce are not imperative in this way, and in fact they're often used to 'remove imperativeness' in many languages (e.g. defining map, filter and reduce with loops, then avoiding loops elsewhere). In fact, reading through "Recursive functions of symbolic expressions and their computation by machine, Part I" I see that a `maplist` function is used as an example :)

PS: It looks like you're refuting an argument like 'Haskell is the best language, it invented map/reduce and can automatically parallelise all code, if only people stopped catching exceptions'. That's not at all what I'm saying :)

> Of course. I'm not saying that languages or implementations shouldn't pick (and document) an evaluation strategy. I'm saying that confluence is a useful feature, that exception handlers are a useful feature, that simplicity/predictability is a useful feature, but surprisingly (to me) we can only seem to pick two.

My point is basically that you end up making this trade-off far earlier than that, as evidenced by Haskell, so it's not really right to blame exception handlers; `try (fst (0, E)) 1` isn't any more important than `fst (0, E)` and programmers won't let language writers leave the latter unspecified.

> Note that I'm assuming E behaves like _|_: if we "trigger" an E in any sub-expression, then the whole expression becomes E, and this propagates upwards until either the whole program is E, or we reach a `try x y` expression. In tht case an exceptional x would cause this expression to reduce to y.

I feel this gets to the crux of the issue: if you assume this, then a lot of what you said does hold, but you don't have to assume it and many languages (eg. Haskell) don't. Why are you assuming this anyway?

> You're the one who's talking about lazy evalution and automatic parallelisation :)

You brought it up, not me!

> having the same semantics for both serial and concurrent uses (e.g. no need for the "thread safe"/"not thread safe" notes which litter Java's class reference)

Thread safety of this kind just requires a lack of shared mutability (see Rust); purity and more extreme forms like being agnostic to order of evaluation don't really add anything in practice.

> [On Lisp]

I'm not saying Lisp isn't functional (it is), but arguing against your claim that "MapReduce is clearly inspired by functional programming ideas like confluence".

IMO it is good to call this a crash in the sense of fail-fast. Then you do not get into the "but my program doesn't run anymore after that, what do you mean that's not a crash" discussion. But it is not undefined behavior. It crashes _all the time_ in that situation, which makes life so much easier than UB.

Perhaps it's an "emergency landing".

When this controlled program exit happens, does your monitoring system wake your operations staff up? If so, it's a crash.

Yes, but you also immediately know where it happened, what happened, and how to fix it.

That’s a massive improvement over alternatives

Whether it's a crash or not is independent of whether you even haven a monitoring system, an operations staff, or even if you run the program as a service or not.

Heck, your monitoring system could still notice the program exiting and call your operations stuff in Rust's case too.

And a user with sudo rights killing -9 a program you run might or might not send anything to your monitoring system -- but that wont be a crash either.

I think the important point the parent is trying to make is between a crash and a controlled exit, that is whether you get a stacktrace, things can be called to cleanup, etc.

Merely calling all the cases just "crash" would lose that distinction. It's like calling all vehicles "vehicles". Sure, it's accurate, but I want to know if it was a car, a motorcycle, a truck or whatever.

In Rust a panic brings down a single thread (unless you have it set to panic=abort, which is the case in firefox but not elsewhere).

Panics can also be "caught" like exceptions, though this is not something you're supposed to use to implement exception handling. The idea is that if you want to make your application robust you can catch these panics near the top and try to recover.

Often panics bringing down a single thread will bring down other threads that try to read messages from it and have declared that they will panic if that is not possible. (.recv().unwrap() is a common idiom).

So it depends on how you use it.

And "monitoring system" is assuming the context of a server side application.

You could say the same about NullPointerExceptions, and while they're better than segfaults - in practice they're not that much better.

I think you're hitting an interesting point here. In my experience, there are two kinds of NPEs:

1) On a method call somewhere in a stack with no null checks: here it's not very useful to have an NPE, since it should work (eg. there's no null checks, so why should it fail?).

2) `Objects.requireNonNull(o)` (or an explicit through). This indicates that a pre-condition is that the object should not be null. These tell that the fault is in the preceding call stack, therefore you don't even need to understand what the underlying one is to know where the error is coming from.

Although all types are nullable in Java, I don't expect _everything_ to be null-checked. Nullable types in Rust stand out, therefore should be matched or checked. If I have a panic due to a naked `unwrap()`, either there's a documented pre-condition and the system isn't recoverable (hence the crash), or the error handling is missing and I know what to do.

Hence `unwrap()` is similar to (2), which are useful panics.

It's really all about the practice. If you make liberal use of `panic!` (or things that may panic, like `unwrap()`, then you basically get into situation 1.

You're definitely right that Rust is better than Java in that it makes things that may crash (like the naked unwrap()) obvious and "yucky", rather than "usual business". But that doesn't mean that a panic is not a crash - it still is. The language itself is better - but that doesn't mean you can't write code that crashes :)

In systems contexts "crash" means sigsegv or sigill (but not e.g. an intentional abort)

In non-systems contexts it can just mean "premature termination". Rust, having both kinds of programmers, uses .... both :)

So a Rust panic is a crash, but so is a Rust segfault, depending on who you talk to.

Generally I try to explicitly say "panic" and "segfault".

What would you say are the advantages of cargo over any other package manager / build system?

Quoting what I just replied in that thread:

> Given that this Servo code replaces an existing code base, couldn't we get a "guestimate" by looking at how many unsolved bug reports are now closed because their associated previous (presumably C++) code has been replaced? How many open bugs existed in Stylo's precursor that are removed now?

But was the existing code base that was replaced running concurrently?

Ah, true. But I thought Rust was supposed to help with more than just concurrency related bugs?

I'd love to know what this concurrency thing is and why it's so fearless in rust.

Rust ensures that while one thread is accessing a datastructure that no other thread will modify it. It does this by only allowing you to have either one mutable or multiple readonly references to a datastructure. Code marked unsafe could violate this, which is why it should be rare and well reviewed.

Here is a nice game about concurrency. It should answer your question why it should be feared by understanding how it works: https://deadlockempire.github.io/

As of my understanding it is fearless in Rust, because the problems that might came up are solved on language level or the compiler warns you about them. (I have never coded Rust, I just deduced this from the comments)

"Warn" is nicer phrasing than what the compiler actually does. Rust's compiler will straight up refuse to compile your code if it thinks it has a concurrency bug in it.

> if it thinks it has a concurrency bug in it.

This makes it sound like it is wrong sometimes. I'm not using Rust, if this is what you meant, could you give examples of that?

The borrow checker can't reason about certain scenarios that are (obvious to the human) safe. When that happens it'll error on the side of caution and prevent the safe code from being compiled.

The discussion here: https://news.ycombinator.com/item?id=14915539 and the linked previous discussion and ELI5 have a good collection of examples with explanations.

The compiler enforces rules that when followed mean that your code should not have any concurrency bugs. Those rules enforce a subset of behavior that is actually safe. There are other ways of structuring the data and processing that are also safe that Rust doesn't allow because it's not smart enough to reason about (and to be clear many people aren't smart enough to easily reason about either). In those cases, you can wrap the code in in question an unsafe block, which tells Rust to relax certain rules for the scope of that block, and gives you control somewhat analogous to what you get in C. The benefit is that when used sparingly and concisely (if at all), this allows certain bugs to be reduced to originating in very limited parts of the code base.

As an example, if you have a data structure or algorithm that you know is safe when implemented correctly, you can implement it in an unsafe black and expose it through a safe API/function. To my understanding, a lot of Rust's standard library core containers do exactly this. It's not guaranteed to be bug free, but it does provide quite a bit more assurance.

It would probably be more accurate to say "if it can't prove there aren't any data races".

https://blog.rust-lang.org/2015/04/10/Fearless-Concurrency.h... Is from just before 1.0, and so is slightly dated in some ways (scoped threads are in a package, not in std) but should give you some explanation.

Have you followed any of the links?

Such comparisons are at best guesses. An existing code base has a history of feature and bug accretion that any new, replacement system benefits from with respect to design and implementation.

For those not reading the other thread, I'll quote Manishearth's reply:

> I don't think there would be many of these. Specific threading bugs once found get fixed, and we don't know how many go undetected.

I've spent the last few days aggressively parallelizing some Rust code with crossbeam, and it's really just... painless (once you're used to Rust). Rust actually understands data races, and it grumbles at me until my code is provably safe, and then everything Just Works.

The Rayon library is also lovely for data parallelism.

Sometimes, I think, "Rust is basically a nicer C++ with the obvious foot guns removed and great tooling", but then there are those moments where I'm just blown away by what a good job it does.

(I think it helps that I have some functional programming experience under my belt, and that I tend to use mutability sparingly, and mostly in very simple ways.)

> painless (once you're used to Rust)

This needs to be Rust's motto or something.

Rust: Front-loading the pain so the rest is relatively easy. Just like High School. ;)

> This needs to be Rust's motto or something.

Rust: climb that mountain so you can see further.

Rust: it's relatively easy when you're an expert!

Rust: Expected String, got str.

painless (after some amount of pain)

I needed to search for the crossbeam library you mentioned, and their version of `Atomic` is just what I need in one of my libraries. Thank you for that. I need to dig deeper what other useful tools it contains.

> basically a nicer C++ with the obvious foot guns removed

For those stuck with C++, you're not out of luck in terms of safe and easy sharing between threads. The SaferCPlusPlus library provides "access requesters" that provide functionality and (data race) safety similar to Arc<T>.

Unfortunately the documentation[1] is "in transition" at the moment and does not currently refer to the very latest version. But the main difference is that the latest version implements something akin to rust's "sync" trait to help prevent you from attempting to share inappropriate data types.

Again, documentation is a work in progress, but if you're interested you can check out an example that demonstrates an array being safely modified simultaneously from multiple threads[2].

[1] shameless plug: https://github.com/duneroadrunner/SaferCPlusPlus#asynchronou...

[2] https://github.com/duneroadrunner/SaferCPlusPlus/blob/7e3574...

Rayon is pretty cool. It's about as powerful as OpenMP, and a bit easier to use.

The fearless concurrency really is ass-saving. For example, my most recent non-bug: I launched two parallel tasks where one would free a shared resource when done. In C that would be intermittent use-after-free. In Rust it was a compile-time error.

Congratulations to the Mozilla and Rust teams!

Accounts like this one really help people who advocate investing in and building new tools to help against the heavy-handed application of phrases like "a bad workman always blames his tools" [0][1].

[0] https://en.wiktionary.org/wiki/a_bad_workman_always_blames_h...

[1] https://en.oxforddictionaries.com/definition/a_bad_workman_a...

Edit: formatting and typo

I hate that phrase and its reflexive usage so much. Yes, a bad workman blames his tools, but so does a good workman using lousy tools.

You can respond with "that's because a good workman doesn't use poor tools."

It nicely suggests that maybe it's the programmer's responsibility to advocate for something better.

When I was a kid I used to pride myself on how many things I could take apart and reassemble with just a butter knife. But that was only because I didn't have access to real screwdrivers.

Likewise with software, each improved tool I learn (or new but worse tool I learn to avoid) makes me appreciate how much time I could have been wasting by using inferior tools/practices/languages.

Firefox Quantum is blazing fast, and I finally ditched Chrome.

My only problem is that it drains my battery life fast. So whenever I'm not plugged I use Edge. Other than that, FF is amazing.

It is now my default browser and I even wrote a FF add-on a few days ago using their new API!

Can someone confirm this? Like do a simple energy consumption comparison pre- and post-quantum?

On what OS? It will vary for sure.

As a front end Dev I'll never fully ditch Chrome as its Dev tools are vastly superior. I've tried the Firefox inspect menu and I'm just immediately turned off and confused..

However Firefox has been, and always will be my daily driver for all Web browsing. It's eco system is richer, noscript and the fact that it's not a Google product is a huge selling point.

From a feature standpoint, I can't think of anything that Firefox devtools are lacking. This seems like a gripe about the UI, and my feeling is precisely the opposite of yours, but equally subjective.

Just testing out FF devtools, I do appreciate how similar they are to Chrome. The biggest differences seem to be geared towards PWAs, such as auditing w/ Lighthouse.

I don't feel this way. What are some neat features of Chrome Dev Tools that I might not be taking advantage of? One I know of is the ability to inspect WebSocket connections, I really wish Firefox had that.

Probably network throttling and responsiveness is the only reason I would switch to chrome for testing.

Today is the first day I spent my entire day in Firefox and didn’t get angry. I love it! Ditched chrome for default browser now.

Network throttling is present, but only in responsive design mode. And responsive design mode has the same features with Chrome, but its a bit wonky at the moment.

I think you should open a bug report about this at bugzilla.mozilla.org I did the same for several feature requests or ideas and the firefox developers and community is very nice and responsive (I'd say most of my bug reports were fixed within days/weeks whereas feature request usuallly takes longer - but is always consider).

They are very open to feedback!

Done!

Mind sharing the link to the bug? I'd like to follow it as I think it would be a good enhancement.

This is the bug for this in case you are interested or want to follow progress: https://bugzilla.mozilla.org/show_bug.cgi?id=885508

Their devtools have come a long way. And are almost caught up with Chrome. I've only noticed the following things missing: When an XHR response comes back as HTML, there is no "Preview" tab like in Chrome. Also, the responsive testing tool doesn't have the device frames (ie, iPhone, iPad, etc) and it also doesn't have the little touch circle cursor that let's you drag and swipe while testing.

Perhaps you're on an older version of FF, no? Because the things you mentioned are present, just not an exact UI/UX as that with Chrome.

Im on FF Quantum 57.0 (non dev, vanilla FF) and here are my observations:

> When an XHR response comes back as HTML, there is no "Preview" tab like in Chrome.

This is consolidated in the Response tab. You can view it as raw response, or JSON (with filtering options).

> Also, the responsive testing tool doesn't have the device frames (ie, iPhone, iPad, etc)

It does now! You can select and edit a list of device.

> and it also doesn't have the little touch circle cursor that let's you drag and swipe while testing.

They do have a touch simulation mode (pointing hand icon), but it doesnt appear to work at the moment. I cant drag the screen.

>> When an XHR response comes back as HTML, there is no "Preview" tab like in Chrome.

> This is consolidated in the Response tab. You can view it as raw response, or JSON (with filtering options).

Yeah, I can see the raw HTML or JSON response. But Chrome actually allows you to see the rendered HTML response in the "Preview" tab. Are you saying there is a way to see the rendered HTML from the XHR response? I'm not seeing it. To see the rendered HTML, I have to copy it, create an html file, paste the HTML, then open it in a browser. That's a lot of extra steps when debugging something.

Oh sorry I misread your comment. Youre right there is no HTML preview indeed

Looks like this has been fixed now: https://bugzilla.mozilla.org/show_bug.cgi?id=1353319

If you grab Nightly (I've been using it for a year now for frontend development and my normal browsing and it's just fine) you can test it.

Switched to nightly and there it is. Thank you!

For XHR there's also an "edit and resend" button as well as "copy as cURL" which is amazing

copy as curl is so underrated

I kinda agree. A few key things missing in FF that I need:

* selecting CSS colors in other colorspace (eg. RGBA) as oppose to just hex

* throttling the network in desktop mode. I think the option just shows up in responsive design mode.

Although I dont think Chrome is "vastly" superior at this point as FF has certainly narrowed the gap.

> I've tried the Firefox inspect menu and I'm just immediately turned off and confused..

I remember feeling the same when I switched from Firebug to Chrome. It takes a few days to get used to a new UI.

> selecting CSS colors in other colorspace (eg. RGBA) as oppose to just hex

I'm not sure if this is what you mean, but you can go into the dev tools config and change the default color unit under Inspector. The eyedropper will show values using the selected unit and style rules will be converted to that unit.

Whoa. I love FF even more.

You are comparing FF Developer Edition to Chrome here?

Nope, stock to stock. Didn't know FF had that, I'll check it out.

It is just a preference thing. I have always preferred FF tools over Chrome dev tools.

Have you tried the new devtools in the recent release? I've heard folks like them a lot more.

From the article it seems Rust is really a great replacement for C++ and all it's complexity and quirks. I wonder how many other C++ projects are considering moving to it, anyone know about any major one like FF?

I'm a former C++ dev who went all in on Rust. I think the main problem is that the learning curve works in Rust's disadvantage here.

If you start learning C++ it's relatively smooth sailing at first, especially if you're already familiar with C. Basic OOP, basic RAII, inheritance, virtual functions, basic templates. Easy peasy.

It's once you start getting to the advanced topics that the footguns become apparent. The sometimes intricate resolution rules (and how they compound with template substitution rules), the various subtleties surrounding copy constructors, const, mutable, concurrency and the way they play with each others, the various quirks inherited from C that sometimes don't play very well with modern constructs etc...

Rust is the other way around. There's a very steep curve right at the start where you need to understand how the borrow checker works and how to make it happy. You have to learn the right mindset right away. You need to get over that to reach the "fearless confidence" goodness.

I think that's going to be a big problem for experienced C++ coders to do the jump (especially if you need to convince multiple devs to make the jump at the same time).

It kind of reminds me of the switch from SVN to git. At first I didn't get it, git felt a lot more complicated and I didn't really see the benefit compared to good old SVN. Of course after a few years I'd curse under my breath every time I had to use SVN for some legacy codebase, it feels so clunky and limited now that I'm familiar with a proper git workflow.

Already basic C contains more than enough footguns. If you think basic C++ is relatively free of footguns you're kidding yourself. Rust has a steep learning curve because the compiler nags you a lot about things that would have been a potential footgun in C. Unfortunately it is not smart enough to see in all cases that your code wouldn't have triggered that particular footgun and has to be overly conservative.

Not to belabour the metaphor or anything but if your footartillery doesn't fire, you still have feet.

This is probably the point that simias is making and I tend to agree. It's straightforward to get a simple C++ program working with decent performance if you know C. And RAII, std::vec etc makes you feel quite productive.

Now, at some point you'll notice that debug and release don't act the same way. Occasionally. Or you try to add threading to a single-threaded program. Then the love will die (due to bloodloss from blowing off the whole leg). But that's often after months of reasonably productive development.

Rust, to a large extent, will prevent you from ever getting to that state and that's a very good thing. But the upfront cost is quite high. My guess is that those of us who have experienced the C++ pain will appreciate the value that that cost is buying.

But I'm hopeful that Rust will draw a wider audience primarily because of your last point. It's really quite conservative at present. The NLS work is already expanding the scope of acceptable programs and, I presume, that what is learnt from that effort will seed further work in that space.

I meant that basic C++ seems rather straightforward and at first doesn't appear to introduce too many new footguns over the C subset. Actually thanks to vectors and RAII it can remove whole classes of classic C mistakes.

It's only when you start to have all these elements work (or not work) together that you realize that it's not as simple as it first seemed and the various side effects, overloadings and implied constraints sprinkled throughout the code turned it into a virtual minefield.

Case in point, this video I've watched the past week: https://channel9.msdn.com/posts/C-and-Beyond-2012-Herb-Sutte...

It's a very interesting talk (and probably worth a watch if you're a C++ developer) but the amusing thing to me is that he begins by showing two short pieces of rather simple C++ code and asks the audience if they are UB or not. Seems like nobody can (or want to) answer that question.

Spoiler: the conclusion of the talk is that one of these pieces of code is only legal if the copy constructor of the custom type adheres to certain implied constraints that are not enforced by the compiler and, it seems, few people are aware of.

C++ gives you an embarrassment of footgun riches above and beyond what C provides. On the other hand, idiomatic modern C++ also deftly avoids many of the C pitfalls. e.g. You don't ever touch new, delete, or raw pointers — instead favouring RAII-based smart pointers, and references. You avoid arrays in favour of vectors. This style (as opposed to old-school C-with-classes C++) is way more productive, and way safer.

As somebody who tried his hands on C++ a couple of times only I have to say that basic RAII, templates and virtual functions are not really easy. I mean maybe compared to even worse stuff in C++, but compared to almost every other language, they are still a pain.

Well I was mostly talking from a C dev standpoint. At first I think it's easy to think of C++ as "C with classes". It just feels like syntactical sugar for stuff you'd do in C. RAII just means calling a destructor function at the end of the scope. Virtual functions are like stuffing function pointers in structs, templates are like "macros on steroid" etc... It's like C with some stuff on top.

Of course it turns out that it's a lot more complicated and powerful than that and it's probably the wrong way to look at C++ but it might take a while until you realize that you're not in Kansas anymore. Rust on the other hand forces you to become acquainted with its paradigm right away, unless you're willing to put all your code in unsafe blocks.

I hope there is a new language which combine "C++" + "thread safe and mem safe". That will be a killer language

There is. The name is Rust, I heard

It's not moving, per se, but I've almost convinced my colleagues to use Rust for future projects to move our prototype and research grade machine learning pipeline implementations and data services to Rust. The alternatives were go and C++.

Rust is a clear winner over either in my view because of its traits system and immutable-by-default story. It feels like the kind of C++ I write, but with less boilerplate and typing. The safety story is irrelevant in our case (it's nice, but wasn't an explicit factor in the decision: the other features in Rust may have been developed to support safety, but safety need not exist to provide them). The biggest pain point is that it's not an appropriate language for high performance numerical computation, so we'll have some "glue" there. How that will work is yet to be determined, but the data engineering and infrastructure around the pipeline is definitely going to Rust.

Could someone give a very simple, to-the-point example of a kind of concurrency bug that Rust prevents, for those of us who don't know Rust? (The author explicitly fails to think of any, so I'm hoping someone else can. It'd be more convincing to see one.)

EDIT: I meant a code example, not a paragraph. And I would obviously expect to see how the intended goal is achieved without the bug... otherwise it'd be trivial to prevent any bug (just make everything impossible).

The most basic example would be two threads trying to write the same variable concurrently without synchronisation. As far as I know, as long as you don't use any unsafe code block, it is not possible, i.e. the compiler will protest loudly and the program won't compile.

You are forced by the compiler to implement an explicit exclusion mechanism.

The point is that kind of issue are silent bugs most of the time(up to point) in C, C++. It can work in most cases, and one day, something goes awfully wrong because the thread scheduling is slightly different than usual.

For example, accidentally sharing a lock-less cache or a non-atomic reference counted pointer between threads.

For example this code, which tries to send a reference-counted pointer between threads, which can cause the reference counter to become unsynchronized and random use-after-free:

    use std::thread;
    use std::rc::Rc;

    fn main() {
        let rcs = Rc::new("Hello, World!".to_string());
    
        let thread_rcs = rcs.clone();
        thread::spawn(move || {
            println!("{}", thread_rcs);
        });
    }

Is detected by the compiler and causes this error

    error[E0277]: the trait bound `std::rc::Rc<std::string::String>: std::marker::Send` is not satisfied in `[closure@src/main.rs:8:19: 10:6 thread_rcs:std::rc::Rc<std::string::String>]`
     --> src/main.rs:8:5
      |
    8 |     thread::spawn(move || {
      |     ^^^^^^^^^^^^^ `std::rc::Rc<std::string::String>` cannot be sent between threads safely
      |
      = help: within `[closure@src/main.rs:8:19: 10:6 thread_rcs:std::rc::Rc<std::string::String>]`, the trait `std::marker::Send` is not implemented for `std::rc::Rc<std::string::String>`
      = note: required because it appears within the type `[closure@src/main.rs:8:19: 10:6 thread_rcs:std::rc::Rc<std::string::String>]`
      = note: required by `std::thread::spawn`

Many (most?) of the non-deterministic bugs in concurrent programs arise from two threads trying to write to the same place in the same time, or one thread writing to one place while the other thread just read that place and assumed that it was going to be constant for the moment.

Rust prevents that by having the concepts of "ownership" and "borrowing" built into the language. The Rust book probably explains this better than I ever could, but the basic idea is that you can only have one actor writing to a variable, OR any number of actors reading a variable. But you cannot have multiple actors having write access the same variable at the same time, or one actor writing to it while anyone has read access to it, unless you use some sort of serialization or copy-on-write mechanism.

Sure, the obvious example is your classic thead1 i++, thead2 i-- bug. In C you run each thread a large number of times in parallel and you end up with something other than 0. The solution is to use atomic operations or locks.

Rust doesn't allow having two mutable references to the same memory location at the same time, so you would never encounter that.

> Rust doesn't allow having two mutable references to the same memory location at the same time, so you would never encounter that.

Wouldn't that make a whole class of efficient algorithms impossible though? Like let's say you have an std::list<T> and you want a sorted "view" of the elements. In C++ you'd create an array of pointers and then sort it, and after that you can just modify whatever each slot points to. In Rust you... can't do that because they'd all need to be read-only? I don't imagine you can turn the read-only to read-write on a whim (otherwise if you do this with two of them how the hell would the compiler figure out if two of them point to the same slot?) so the whole thing is just impossible?

There's always the `unsafe` escape hatch - if you need to potentially alias mutable pointers, you can (via using the 'raw pointer' types `const T` and `mut T`).

You can only do this in an `unsafe` block. If you get a bug caused by aliased mutation, then you know exactly where it comes from.

> Wouldn't that make a whole class of efficient algorithms impossible though?

Correct. What it does is prevent data races; this is per se nothing new, but was done as early as the 70s [1]. There've been various and sundry approaches to the same problem over the years (the 90s and early aughts produced a lot of research in this area).

In a way, it is frustrating that so few programming languages offer prevention of data races, so I'm grateful that Rust pushes that; but in another way, it is understandable, because all such mechanisms restrict what you can do, often in undesirable ways. Google "benign data races", for example: data races can be purposely exploited for more efficient implementations.

The other problem is that data races are really the easy part to solve about concurrency. The hard parts are general race conditions, non-determinism/causality, liveness properties (such as freedom from deadlocks and starvation), and performance.

Performance is one of the trickier aspects: in a shared memory system, performance is primarily a matter of reducing contention. But virtually any system that relies on some form of mutual exclusion to prevent data races introduces contention, and you have a constant tug of war between the two concerns.

Particularly frustrating is that all of these issues are what software engineers call "cross-cutting concerns" [2], i.e. things that cannot easily be modularized. For example, performance may suffer from a serialization bottleneck in a module's implementation where you cannot avoid that bottleneck without exposing the module's internals.

[1] https://en.wikipedia.org/wiki/Concurrent_Pascal

[2] https://en.wikipedia.org/wiki/Cross-cutting_concern

I'll add SCOOP to your list since it was commercially deployed as part of Eiffel. I'll throw in the Eiffel paper from the 1980's for people curious about the overall language and methodology.

https://www.eiffel.org/doc/solutions/Concurrent%20programmin...

http://se.ethz.ch/~meyer/publications/acm/eiffel_sigplan.pdf

http://www.eiffel.com/developers/design_by_contract_in_detai...

Since they have strong QA focus, they also have things like test generation from contracts specifying intended behavior. So, safe concurrency was just one benefit of the work that started in the 1980's. Rust takes things further with a simpler and more flexible approach that supports quite a few different styles of concurrency. That's on top of the temporal safety inspired by the Cyclone language for safe, C-like programming.

Rust's real success is it's the first of the safe, system languages to take off with a massive ecosystem due to their great community efforts. That's probably what Ada and Eiffel needed on top of earlier FOSS implementations.

> In Rust you... can't do that because they'd all need to be read-only?

Here's one way to think of it: There's the kind of code that Rust "wants" you to write, and then there's the kind of code that it allows to write.

Rust "wants":

- Lots of immutable data, with relatively sparing use of mutability.

- Clear ownership and borrowing.

Rust allows:

- Pretty much anything you could write in C, with the possible exception of bitfields in structs. You wanna directly access the pic8259 chip from kernel space? Have fun: https://github.com/emk/toyos-rs/blob/fdc5fb8cc8152a63d1b6c85...

The question is, is it worth exercising that full power for "ordinary" Rust code? To use your example, if you have a data structure D and a view V, can you reach "through" V and mutate the underlying D? Of course you can do this, if you really need to. You have lots of choices, including (for example) cells, locks, tricky lifetime annotations and/or unsafe code.

But in practice, it's usually not worth the hassle. What I do in a case like this is step back and ask myself, "How would I solve this problem in a functional language like Clojure, ML or Haskell?" Oftentimes, that will give me a nice, simple solution that Rust likes. But sometimes I actually do need to do things the hard way.

Part of being happy in Rust is learning how to minimize the use of mutable data. For example, if you're writing a video game and you need to update your "world state", do you really need to mutate the existing state? Or can you write a function that takes the old world state and the player's input, and uses them to compute a brand new world state? The latter actually eliminates all kinds of subtle bugs, and it can be done efficiently.

The problem isn't as bad as you might expect; it is for example possible to make a vector of pointers into each of the elements of the LinkedList and sort that; eg.

    let vec_of_refs = ll.iter_mut().collect();
    vec_of_refs.sort_unstable();

The downside is you can't have this vector at the same time as you access the underlying LinkedList. Another option you can do with other container types is to have a vector of indices, but this is extremely inefficient with a LinkedList.

A popular approach is to have both views be indices into an underlying vector where the actual, mutable data is stored. If that isn't good for your situation, it's probably time to use `unsafe` to build an appropriate data structure.

"whole thing is just impossible?"

Some stuff is impossible without using 'unsafe', although less than you'd think at first glance.

At first glance you'd think "If you have to use unsafe, why bother using rust?" But it's actually awesome. If 0.01% of your code is marked unsafe, that gives you the bandwidth to audit that part of the code very closely.

You can do this with internal mutability. Mark the contents of the list nodes mutable by wrapping it in `Cell` or `RefCell`, and Rust will guarantee that any mutations still preserve memory safety.

Or, as Veedrac mentions, you can do it without any tricks if you can give up accessing the list itself while you have the array around.

Borrowing from https://blog.rust-lang.org/2015/04/10/Fearless-Concurrency.h... (linked to from the original post), the following code tries to access a lock-protected vector.

    fn use_lock(mutex: &Mutex<Vec<i32>>) {
        let vec = {
            // acquire the lock
            let mut guard = lock(mutex);
    
            // attempt to return a borrow of the data
            access(&mut guard)
    
            // guard is destroyed here, releasing the lock
        };
    
        // attempt to access the data outside of the lock.
        vec.push(3);
    }

It doesn't compile because the lock is not held long enough.

    error: `guard` does not live long enough
    access(&mut guard)
                ^~~~~

There are several more examples in that article, but you can read them there rather than here!

Thanks (upvoted)! However how in the world would the compiler know if the mutex belongs to the same vector you are accessing? (Unless you're saying the mutex wraps the vector implying that each vector can have one corresponding mutex at most? Which seems quite limiting?)

Thanks!

Yes, when you construct a mutex, you give it ownership of the vector you want it to protect. Due to the way Rust works, once you've given ownership to something else, you can no longer access it yourself.

The only way you can get access to the data again is to "borrow" a reference to it, but this borrow has a "lifetime", which is tied to the period for which you're holding the mutex. This is how the compiler can spot that you've tried to access the vector after the mutex has been released.

As you say, this means that each vector can only have one mutex wrapping it - at least for this implementation of mutex, known as std::sync::Mutex ( https://doc.rust-lang.org/std/sync/struct.Mutex.html).

However, if you're looking for something like a read/write lock, Rust supports that too via std::sync::RwLock - see https://doc.rust-lang.org/std/sync/struct.RwLock.html for more details.

Obviously, the example uses a vector, but Rust has a pretty strong system of generics, so your mutex (or read/write lock) can wrap pretty much any type, e.g. a struct you've defined.

Hope that makes sense - this was one of the areas of Rust that impressed me most!

Yeah that makes sense! It strikes me as pretty similar to volatile-correctness in C++ (it's a technique -- look it up if you haven't heard of it) with the additional constraint that there is only ever one mutable reference, for better or for worse.

For cases where you need more than one mutable reference, you can use `Cell` (single-threaded) or atomics, which allow mutation through shared references.

> Unless you're saying the mutex wraps the vector implying that each vector can have one corresponding mutex at most?

The mutex owns the vector. Locking it returns a MutexGuard, which is a smart pointer/reference to the data it owns[0]. You can nest mutexes and structures if you need something slightly more flexible than a single big lock.

Having multiple (side-by-side( locks for a single structure sounds like a recipe for concurrency bugs & deadlocks.

> Which seems quite limiting?

If your semantics call for a mutex it makes perfect sense. There are other types of locks if you don't want a 1:1 relationship e.g. RWLock allows multiple readers XOR a single writer.

[0] mutable, so you can replace the structure behind the guard, you just can't keep a reference to it after you release the lock due to Rust's borrow checking semantics

> Having multiple (side-by-side( locks for a single structure sounds like a recipe for concurrency bugs & deadlocks.

It is, however, often necessary in order to reduce an undesirable level of contention. For example, hash tables often use fine-grained locking (with one lock per bucket) in order to increase actual parallelism and to prevent the hash table from becoming a serialization bottleneck.

Another common example is the implementation of FIFO queues with two locks (one for the head and one for the tail), which allows threads that read from the queue to avoid/reduce contention with threads that write to the queue.

This is both normal and frequently necessary for performance.

I think the way that you'd implement the per-bucket locks in a hash table is by having a globally shared (but read-only) vector of buckets, each of which would be a mutex wrapping a vector of key-value pairs. In that way, you don't have multiple locks on the same data structure - you instead have one lock per bucket, which falls back to the simple case described above.

I think an approach similar to this (except with multiple sub-hash-tables, each of which is individually locked) is used by https://github.com/veddan/rust-concurrent-hashmap.

For the two-lock FIFO case, you can definitely do this in Rust but you might need to mark it "unsafe". (I'm happy for someone to prove me wrong on this, though!) "Unsafe" tells Rust "look, I know what I'm doing here" and allows you to do things like operate on raw pointers and generally avoiding checks. However, since Rust is no longer checking these assumptions, you need to be very sure of them yourself!

On the plus side, the rest of your code (e.g. the code using your two-lock FIFO) need not know that an implementation is unsafe - the Rust compiler can still check all the assumptions in that code.

But you can implement the same abstraction in C++. It bugs me when people tout library features as language features.

You can't implement it soundly.

Define "soundly". Follow the rules and it won't break. Will it break if someone memcpys some object internals or something? Sure. Just don't do that. The rules that make this abstraction robust in C++ are easy to follow, just like the rule that says "don't use unsafe" in Rust. Just like Rust, you can break the rule if you know what you're doing.

> Follow the rules and it won't break.

That's the problem here: you're on the honor system that everyone knows and follows every rule every time. Maybe you have a really top-notch team, great code review, etc. but can you say for a certainty that this will always be true, or that it's true of every bit of code you use?

Being able to prove that in advance, especially in more complicated scenarios, has a significant value from checking on every build, especially when you think of the many bugs which have been caused by maintenance code breaking some of the assumptions which the original authors had.

You can't just wrap a statement the compiler has deemed to not pass the borrow checker and make it compile. Unsafe blocks can bypass the type system, but you have to call extra magic to do so.

So, really, what we're discussing is how much ceremony a language ought to require before enabling unsafe behavior. I think C++ provides enough ceremony that you can write decently safe programs in C++.

That has empirically been proven false over a decade of C++ use. Additionally, C++ allows for plenty of memory safety problems without any syntactic ceremony.

It's not about ceremony - it's disingenuous to say that C++ can enforce the same safety rules as Rust w.r.t. to lifetimes or that C++ is as safe as Rust is within it's unsafe blocks.

Whilst currently the borrow checker is a bit greedy, it's safe to say that most of the time the borrow checker is checking the code is safe the same way I try to reason about memory accesses in C and C++ when dealing with concurrency or freeing memory. The power of the borrow checker shows itself when reasoning about a local function and no longer requires you to reason about the whole application state as the compiler ensures locality to the extent that shared mutable state is not allowed.

It’s an exactly same reasoning behind static & dynamic type system. “Don’t put null value to this function and it will not break. Just put object with these properties into this function and it will not break.” Static type system allow you to ENFORCE those contracts regarding what is the data. Same with this, Rust’s affine type system allows you to ENFORCE the contracts regarding HOW to use the data (i.e. if already be free, it couldn’t be used any more) as opposed to the infamous guideline “Don’t use the data after it’s already been freed “

Virtually no large scale C++ app has successfully consistently followed the safety rules. It is too easy to, for example, store references to the mutex-protected object somewhere and have those references persist past the unlock operation, allowing unsynchronized access. References and raw pointers are created invisibly in C++ (example: "this"), unlike in Rust where "unsafe" is always explicit (and can be forbidden with a compiler switch or source annotation).

It's arguably even a little bit nicer in SaferCPlusPlus[1]:

    template<typename TVectorPointer>
    void write_foo(TVectorPointer vec_ptr) {
        (*vec_ptr)[3] += 1;
    }
    
    template<typename TVectorPointer>
    auto read_foo(TVectorPointer vec_ptr) {
        return (*vec_ptr)[7];
    }
    
    template<typename TVectorAccessRequester>
    void use_shared_vector(TVectorAccessRequester vec_ar) {
        {
            // obtain a non-const pointer to the vector from the "access requester"
            // blocking if necessary
            auto wl_ptr = vec_ar.writelock_ptr();

            write_foo(wl_ptr);

            // the pointer owns a lock on the vector so as long as the pointer
            // exists it is safe and valid to use
        }

        // obtaining and using a const pointer to the vector
        auto res1 = read_foo(vec_ar.readlock_ptr());
        
        // without a "lock pointer" obtained from an access requester, there is no way
        // to access, or even refer to the shared vector, so you can't access it in an
        // unsafe manner
    }

[1] shameless plug: https://github.com/duneroadrunner/SaferCPlusPlus#asynchronou...

The new Firefox is actually faster.

YEP really!

My only complaint is the marketing.

Firefox isn't slow, sure enough. But it wasn't slow last week either. I've been using it as my primary browser for a long time, and performance was never an issue.

Also, 'Quantum'? Come on now. If they're aiming Firefox at power-users (which they should be), they should know that kind of buzzword abuse is only going to annoy.

Anecdotal evidence, so take it with a grain of salt.

I switched to Firefox 2 days ago, after reading a post about how Firefox has recently got faster. I have been a Chrome user for more than a couple of years now, when I switched from Firefox because it was slow in comparison.

After using Firefox for 1 day, it already seemed slower than what I was used to in Chrome. This was most noticeable on ad-laden sites that displayed video ads or similar. I hadn't installed ad-blockers I had on Chrome yet, so maybe it was just that.

After 1 day of use, I was ready to switch back. But then I installed Quantam yesterday, and boy has it been amazing! It is noticeably faster, without any ad-blockers or extensions.

So atleast for me, it was slower last week, and just got fast.

Why did you switch 2 days ago instead of waiting for 57?

Just a co-incidence?

Yes. :)

Exactly the same situation here. As a new (returned) FF user, v57 is noticeable faster than v56.

Agree about marketing. Mozilla needs to get the word out to those who don't follow the tech scene in any way, shape or form. That's the real challenge, but there also lie the bulk of its potential user base. Not sure exactly how they can do that without resorting to annoying, almost sleazy stuff that Google do (like bundling their browser with many other s/w and OEM system builders)...

Quantum... I like to think of 57 and beyond as a quantum leap from before. They did after all let go of XUL overnight, and people do praise its speed overnight. :-)

> quantum leap from before

So, the smallest possible leap? :P

I like to think that it's loosely related to the multithreaded work-stealing features, breaking as much as possible up into "quanta" of work to be grabbed by the next available thread.

It's a decently memorable codename in any case, and works well enough for that purpose.

I think of it less as "the smallest possible" and more as just "one indivisible unit"- going from 56 to 57 without rewriting the style system in Rust would be a long and probably impossible task.

"quantum" doesn't mean "small".

"quantum leap" means an abrupt, large, leap.

See https://en.wiktionary.org/wiki/quantum_leap

It's a joke. A quantum leap is what you say, but a quantum in physics is all about being the smallest discrete unit of some physical property such as energy.

Right, but it doesn't mean "small", some quanta are quite large.

I agree, not small (seems subjective), but still the smallest! The smallest leap is probably not very large :D

No, a quantum leap is any abrupt leap between any two energy levels (i.e. you can span multiple), it characterizes the abruptness, not the size.

IIRC the term's usage in English derives both from the fact that it is an abrupt leap, and also from the radical change to physics brought forth by quantum mechanics.

Firefox was, up until today, noticeably slower than Chrome. Today it's faster.

Especially on Mac. In my experience, FF on Windows was pretty snappy. Quantum is on a whole other level though.

I agree, to me Firefox and Chrome were both seemingly to the naked eye for me at about the same speed but now I notice some pages Chrome chokes up on Firefox just keeps on rolling along. As for the Quantum comment, I just think of Nuka Cola Quantum and it amuses me a bit, but maybe I just play a little too much Fallout 4.

I don't think Quantum is a buzzword here. It's a code name that is more memorable than "57".

And yes, it was slow. Performance was not an issue for me either, I've been using it for a long time, but solely because of the features, not the speed, and the difference is noticeable.

It was, but lacking NoScript support triggered a speedy manual downgrade to 56.0.2

My kingdom for Tab Groups! https://addons.mozilla.org/en-US/firefox/addon/tab-groups-pa...

Any side effects on web applications? E.g. event handling race conditions or different behavior?

> It replaces approximately 160,000 lines of C++ with 85,000 lines of Rust

Wow! This is great, especially considering how bad and complex (in the bad sense) C++ is. Maybe Rust and Go will finally make the Frankenstein go to sleep

Is this reduction in line count typical? I'm surprised. Or is this discounting parts of the code that have been moved out into separate crates?

There's some good discussion of how it happened here: https://www.reddit.com/r/rust/comments/7cwpbq/fearless_concu...

One of the biggest contributors is "custom derive," which cuts out a lot of boilerplate. Another is that Gecko C++ is also pretty old and so doesn't rely on a lot of what is now standard. It's also just a ground-up rewrite, which has the advantage of hindsight.

It's not a trick of moving code around.

Oh, thanks. Perfect link. It makes sense that rewrites are smaller, but the reduction factor was still surprising.

Is it a concern that acid3 test as terribly faild? - http://acid3.acidtests.org/

I'm sorry? I just scored a 97/100 using FF v57.0 x64. That's decent right? The newest version of Chrome fails likewise.

Also this:

>Acid3, in particular, contains some controversial tests and no longer reflects the consensus of the Web standards it purports to test, especially when it comes to issues affecting mobile browsers. The tests remain available for historical purposes and for use by browser vendors. It would be inappropriate, however, to use them as part of a certification process, especially for mobile browsers.[0]

[0]http://www.acidtests.org/

Both firefox dev edition and chrome (mac) score 97 for me.

So basically it has something to do with my system. It scored only 93, with really dirty rendering. I need to check what's wrong

Fearless, indeed: https://github.com/servo/servo/issues?page=1&q=thread+race

Rust only saves you from simple races, not more complex ones. That's quite a lot already.

Most importantly, though, it preserves _memory safety_ in concurrent situations, so your stuff won't randomly crash, but properly panic.

It's no silver bullet, but it _is_ the "magic sauce" behind Stylo.

More precisely, Rust saves you from data races but not race conditions.

https://blog.regehr.org/archives/490

I’m pretty sure Rust saves you from ALL data races so long as you stay within the boundaries of safe code. Do you have anything at all to reference otherwise that says only certain data races are detected while others are not?

To my knowledge you can defeat the compile-time data race detection if you are either doing unsafe or certain scenarios with Cell/RefCell but even in that case you are guaranteed runtime detection rather than compile time detection.

These feature alone is worth its weight in gold.

qznc puts that better then I do. It saves you from data races, but not from race conditions.

Cell and RefCell are both not Sync (that means they can't be used from multiple threads) for a reason. RefCell does, however, allow borrow checking at runtime, for wrapping it with something that establishes Sync.

Given the a "data race" is essentially defined to be the class of races that Rust's type system guards again, yeah it saves you from all of them.

Rust's definition of "data race" isn't just "what the Rust compiler rejects", it has a specific meaning:

"Safe Rust guarantees an absence of data races, which are defined as: 1. two or more threads concurrently accessing a location of memory, 2. one of them is a write, 3. one of them is unsynchronized."

https://doc.rust-lang.org/nomicon/races.html

Here's a potential use-after-free, listed in my query:

  https://github.com/servo/servo/issues/14014

Of course it's using unsafe code and doing other crazy stuff--a lot of these issues are related to shared, mutable, tree structures. But that's precisely my point. When you're implementing something as sophisticated as Servo and trying to keep things performant and multi-threaded, concurrency is hardly fearless. Servo does this and they have bugs.

Indeed, being "fearless" is precisely how you end up with these bugs, in Rust or any other language. If you're fearless you're more apt to move from a big lock to a fine-grained locking mechanism. That's error prone, including in Rust.

It's like that dude in Florida whose Tesla flew under the tractor trailer. He was fearless in the same way inexperienced engineers using Rust will be when they hear "fearless concurrency". They'll push the envelope when they have no need to, because that's what inexperienced engineers do who haven't been burned, especially when they think their tools make them fire-proof.

I clicked a few of those links and they were mostly instances of the word "race" appearing as part of, for example, "Traceable".

Somewhere on the second page I found a brief mention of a data race in rustc itself (that was fixed).

Looks pretty fearless to me!

  After some digging I found that the issue is that whenever
  src is set and then contentDocument is called right after,
  there is a race condition and contentDocument will return
  null because the Document hasn't been created in the script
  thread yet.

  -- https://github.com/servo/servo/pull/14764

I decided not to cherry pick because I figured people could look through themselves and decide. And because culprits are not always found (or at least documented) before a fix is committed. This is how thread races work in any language--they're hard to definitively pin down, not to mention reliably reproduce, but generally easy to fix (or at least make disappear) once something suspicious comes to your attention.

Concurrency is easy when you don't share mutable data. But when you do have to share mutable data--core, performance-sensitive shared data structures--Rust hardly makes doing so "fearless". What Rust does is make it difficult to _accidentally_ share mutable data.

Try opening some of those and ctrl+f to realize what was actually matched. Hint: not "thread race"

Some are, some aren't. It's difficult to pick them out because some aren't resolved, and some that are resolved were fixed with "works for me" without nailing down the culprit.

I just take issue with "fearless concurrency". All concurrency is fearless if you use a share-nothing architecture. But often times for performance you can't do that. And while Rust may be better than most languages about making it more difficult to screw up, even the Servo folks create race conditions.

Reply on news.ycombinator.com