Saved by the compiler: Parallelizing a loop with Rust and rayon | Hacker News Reader

Saved by the compiler: Parallelizing a loop with Rust and rayon

blog.faraday.io

179 pointsGolDDranks32 comments

Author here. I'm happy to answer questions!

I don't have any performance numbers for this loop because it was already very fast. This was just a warm-up exercise before trying to parallelize more complicated loops.

My experience with Rust has been interesting. On the one hand, if you compare Rust to Ruby or JavaScript, there really is a "Rust tax"—you have to think about ownership and references and types. But Rust pays me back by making refactoring very easy, and by catching some surprisingly subtle errors at compile time. Plus, I really like Cargo and the crates.io ecosystem.

Rust might not be a good match for every project or every programmer, but I'm really enjoying it for low-level, high-performance code.

A huge payoff that Rust provides vs Ruby or JavaScript is when writing development tools. It saves so much time and trouble to install a binary than to try and get someone to download the right version of Ruby, gem install, etc. Rust, while having its tax, is really great about telling you why your code won't compile and keeping you from footgunning yourself.

Trying to get ack or ag working properly on Windows is a bit of a pain. Now there’s an equivalent in Rust, rg, and I can get something that’s mostly superior to these others with `cargo install rg`. (OK, so I had to have installed Rust first. But I had done that.)

There does not appear to be a crate called "rg". Are you perhasp referring to https://crates.io/crates/ripgrep?

He definitely is, it's just that "rg" is the binary that compiling ripgrep produces.

Whoops, yep. `cargo install ripgrep`.

You always have to install something or it must exist first. Node, FCC, clang, Ruby, Python... you just need to think of rustup as the the equivalent of getting you rust compiler environment setup.

What's difficult about it? Just use chocolatey.

You can also get a Windows binary that I think should Just Work :-) https://github.com/BurntSushi/ripgrep/releases

Hooray for standalone programs!

This was a great article. I love step-by-step journeys through what programming actually looks like like this.

A while back I did a whirlwind tour trying out different parallel looping abstractions in different languages, comparing their performance and ease of use. Rust+Rayon was one of the best. Simple code, fast runtime: https://jackmott.github.io/programming/2016/08/30/think-befo...

The first phrase is objectionable, the Pentium D was a (pretty badly implemented) reaction to AMD's announcement that they were going to bring their workstation and server dual-cores to the desktop. And while Intel technically did manage to take the crown the D was a pretty bad dual-core compared to the A64X2.

From an editorial perspective that digression wouldn't have added a whole lot to the point of the page. What would you be hoping to communicate?

It might be slightly good to at least mention that AMD and Intel both worked on this at about the same time. I may make this edit next time I push an update for something else.

I welcome pedantic critiques because, well..I too am pedantic.

Alternatively, remove that mention and just note that over the last 10 years pretty much all computers have become multi-core, even expanding to multi-core phones?

> From an editorial perspective that digression wouldn't have added a whole lot to the point of the page.

Then remove it altogether and only talk in generalities? The rest of the paragraph doesn't namedrop specific brands.

I'm sorry, somehow I thought there was something interesting behind this beyond mere brand loyalty/antipathy. I see now I was mistaken.

Er… what? You're the one who objected on editorial grounds that a digression into the more interesting facets of the issue would detail from the core of the article. I'm providing an easy fix of not raising the issue by not putting in objectionable details in the first place, that has nothing to do with "brand loyalty/antipathy".

As you already hinted at in your article, the choice of MSVC++ as the "best" Windows C++ compiler is highly debatable. GCC, Clang and ICC will all generate significantly faster binaries for your code examples and, in the case of ICC, automatically parallelize the loop and take advantage of any SIMD instructions (up to AVX-512) available on your hardware. It doesn't always get everything right, but for simple cases it's usually very close to optimal in my experience, and vastly superior to MSVC++.

Your homepage crashed my browser. Not sure what you've got going on there. :/

Nice name but how does it relate to your product? I assume it refers to Michael Faraday.

I have a bit of history about him and how he compares to Tesla, along with a tour of monuments to him in London here: https://unop.uk/faraday-tour

Maybe you should switch to a browser written in Rust.

Ironically, it is that browser.

IE6? Dude...

Servo isn’t exactly hard to crash at present.

(“But hang on, it wasn’t supposed to crash!” I hear you saying. These are controlled, deliberate terminations due to detected logic errors, rather than crashes owing to memory unsafety issues like null pointer dereferencing.)

> Your homepage crashed my browser.

Ouch. Sorry to hear that. What browser and version were you running? I'll file an issue.

I asked our CEO where the name came from, and he said that Faraday is famous for being an experimentalist, which ties into our predictive modeling. (We also have lots of customers in the residential solar industry.)

I wouldn't worry, it's heavily customised. If you tested in all the main browsers you're probably fine.

I suspect it was olark, as I can't see anything else obvious. That screenshot image is pretty heavyweight too, I can see it downloading.

I'm on Firefox 49.0.1 and the homepage did crashed mine too. When I visited the homepage, the bottom part was just a white space with nothing in it. Every other windows where unable to display their content anymore. (not the same guy as the first one who's browser crashed, but since my browser was not heavy customized I though would like to know).

I'm actually on Firefox 49.0.1 too. I just thought it was one of my add-ons or DNS hacks. Don't want to report a bug for an edge case but if others have the same issue it might be something more serious. Worked fine in Chrome.

Thank you for confirming this problem! We've reproduced it internally and we're working on it.

Rayon is a great library. It's perfect for spots where you think parallelism might be advantageous but you really want to base that decision on available resources at runtime

Result has a FromIterator impl for converting e.g. from a `Vec<Result<a, b>>` to a `Result<Vec<a>, b>`. Is there a reason to prefer the reduce_with method here over FromIterator?

Reply on news.ycombinator.com