Comment by twotwotwo | Hacker News Reader

Comment by twotwotwo | Hacker News Reader

Parent Full thread

Sommers is an absolute beast, does totally astounding work and is an asset to the community. But even when we say "performance-critical" most of us don't mean getting 3M req/sec from one box like she did w/Haywire (https://twitter.com/kellabyte/status/547063455048404992). In other words, pushing the hardware absolutely to its limits is another game (Sommers' game :) ) from simply getting a tricky part of your app performing at the level you need. And, confusingly, "high-performance" or "performance-critical" are terms you might use to describe either task.

For context, the person whose presentation triggered Kelly's comment, MIT scholar Neha Narula, was experimenting with an 80 core machine, and she replied "it's kind of amazing I could push Go that far," and "to be fair they weren't really optimizing for my use case :)" -- her whole presentation is on YouTube at https://www.youtube.com/watch?v=Mbg1COjhsJU (some wild stuff--she got improvements for >48-core machines pushed into the Go GC) and those replies I quoted are at https://twitter.com/neha/status/564569903219634176 .

But even when we say "performance-critical" most of us don't mean getting 3M req/sec from one box like she did w/Haywire

We are not all writing web apps. Some of use are in machine learning, NLP, signal processing, etc. where squeezing out as much performance as possible does matter.

In those fields Go is still weak. No autovectorization, no OpenMP, no direct CUDA integration, GC overhead, etc. Luckily, this can often be worked around since cgo is so good. One can write performance-intensive parts in C or C++, compile with the latest gcc or clang and link it with the Go code to drive it. This is often an understated advantage of Go compared to Java, where JNI calls are expensive. But the Go camp always advocate for porting everything to Go (because fast compile times).

What does OpenMP do for you that Go's built-in concurrency doesn't?

OpenMP is for parallelisation, not concurrency. OpenMP is designed for embarrassingly parallel loops that you typically encounter in numeric code, where parallelization is as simple as tagging a for loop with a pragma and (if necessary) marking variables that should be synchronized and/or critical sections.

E.g. you can make the libsvm library parellalized and scale up to many cores by adding two pragma statements.

I have a Go package that attempts to bring some of this functionality to Go [1]. But it's definitely not the same as having OpenMP.

[1] http://godoc.org/github.com/danieldk/par

Indeed, we agree that Go is suitable for some things people call high-performance but not others. I do not advocate porting everything to Go. Not only is it not always a good fit, but there's a very high bar to clear to justify revamping any perfectly good codebase. (I use Python at work and wouldn't advocate for porting anything.)

If you read her tweets she states C++ would have been a better option, though.

Reply on news.ycombinator.com