So You Wanna Go Fast? | Hacker News Reader

So You Wanna Go Fast?

bravenewgeek.com

95 pointstylertreat35 comments

> When performance really matters, you have to start thinking at the hardware level. Formula One driver Jackie Stewart is famous for once saying, “You don’t have to be an engineer to be be a racing driver, but you do have to have mechanical sympathy.”

If you are going to reference that quote, talk about ring buffers, and false sharing. I think it behooves you to credit Martin Thompson or LMAX in some way. That is a near verbatim quote from the Disruptor papers.

That was the same thought I had when I read through the ring buffer section.

One glaring problem in the Golang space is the memory model doesn't offer Happens-Before guarantees. I wrote a project called Go-Disruptor (github.com/smartystreets/go-disruptor) and found that I can't guarantee ordering of writes and reads between threads which is absolutely critical for a ring buffer implementation.

Would you mind expanding on this? I've never implemented a ring buffer and am genuinely curious.

I haven't written Go, but from what I've heard it lets you mix in platform-specific assembly. Can't you use that to put in memory barriers and/or issue the specific load/store instructions you need?

You can, the Go runtime and standard library use assembly for a number of operations. A drawback is difficulty reading and reasoning about assembly. C style intrinsics are easier.

> One glaring problem in the Golang space is the memory model doesn't offer Happens-Before guarantees.

It does: https://golang.org/ref/mem#tmp_2

> That was the same thought I had when I read through the ring buffer section.

I am glad to see this being called out. I had the same precise 'wait a minute' response to that article.

Replacing a channel with a ring buffer that spins (and uses runtime.Gosched every 1000 failed attempts) is really bad general advice. The other advice is similar.

It may be a good idea for very specific use cases and might have served the author well, but the channel will scale far better in the general case and suffer from have less degenerative cases.

Looking at artificial benchmarks is very misleading. If you take the ring buffer linked in the article and throw 1000 producers and one consumer at it, it will completely and utterly fall over. Channels on the other hand, will behave just fine.

My take: use the language primitives unless you really know what you're doing. Then you should also know that it's not a general optimization.

The problem is 2 fold. The marketing on Go is that it has great concurrency built in, but it doesn't support 2 extremely common concurrency use cases out of the box:

1) lock free queues

2) concurrent maps

Worse than that, the language makes it very difficult to write libraries to support those use cases.

So sure, in your use case, and maybe the Golang language maintainers use cases, dispensing with channels is bad advice.

But it turns out that nearly every single team that has tried to scale golang servers has come to the same position. Give up channels as your concurrency primitives and revert to the sync package if you need to really scale golang programs

Assuming you've got a buffered channel with non-blocking send and receive calls, I seriously doubt that the three lines protected by a mutex are the bottleneck for scaling your service. It's basically exchanging one set of atomic ops for another one.

I'm not saying that you couldn't optimize if you have a very specific need, but it's disingenuous to pretend that the lack of a "lock-free" queue is somehow a fatal flaw or preventing you from scaling go programs.

Normally if you have too much overhead with a channel, you just send more at a time. For example if you are using a channel to distribute out work, send 100 items at a time rather than 1. Then you do fewer channel read and write operations.

Concurrent maps have poor performance because of cache-line bouncing and locking overhead. Everyone I've ever talke to who has benchmarked them has said there isn't much point.

> lock free queues

That may change. Lock free algorithms generate quite a lot of garbage [1] & the Go GC story is much better now than before.

> the language makes it very difficult to write libraries to support those use cases.

Not sure why you hold this view. Atomic op primitives are provided. Is the issue the syntactic white noise of using the unsafe mechanisms?

[1]: can't find the cite of Cliff Click addressing this point. Intuitively, consider that lock free algorithms trade space for time.

Your point about lock free algos generating garbage is true of some algos but not others. The disruptor (last I looked) was lock free and garbage free.

Without starting a huge debate, lack of generics, makes writing collections (especially concurrent safe ones) a trade off between correctness of algorithm & type safety.

I was addressing the GP's map/queue.

> .. generics ..

It should be obvious by now that Go's IDE is the command line. So Go generate young wo/man :) But seriously, having recently switched to C11 from Go, I would take C's crappy pre-processor over poor man's metadata in comments approach of Go. [p.s. edit: in the sense that C11 actually has a far less painful way to achieve type-safe generic code.]

"The marketing on Go is that it has great concurrency built in"

I say this every so often, but it's worth repeating: This is why I tend to consider Go a scripting language rather than a "systems" language. If you're used to Python or Ruby or PHP or something, holy cow does Go ever offer great concurrency support with blistering performance.

If, on the other hand, you've got a task where you're literally counting cycles in the CPU and intensely worried about L1 cache and you're using lock-free structures not because you read a blog post about lock-free last night but because you've tried everything else and it's the only way to hit your performance targets, Go is not great choice. It's probably still miles better than Python/PHP/Ruby/etc. even so, but that's damning with faint praise.

I think Go occupies a rather nice spot on the cost/benefit chart, but it certainly does not occupy the highest performance slot. But to get to that point, expect to pay some more on the cost side, and for a lot of people, it is more than fast enough, with at most just a bit of performance analysis.

Also, I'd suggest that the defining characteristic of channels is "select". If you've got a channel that will never ever be used in a select statement, you're definitely paying a lot for that capability. On the other hand, if you really need it, I suspect it can't get much cheaper. Again, it occupies a rather nice place on the cost/benefit chart, but it is definitely not in the highest performance slot, and, again, for a lot of people it is more than fast enough and a major upgrade over what they are currently using.

>Ultimately, you hit the ceiling of what you can do with it, and you have to migrate off (if you’re a business that is trying to grow, anyway).

I read this post and the one that was linked to and I still don't understand what the problem was that they faced with Google App Engine. A few specifics would have been helpful.

So he hinted at it with, "App Engine was stateless". You can't hold a model in memory on the server with App Engine. You can if you own the server.

Also, App Engine has request length limits (like many platforms). This means you can't hold a persistent connection for real time communication, and you can't run long tasks. Even using App Engine Backends or Tasks are time-limited to about 10 minutes, and your long-running task will often be killed unexpectedly.

Last I checked, App Engine was using an old version of Go, and I don't know if they'll ever move off Python 2.7.

As a platform App Engine is great, but it's not the perfect fit for everything.

Disclosure: I used to work for Workiva.

Thanks for the clarification. He advocated stateless anyway, but the other limits you describe could make things hard and preclude any type of actor model.

My favourite bit of this article was where they pad the struct to move each field onto separate cpu cache lines and see a 15% speed increase.

At that point maybe you should think about switching to using a struct of arrays rather than an array of structs.

This article became hilarious when I reached "App Engine was stateless, so the natural tendency once you're off it was to make everything stateful. And we did." Oh my.

Seriously, if you care about lock-free algorithms, zero-cost abstractions, control of memory management and avoiding false sharing, why use Go in the first place? C/C++ and Rust are much more practical choices, even Java has mature concurrent data structures. IMHO, Go is a sensible choice to achieve 80/100 raw performance. Beyond that, Go is just not the right tool.

It seems like if you're going to care about performance this much you might want to use a language that doesn't discriminate between heap and stack.

Sure you can do all these gymnastics, but why not use tools that were built to do this from the start like Rust and C++?

If you're not first, you're last.

OK, this is completely off topic, but this sentence start from the article jumped out at me:

It’s honestly a good question to ask

Why is it that so many people now start sentences with some variation of "honestly"? I see it in comments here on HN all the time, and in lots of tech blog posts.

Is there a new paper out on how this is effective rhetoric? Is it a regional tick I haven't figured out yet?

Honestly, I find it depressing that so many people feel the need to preface what they are saying with a declaration that they are not, in fact, lying to us.

are many people really doing that?

honestly, perhaps it's some cognitive bias or frequency illusion.

To be honest, I can't tell. Maybe it's just me, honestly.

I have always thought this was code for “I was previously lying to you, but now I am telling you the truth”. I think not.

I have seen this specific complaint on HN several times lately. It's a figure of speech. It means "frankly." Your refusal to pick up on English language idioms -- illogical as they may be -- is, frankly, not interesting. Can we stop?

There's a fine line between idiom and annoying cliché. "Can we stop doing X" is another one.

It's a verbal tick, a meaningless bit at the start of the sentence because the person feels uncomfortable starting the sentence right off and wants a bit of lead-in, or as a short pause for rhythm. Same thing with "basically" - I've seen people start basically every single sentence with "basically" when they're even a bit stressed.

Interestingly, I'm a big fan of the word "interestingly" to start a sentence. It makes sure people understand that what I say is quite fascinating.

"Honestly" is used to add emphasis, to convey a sense of frankness and immediacy, and sometimes to suggest that the writer finds the statement to be an unpleasant but necessary truth. I think it's likely that you know all this already and are indulging in a bit of feigned surprise.

Perhaps it's the new "literally" ?

You may find this simularity in the word "just".

"Let's stop overusing the word honestly." => "Honestly, let's just stop overusing the word honestly."

What is the purpose of "just" in the sentence? Is it a specifier of motivation? Was the task more complicated or difficult to grasp in a previous context? Could we just have a reason for it?

Reply on news.ycombinator.com