Hacker News Reader

Comment by hharnisch | Hacker News Reader

I think a lot of people are overlooking something really important here. There are numerous comments about optimizing the code to perform better on this task. I might even go as far to say people are taking offense because their language of choice didn't perform as well they'd hoped. Many of these comments add complexity to the solution and make it harder to maintain. I like the fact that the author kept things simple. It's kind of like using the default settings for each language and then comparing the results. Both the simple version and optimized version are important data points, and they give you a better view of each languages limitations.

View on HN

A true coder wanting to implement things near correct with algorithm complexity (big O), will try to find how to implement the right code by taking time to research understood and time tested patterns they've learned (it is an applied science after all) and implement them.

Saying the average joe doesn't care to understand what his/her Lang is doing under the hood which will doom them to repeat complexity that many here will respond with "duh" is not justification that a Lang is better - because it supports that behavior "better".

I don't know erlang that well but I'll be damned if I don't spend the time to find out how to properly do M:N with efficient tail recursion.

If one doesn't care for efficiency at all then who cares about benchmarks?

You missed the point. People aren't posting better algorithms, they're posting domain specific knowledge to improve efficiency. The kind of stuff that takes you away from building new features that add value. Benchmarking and efficiency is an important datapoint when picking a language. It's important to understand how un-optimized code performs too.

In my comment I support domain knowledge experts.

In my comment I purposely mentioned erlang because to implement algorithmic efficiency you must have domain expertise.

"I don't know erlang that well but I'll be damned if I don't spend the time to find out how to properly do M:N with efficient tail recursion."

Tail recursion is an efficiency, doing it right in erlang requires domain expertise.

Building new features and adding value is business. Someone with domain knowledge in any of the langs will be able to implement those business values in the best possible way.

If your business is built with shitty coders doing shitty things you will eventually find yourself in hot shit. Even if it takes 5 years. What comes around goes around.

I would take the domain knowledge developer over any monkey anytime. That's your 10x developer. That's who built all that awesome open source stuff you use all day. Not the business feature value guy.

He builds businesses that fail 99% of the time. That other guy building that obscure Lang which turns out useful for that niche topic (I.e concurrency) lives on even in concept to build or inspire other langs/ers.

You see, that domain expert knows to pick the right algorithm for the right job.

He knows the right Lang for the right job.

Even if that job is only business value.

Long live domain knowledge experts and those who wish to attain it.

Absolutely. Benchmark should be done with idiomatic code you'd like too see in a documentation. Not ultra optimized ones. At least not only.

sorry but the scala version is actually pretty pretty dumb.

also go is not better than all of these languages. what we learned here, is actually that go is easier since you only have one common way to do this task. go has goroutines, but go doesn't have runnables, futures, actors, completionstages, executioncontexts. GO has GOMAXPROCS and not dispatchers, thread pools, etc..

Also the JVM eve Java only code would actually beat golang in many ways, but thats nothing you should be scared of. Golang is 1 1/2 year old, the JVM lives like 20+ years and they spent numerous times about the GC implementations (they could be even changed so that you could use a better GC for conurrency or for parallelism or for synchronous execution). Go is good for the task it is created. Other languages maybe tend to be more general so that you could do many things and due to their years of knowledge they are actually more optimized than go could be / they could be more optimized than go could be.

> go doesn't have runnables, futures, actors, completionstages, executioncontexts

Some of us (most of us, I would contend) who have been successfully programming highly concurrent software without them for 20+ years are totally OK with that.

> Some of us (most of us, I would contend) who have been successfully programming highly concurrent software without them for 20+ years are totally OK with that.

I work on parallel software every day and I would not be OK with having threads and channels as my only tools. I would agree that they're usually the best thing to reach for initially, but it's essential to be able to have low-level control over the scheduling if you want to get good parallel performance out of the hardware. I have workloads that today result in 3x speedups on 4 cores but would become slower than the sequential algorithm on 1 core if I were to rip out the highly-tuned parallel scheduler and replace it with threads and channels.

To be fair, though, I'm working on CPU-bound software and if my workload were I/O bound I'd likely have a different outlook on this.

Actually, Go is at least 6 years old; the original announcement was Oct 30, 2009:

https://youtu.be/rKnDgT73v8s

yeah sorry however I would actually use go version 1 as a reference. everything before was changing just too often, there was go fix however still many changes. also everything before 1.0 wasn't targeted too much about performance. also I can remember how much the template spec changed. thats what I used these days quite heavily. I have a book about go that dates 2011, and I think that was go 0.7 as far as I remember.

Fair enough; in that case Go 1.0 is just about 4 years old:

https://blog.golang.org/go-version-1-is-released

I am scared that Java takes an order of magnitude more memory than Go with its 20+ years of optimization and all. Java's GC does not appear to be significantly better than Go's one. Also Go's runtime and GC both written in Go, something Java has not been able to do so yet. So Java appears less general purpose to me. Yes Java has humongous number of libraries and frameworks but that is expected of a popular language that is 20+ years old.

> Java's GC does not appear to be significantly better than Go's one.

Java's GC is generational, allowing bump allocation in the nursery. That is a huge advantage.

Considering Go is showing under ~10ms GC pauses. Java's advantage seems theoretical rather than real world. I work in Java at my job and I have not seen such low GC pauses in Java ever. edit: Also considering huge garbage idiomatic Java libraries and user code generates, Generational GC in Java is a basic requirement not a powerful advantage over Go.

Bump allocation in the nursery is a huge advantage. There's a reason why fast Go code goes out of its way to avoid allocations (notice that the benchmark tool contains an allocation profiler), and it's because allocations are expensive when you don't have a generational GC. If you do have a generational GC, then allocations are incredibly cheap: just a pointer bump and check. It's like 4 or 5 instructions compared to the dozens in a typical malloc. With that kind of collector, you don't have to spend nearly as much time tuning your program to avoid allocations. This is the reason why Java has been slow to add value types: when you have a bump allocator, they really aren't needed as much (which is not to say they aren't needed at all).

In the HotSpot VM, you can tune the GC to get any type of max garbage collection pause you want (-XX:MaxGCPauseMillis). This is basic functionality that any concurrent garbage collector supports. The downside, of course, is that pauses will happen more frequently, reducing throughput. That is why just saying "it's under 10 ms, so it's really fast" is very misleading: when your GC is interruptible, you can of course get any max pause time you want. But the GC may not be able to keep up with your allocation rate, and it may start being invoked too often. That's why you need to not just look at latency, but throughput--something that sound bites won't capture.

Side note: many functional languages use the same bump alloc method because they tend to have higher allocation rates than imperative programs.

I really hope this doesn't become a meme.

Pause time is only one metric that's relevant when looking at GC, another very important one is throughput.

In fact you can get very low (10-50 msec) pause times with the latest JVMs even with very large heaps, you can just request it at the command line, but you'll pay for it with slower execution. There is always a tradeoff. Go doesn't have anything fundamentally new in this field.

There is a report here on an adapted HotSpot (not integrated into the mainline JVM though) that gives <10msec pause times at the 99th percentile and an 8 gigabyte heap:

   http://welf.se/files/OL15.pdf

But you pay for it with a 15% CPU overhead.

Go's GC seems to actually do a full concurrent mark/sweep of the entire heap every single time. That suggests to me that throughput could be rather poor.

Right, scanning large heaps means someone has to pay along the way. The Go GC pays with lowered throughput much like the adapted HotSpot VM is showing. The 10ms pause times for go were reported for a 250 gigabyte heap, however, and that is pretty amazing.

But the lowered throughput has to be present as you are scanning the heap and you have enabled the write barrier to make the concurrent scanning work its invariants. In fact, the newest versions will make aggresively allocating goroutines run slower by using some of their timeslot when they request data do run the mark phase.

> Go's GC seems to actually do a full concurrent mark/sweep of the entire heap every single time. That suggests to me that throughput could be rather poor.

Not to mention that you lose bump allocation in the nursery. Nongenerational GC is a lose-lose all around.

Have you tried using the G1 garbage collector? http://www.oracle.com/technetwork/articles/java/g1gc-1984535...

See the -XX:MaxGCPauseMillis=NNN argument, you might be able to set it to 10ms on highly performant hardware.

Then "Skynet" should be removed from the description, as well as Actors. I guess the word "whatever" deserves to stay though.

Summing a bunch of numbers is a silly task anyways

In the end, isn't that all we're really doing?

This has no external data so workers may avoid a cache fetch let alone having enough data for regular cache misses. Even a simple strlen call in each worker could begin to add the kind of issues that affect meaningful performance.

How a language handles misses/blocking/threading is more important than what it can do in a perfect cpu only world.

Reply on news.ycombinator.com