Playing with Go1.1beta2, it's much faster
blog.ant0ine.com
blog.ant0ine.com
Go has a some way to go still and it can be improved in all areas: GC, algorithms, heuristics, code generation, ...
I'm actually interested in what you have to share regarding performance characteristics of Go versus Java - but a bit more detail than, "Go has a some way to go still and it can be improved in all areas: GC, algorithms, heuristics, code generation," would be useful, particularly in comparison to Java, which you have called out.
Also - I think 36% is a pretty amazing improvement - particularly if Google can persist these performance increases.
In terms of performance data check out quad core results for Go 1 (go 1.1 would need to improve by considerably more than 33% to best the JVM):
http://benchmarksgame.alioth.debian.org/u32q/benchmark.php?t...
Also see Netty (Java) outperforming go by a wide margin here: http://www.techempower.com/blog/2013/03/28/framework-benchma...
The poster's comments regarding GC, heuristics and code generation are vague, but it's no secret that IBM/Sunoracle and co. have very sophisticated GC and JIT implementations which they have been optimizing for 15+ years and it shows through in the benchmark results.
I like Go, I just think that we need to be wary of expecting it to be as fast as the JVM and even MSCLR straight out the gate. It will likely take years to reach that level of sophistication.
I am sure if Go wasn't being done at Google, most hackers would actually ignore it.
Just check how successful the Go like predecessors from the authors were, when working at other companies.
I think the language suffers from an hallo effect.
Most of the negative Go posts I've seen are to do with limitations of the language and performance relative to JVM languages. As much as I think go has a lot going for it I'm not about to claim that it's anywhere near as expressive as Python or even Scala. There has also been some frustration about denials over the performance gap between Go and the JVM despite very clear data in favor of the JVM. I'm not as bothered by the performance gap as I am about the head-in-the-sand denials. The JVM's have been optimized for 15+ years, so it's ok to admit it may be a few more years before performance can match the JVM.
That might be enough to draw interest, but what sustains that interest is the quality of the work and the utility of the environment.
> If Go was a random project on GitHub
Where it's hosted is irrelevant. It's about the work. Plenty of active, popular projects are hosted on GitHub. Some of them were just "random projects on GitHub" when they started. What distinguishes them is that there are motivated groups of people working on them.
That is pretty much the zinger. If Google hadn't been involved, Go probably wouldn't be were it is at all.
Makes it easier to decide whether Go is the right language choice for the problem at hand. Quite often it won't be and that's fine.
I would hope that we hackers would pay at least some attention to any project involving Ken Thompson and Rob Pike though.
http://benchmarksgame.alioth.debian.org/u64/benchmark.php?te...
http://benchmarksgame.alioth.debian.org/u32q/benchmark.php?t...
The 64 bit quad core benchmark is using go 1.0.3:
http://benchmarksgame.alioth.debian.org/u64q/benchmark.php?t...
Note: The much faster version Go's DNA regex that uses PCRE rather than "regexp", but it is currently broken due to a build error. (http://benchmarksgame.alioth.debian.org/u64/program.php?test...)
Go uses less memory than Java 7, but then again so does Java 1.1
http://benchmarksgame.alioth.debian.org/u64q/benchmark.php?t...
That said, Java still beats Go quite readily in this benchmark spread, particularly in regex and binary-trees where Go does extremely poor. It should be noted that lots of languages implement their regex libraries in C for speed, while Go's implementation is written in Go. Still, I think it should do better than this.
That Java beats Go overall is not particularly surprising, Java is very mature and it's jit-compiler has very advanced optimizations, Perhaps by Go 2.0 we'll see where Go 'lands' in terms of performance, there's obviously alot more to be had.
http://benchmarksgame.alioth.debian.org/u32q/benchmark.php?t...
http://benchmarksgame.alioth.debian.org/u32q/program.php?tes...
Note that:
1) It only ever uses one core, because
2) It isn't configured with GOMAXPROCS, which therefore defaults to 1.
So, sure, they have an N-core machine, but they're only allowing go to access 1 core.
Notice on the list provided by `trailfox` the "≈ CPU Load" column -- when the programs are written to use 4 cores "they're" allowing Go to access 4 cores.
The go program in question was written without that instruction; therefore, it's only allowed to use one core.
Only "allowed" by who?
If you mean that "the Go program in question" is by-design a sequential program then -- yes of course it is!
The contributed Go programs that were written for multi-core use more than one core -- http://benchmarksgame.alioth.debian.org/u32q/program.php?tes...
At program startup time? Java is a joke in this area.
At memory consumption? Java has a high fixed memory cost. It also uses more memory than Go does. There are fundamental reasons for this. Java needs to keep around runtime type information because the code can change at runtime due to new classes being loaded. Go does not. Java also chose to implement a scheme where any object can serve as a mutex or condition variable. The result is a few words of extra memory per object when compared with Go... usually somewhere in the neighborhood of 8 or 16 bytes per object, depending on architecture. Using memory efficiently is HUGE in modern architectures and Go is better at that.
Go also gives you more control over how your program uses memory. This is because Go has both pointers and value types. The result is that things that would have had to be "boxed" (i.e. separate memory allocations) in Java can be unboxed in Go. If you know C, think of it as the difference between doing a malloc for each structure element, and just having a bunch of structs stored by value inside another struct.
Go is faster at interfacing with native code. Have you checked the speed of JNI lately? Speaking as someone who's written a lot of JNI... it's not pretty.
Java still has the edge in one area: HotSpot has a precise garbage collector, something Go doesn't currently have. But I have a feeling that gap is going to be closed.
Note: I am oversimplifying the unboxing discussion a bit. In theory, it might be possible to unbox certain structures in Java, such as non-nullable final fields. However, Go preserves a lot more of the intentions of the programmer in this regard-- and without knowing how the programmer intended to use the memory, most optimizers will struggle.
No. In actual performance for most cases of algorithmic code.
http://benchmarksgame.alioth.debian.org/u64q/benchmark.php?t...
Java is 2-6 times faster and uses more memory for slightly more code. I'm willing to trade.
Yes, "those are not real world programs etc". In general the shootout gives a quite realistic results (not to mention that anybody who thinks he can do better can go code his own solution). Programs are programs. In my own tests with larger programs Go has been 1/2 the time of Python and slower than Java too. If you're not IO bound, those things matter. If you are IO bound, you could as well run Python, since you aren't missing much.
Startup time doesn't matter at all except if you do scripting or old-style CGI. Java (and Go for that matter) target long running network servers and such.
>Go also gives you more control over how your program uses memory.
On the other hand Java gives most control to the JVM, which is smarter than most programmer's fine tuning and knows a whole lot more about the code's runtime behavior.
The programs in the shootout are nothing like the programs people typically run and write. For instance, most of the programs I write do a lot of I/O. The shootout programs do none.
> Startup time doesn't matter at all
It certainly does in cluster environments where you regularly spin up new server instances. It's also important for command-line tools.
This is certainly true, however not because of the startup time per se which is entirely negligible, especially compared to the time it takes to boot the whole freaking OS, but rather because of the warmup time of the JVM ... for high-volume apps, cold JVMs don't provide a good throughput and so you need a smart load-balancer that knows to not route as many requests as it can to cold machines.
To alleviate that, when the auto-scaling feature kicks in to add new machines on Amazon's EC2 (using their load balancer called ELB), it ads 3 machines instead of one.
Then the auto-scaling feature kicks in again when the latency levels are lower than a threshold for 30 mins, removing machines from the cluster and so it stabilizes itself to the optimal number of machines required.
Compared to Go however, I would still go with Java, because you can do a lot more with a Java server. Our app is both CPU bound and I/O bound and is able to process about 4000 requests per second per server in under 30ms per served request on c1.medium instances. And this is for a real world app that needs to do logging and requests to Memcached/DynamoDB and make predictions in real time.
In my experience, the only way we could have better bang for the buck would be to build the app in C/C++.
Which is as well, because you only want to benchmark for CPU bound parts of the program.
IO is the same speed for all languages, so it cancels out.
>It certainly does in cluster environments where you regularly spin up new server instances.
Not really a common use case.
>It's also important for command-line tools.
I don't think command-line-tools are Go's or Java's niche.
That's not true, particularly in concurrent systems. Recent improvements to Go's scheduler (that make it more aware of various I/O concerns) have made it 2x faster under some workloads. You don't just optimize the compiler to produce the tightest code, you must also finesse its integration with the rest of the world.
> Not really a common use case.
If you say so.
> I don't think command-line-tools are Go's or Java's niche.
There are a lot of Go command-line tools out there. I think Go is a superb language for this area.
Why the irony? How many cluster deployments you know where you "spin up instances all the time"???
Compared to general server / network application programming done in both Java and Go it's LESS than a drop in the bucket.
If you work somewhere that this does not hold, you could also be an outlier.
> Why the irony? How many cluster deployments you know
> where you "spin up instances all the time"???
Any cluster environment whose applications abide 12-Factor semantics stands to benefit greatly from low startup times. This describes both SOA environments like Amazon and Netflix, and PaaS environments like Heroku and [what is enabled by] Docker.How do you know what programs people typically run and write?
Assumption? Guess? Data?
But you can draw the same conclusion by browsing GitHub.
1) I think the way in which the benchmarks game programs are most unlike a lot of other programs is that these are tiny tiny toy programs.
"Toy programs" as-in -- http://books.google.com/books?id=gQ-fSqbLfFoC&lpg=PP1...
And that's intentional.
2) Why do you say "...I/O. The shootout programs do none."?
regex-dna, k-nucleotide, and reverse-complement do read data in FASTA format before processing.
(fasta and reverse-complement also write data in FASTA format -- but redirected to /dev/null)
Not TBs of I/O but not none.
regex-dna
reverse-complement
k-nucleotidehttp://www.techempower.com/blog/2013/03/28/framework-benchma...
Memory consumption is often not a major concern on modern systems, saving 50 MB or even 500 MB is seldom as important as the ability to process more requests per second.
I'm busy learning Go, and am very impressed by the performance gains in 1.1 (and GC improvements). My main concern at the moment is how much less expressive the language is relative to Python, Scala and Ruby. Go is fast, but so is Scala, and Scala is also very expressive compared to Go.
Go has a definite edge in environments where memory footprint is important and simple interfacing with native libraries is critital. Scala and the JVM are still more appealing to me in most other areas.
rather than this: https://github.com/TechEmpower/FrameworkBenchmarks/blob/mast...
(not to mention the 2 other Java files to setup the server).
Oh, and in the recent shootout, netty gets to 30,000 rpm, whereas Go peaks around 10,000.
Based on these results Go and Netty should be much closer: https://gist.github.com/errnoh/5320784
Here's Go 1.1 vs 1.0.3 on a bit beefier machine: https://gist.github.com/errnoh/5320784
Next benchmark will have Go 1.1 running, can't promise that it'll scale as well as on this server but it'll be significant change anyways.
And most of all, main part of the language is that it's intuitive to write and comes with a clear and readable spec and standard library.
edit: updated gist with Netty using the same configuration
The difference between 50 MB and 500 MB is that the former is more likely to fit in L3 cache than the latter. Ignore that difference at your peril.
Tell that to the folks that have to pay for cloud or VPS servers... Also, less memory usage means higher CPU cache hit rates and fewer page faults.
are you _really_ sure it is not just your bias that is preventing you from "expressing" what you want? ie. you want to write python/java in Go?
http://benchmarksgame.alioth.debian.org/u32/benchmark.php?te...
That's what the original designers of Java thought too. It turns out that this thinking is wrong, though. Main memory may be big, but CPU caches are still small. For example, the laptop I am typing this on has 64 kilobytes of L1 cache, 256 kilobytes of L2 cache, and 3072 kilobytes of L3 cache. That is not a lot. Turns out, when you're spending all your time waiting for main memory, you are not "processing more requests per second." Main memory is at least two orders of magnitude slower than CPU cache.
It is always a matter if you are using a JVM (from which vendor) or compiling to native code.
Language != Implementation
Ideally, yes. But in practice a language and its implementation (whichever one you choose) is a tool. If you're comparing tools, you've got to compare a language with its implementation.
Which from 6g, gogcc, go express you pick for Go?
Which from OpenJDK, Oracle JVM, IBM J9, gcj, RoboVM, Aonix Perc ... you pick for Java?
Which from Sun Forte, IBM xlc, hpcc, gcc, clang, CINT, Visual C++, ... you pick for C or C++?
Which from CPython, IronPython, PyPy, JPython, ... you pick for Python?
For all others there is a definitive, used in the overwhelming majority of cases, implementation:
Go: 6g Java: Oracle JVM Python: CPython Ruby: Matz's Haskell: GHC ...
So of course, the presenter will carefully select the best implementation to prove his arguments, even if there are others available that contradict him/her.
There are advantages to the way Java does things. You can distribute a single binary for multiple architectures, for example. However, it does have costs, and one of those is slow startup time.
Even Sun, nowadays Oracle, sells them as part of the Java SDK for embedded platforms.
That the JVM from Oracle, that most people think as the only way to run Java, does not offer that option out of the box, does not mean other vendors don't do it.
It's pretty common in any language that's seeing significant development, targeting a version that's not going to be too long in the tooth by the time you want to go to production.