The programs in the shootout are nothing like the programs people typically run and write. For instance, most of the programs I write do a lot of I/O. The shootout programs do none.
> Startup time doesn't matter at all
It certainly does in cluster environments where you regularly spin up new server instances. It's also important for command-line tools.
This is certainly true, however not because of the startup time per se which is entirely negligible, especially compared to the time it takes to boot the whole freaking OS, but rather because of the warmup time of the JVM ... for high-volume apps, cold JVMs don't provide a good throughput and so you need a smart load-balancer that knows to not route as many requests as it can to cold machines.
To alleviate that, when the auto-scaling feature kicks in to add new machines on Amazon's EC2 (using their load balancer called ELB), it ads 3 machines instead of one.
Then the auto-scaling feature kicks in again when the latency levels are lower than a threshold for 30 mins, removing machines from the cluster and so it stabilizes itself to the optimal number of machines required.
Compared to Go however, I would still go with Java, because you can do a lot more with a Java server. Our app is both CPU bound and I/O bound and is able to process about 4000 requests per second per server in under 30ms per served request on c1.medium instances. And this is for a real world app that needs to do logging and requests to Memcached/DynamoDB and make predictions in real time.
In my experience, the only way we could have better bang for the buck would be to build the app in C/C++.
Which is as well, because you only want to benchmark for CPU bound parts of the program.
IO is the same speed for all languages, so it cancels out.
>It certainly does in cluster environments where you regularly spin up new server instances.
Not really a common use case.
>It's also important for command-line tools.
I don't think command-line-tools are Go's or Java's niche.
That's not true, particularly in concurrent systems. Recent improvements to Go's scheduler (that make it more aware of various I/O concerns) have made it 2x faster under some workloads. You don't just optimize the compiler to produce the tightest code, you must also finesse its integration with the rest of the world.
> Not really a common use case.
If you say so.
> I don't think command-line-tools are Go's or Java's niche.
There are a lot of Go command-line tools out there. I think Go is a superb language for this area.
Why the irony? How many cluster deployments you know where you "spin up instances all the time"???
Compared to general server / network application programming done in both Java and Go it's LESS than a drop in the bucket.
If you work somewhere that this does not hold, you could also be an outlier.
> Why the irony? How many cluster deployments you know
> where you "spin up instances all the time"???
How do you know what programs people typically run and write?
Assumption? Guess? Data?
But you can draw the same conclusion by browsing GitHub.
1) I think the way in which the benchmarks game programs are most unlike a lot of other programs is that these are tiny tiny toy programs.
"Toy programs" as-in -- http://books.google.com/books?id=gQ-fSqbLfFoC&lpg=PP1...
And that's intentional.
2) Why do you say "...I/O. The shootout programs do none."?
regex-dna, k-nucleotide, and reverse-complement do read data in FASTA format before processing.
(fasta and reverse-complement also write data in FASTA format -- but redirected to /dev/null)
Not TBs of I/O but not none.
regex-dna
reverse-complement
k-nucleotide