Hacker News Reader

Comment by bitcharmer | Hacker News Reader

I had a look at a couple of the Java benchmarks you've linked to and they are horribly written, completely disregarding stuff like OSR, forgetting about the warm up and ridden with absolutely unnecessary allocations.

Either someone did this on purpose or they have no idea how to write efficient Java code and benchmark it.

View on HN

The benchmark game is always full of these things. Some people are very motivated to make their language shine on it, so it has good[1] implementations, almost everything else is bad. Really, really bad. It's best to just ignore it.

[1] With good defined as fast. Many of the implementations there are so contorted that no one in their right mind would use anything like it in production. But they are fast.

Yep. There were two implementations of whatever in Ada, but one of them failed to print the right output, so was disregarded in favor of the slower, but correct one. I fixed the implementation and now Ada is way, way faster. If I remember correctly I actually had the conversation with whoever is responsible for that website here on HN, and I posted the correct code here (via a link).

— programs have to show the correct output

— program source code is shown

— programs can be contributed

What was the problem?

Are you asking specifically about what I had to do to fix the faster, but previously incorrect implementation? It is on HN somewhere. Other than that, there is no problem. Perhaps people who took the site too seriously may have gotten the wrong idea about the language's performance.

You answered "Yep" to someone's comment that the benchmarks game was "Really, really bad." and that didn't really seem to be the experience you described?

> Some people are very motivated to make their language shine on it, so it has good[1] implementations,

I said "Yep" as in "Yep, some people do contribute", and it has good implementations for some, while shitty ones for others, and it should not be taken way too seriously because of what he just said.

Just seriously enough :-)

Yup, enough just to give me a little nudge towards the direction I was already leaning! If my favorite language is slow, I will not take it seriously at all. :)

Your a man after my own heart!

Here are 2 of the programs austincheney referenced. What in particular is wrong about comparing measurements of these programs?

    n-body Node js #6

https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

    n-body Java #4

https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

Please show your improved programs, along with the measurements which show the improvement, so we can confirm your claims.

The benchmarks game does show measurements which take OSR and warm up into account —

https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

Not warming up hotspot seems to be a constant in benchmarks for people comparing languages with Java. I mean, it's probably valid if your use case is a CLI tool (although, in that case an AOT compiler is clearly a better option)

fyi "Wtf kind of benchmark counts the jvm startup time?"

https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

Thanks for that. Jvm startup time used to be an issue which doesn't appear to exist anymore given those numbers - although as that reference said, it would still matter for something that is finished very quickly. TIL

If I'm not mistaken the hallmark of benchmark game is to use same looking source code and see how fast it runs written in different languages. Basically, it presumes that all languages follow the same programming paradigms.

Mistaken-ish.

"So we accept something intermediate between chaos and rigidity — enough flex & slop & play to allow for Haskell programs that are not just mechanically translated from Fortran; enough similarity in the basic workloads & tested results."

https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

What you describe is more like the "Are We Fast Yet?" project —

https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

Reply on news.ycombinator.com