Hacker News Reader

Comment by michalsustr | Hacker News Reader

> the fastest, heavily-optimised Java solution runs in just under a second on my machine

I don’t understand how this is possible. The file in question has 13GB, while the fastest commonly available SSDs are 12400 MB/s. Am I missing something?

View on HN

I think this bit in the baseline section applies to the Java one too

>Note that that’s a best-of-five measurement, so I’m allowing the file to be cached. Who knows whether Linux will allow all 13GB to be kept in disk cache, though presumably it does, because the first time it took closer to 6 seconds.

Yea, I assumed that. Which makes the parallel version improvements still interesting but surely it's very artificial. You can't processes all the data at the same time if you don't have it all yet.

Benchmarking this gets tricky once you realize that the file might be entirely cached if the computer has enough RAM.

Hah, we use different computers. I was thinking, this might be tricky if you aren't careful to keep the file entirely cached in RAM.

there are no disk file read times in the original rules. file is in memfs.

If he runs the test multiple times, the file should all be cached in RAM after the first run.

My guess: If you run the benchmark several times, the OS will cache the file in RAM.

most of the file remains in the OS disk cache after the first run

that might be a sustained single threaded read performance.

What if the method of access was concurrent from different parts of the file, and was operating system cached?

actual 1brc runs with data in RAM disk, he could've/should've done the same.

Reply on news.ycombinator.com