Hacker News Reader

Comment by menaerus | Hacker News Reader

> The total file is ~12 gigabytes, so we should expect to be able to get the task done in 0.3 seconds or so.

How? EPYC-Milan features PCIe 4.0 and theoretical maximum throughput of x4 sequential read is ~8GB/s.

In bare-metal machines, this figure is rather closer to 5-6GB/s, and in cloud environments such as CCX33, I imagine it could be even less.

So, unless I am missing something you'd need ~2 seconds only to read the contents of the ~12GB file.

View on HN

This assumes the file is already in page cache, and therefore already in RAM. The 0.3 seconds is the time to get it from RAM to the CPU L3 cache.

That sounds like cheating. The exercise says to process a large file that's on disk. If the data is in RAM then that's a whole different game.

The benchmark is run 5 times without flushing file system caches. If you mmap it, it will still be in memory between process runs. The author intended this FWIW

That sounds like a benchmarking flaw.

But then, writing good benchmarks is an art in itself.

If the cache is not flushed, it will still be in memory between runs whether you mmap it or not.

That depends a lot on the OS. I would not count on a 13 GB file being cached in RAM after having been read through once.

Isn’t it run multiple times and the slowest is dropped? Why would you expect the file to be evicted from ram so quickly?

That makes sense now, thanks.

Reply on news.ycombinator.com