Hacker News Reader

I think it really depends on your data size. All the benchmarks I can find are on massive datasets, with tens of millions of rows or thousands of columns. I’m sure there are significant performance gains in these situations. Our data just wasn’t big enough.

View on HN

There was a time when I thought 10s of millions of rows was massive. Now it just send run of mill.

With larger data it really depends on the algorithm. If you must iterate over more than a few GB at a time, GPU memory capacity and bus speeds become prohibitive, while a dead-simple implementation on a single CPU with 100+ cores and TBs of RAM goes brrr.

CPU RAM is generally much slower.

8 channels of DDR4-3200 only provide 200GB/s bandwidth. RTX 3090 has 936GB/s. So even 4 socket Xeon won't catch up.

That would only be an advantage if you had to do multiple passes over the data, otherwise the data would still go through the CPU RAM before getting loaded onto the GPU, no?

Definitely the case in the state of the art stuff like neural networks.

Many if not most other algorithms are iterative. Hell, even sorting is.

Reply on news.ycombinator.com