Hacker News Reader

Hacker News

Comment by winterbloom | Hacker News Reader

Parent Full thread

winterbloom

This is a staggering ~3x improvement in just under 2 years since Sep was introduced June, 2023.

You can't claim this when you also do a huge hardware jump

View on HN

They also included 0.9.0 vs 0.10.0. on the new hardware. (21385 vs 18203), so the jump because of software is 17%.

Then if we take 0.9.0 on previous hardware (13088) and add the 17%, it's 15375. Version 0.1.0 was 7335.

So... 15375/7335 -> a staggering 2.1x improvement in just under 2 years

They claim a 3GB/s improvement versus previous version of sep on equal hardware — and unlike “marketing” benchmarks, include the actual speed achieved and the hardware used.

Do note that this speed even before the 3GB/s improvement exceeds the bandwidth of most disks, so the bottleneck is loading data in memory. I don't know of many applications where CSV is produced and consumed in memory, so I wonder what the use is.

"We can parse at x GB/s" is more or less the reciprocal of "we need y% of your CPU capacity to saturate I/O".

Higher x -> lower y -> more CPU for my actual workload.

Slower than network! In-memory processing of OLAP tables, streaming splitters, large data set division… but also the faster the parser, the less time you spend parsing and the more you spend doing actual work

This is honestly something that caught me off-guard a bit. If you have good internal network connectivity, small queries and your relational database has the data in memory, it can be faster to fetch data from the DB via the network than reading it from disk.

Like, sure, I can give you an application server with faster disks and more memory and you or me are certainly capable of implementing an application server that could load the data from disk faster than all of that. And then we build caching to keep the hot data in memory, because that's faster.

But then we've spent very advanced development resources to build a relational database with some application code at the edge.

This can make sense in some high frequency trading situations, but in many more mundane web-backends, a chunky database and someone capable of optimizing stupid queries enable and simplify the work of a much bigger number of developers.

You can also get this with Infiniband, although it is less surprising, and basically what you’d expect to see.

I did once use a system where the network bandwidth was in the same ballpark as the memory bandwidth, which might not be surprising for some of the real HPC-heads here but it surprised me!

Decompression is your friend. Usually CSV compresses really well.

Multiple cores decompressing LZ4 compressed data can achieve crazy bandwidth. More than 5 GB/s per core.

Perhaps, but i think we are well past the moore's law era where a 3x speed up is to be expected just from hardware. Its still a pretty impressive feat in the modern era.

> You can't claim this when you also do a huge hardware jump

Well, they did. Personally, I find it an interesting way of looking at it, it's a lens for the "real performance" one could get using this software year over year. (Not saying it isn't a misleading or fallacious claim though.)

Yea wtf is that chart, it literally skips 4 cpu generations where it shows “massive performance gain”.

Straight to the trash with this post.

But it repeats the 0.9.0 test on the new hardware. So the first big jump is a hardware change, but the second jump is the software changes.

It also appears to be reporting whole-CPU vs. single thread, 1.3 GB/sec is not impressive for single thread perf

Agreed. How hard is it to keep hardware fixed, load the data into memory, and use a single core for your benchmarks? When I see a chart like that I think, "What else are they hiding?"

Folks should check out https://github.com/dathere/qsv if they need an actually fast CSV parser.

I mean... A single 9950x core is going to struggle to do more than 16 GB/second of direct mem copy bandwidth. So being within an order of magnitude of that seems reasonable

4 generations?

5950x is Zen 3

9950x is Zen 5

Sine Zen 2 (3000) the mobile CPUs are up by a thousand respectively to their desktop counterparts. edit: Or Nx2000 where N is from Zen N.

And even with 2, CPU generations aren't what they used to be back when a candy bar cost less than a dollar.

Reply on news.ycombinator.com