Hacker News Reader

Comment by vardump | Hacker News Reader

I wonder how much effort the CPU implementation saw. I'd expect 1 socket, 4 memory channel Haswell CPU to be able to filter 40+-10 GB/s of rows per second, if row data can be in column major order. SIMD is pretty often memory bandwidth limited, whether we're talking about CPUs or GPUs.

View on HN

Yeah, I came here to say the same thing. A properly designed and vectorized in-memory database implementation on a modern CPU should essentially limited by the aggregate memory bandwidth across the two sockets, and I would assume the in-memory representation has been compressed. Based on my experience, the performance numbers suggest that the CPU implementation here is suboptimal.

Yeah, I agree this is surprising. If all the data in their CPU implementation were stored in a well-packed columnar format, I'd expect something close to theoretical memory throughput on the query they describe (intermediate counts table fits easily into L1/L2 cache and results are commutative and associative). Cache-coherence issues in the aggregation code are a possible cause.

If I were building a speed-freak analytics database I'd be focusing on making my CPU implementations as fast as possible, since that's what 99% of potential customers are already running. Assuming you get to 40GB/s on this type of query, that's only a factor of 5 slower than the GPU implementation. I'd imagine that for most workloads, a factor of 5 speedup that requires new hardware and lots of energy is kind of a non-starter.

> If I were building a speed-freak analytics database I'd be focusing on making my CPU implementations as fast as possible, since that's what 99% of potential customers are already running.

Well, this is by Nvidia, so of course their incentive is otherwise...

This work was not done by NVIDIA, it was done by MapD, a startup. NVIDIA is promoting the work of a partner; it's a guest post on the NVIDIA developer blog.

Hi, one of the original authors here.

You're confusing rows per second with bytes per second. We're measuring here in rows per second, with random data that takes four bytes per record. So the group-by is showing around 24 GB/sec for CPU and roughly 1 TB/sec for the GPUs. Admittedly we are a small startup and there are some optimizations still to be made on CPU (we're working on being NUMA-aware, for example), but the CPU performance is not bad and still much higher for group by than you see in other databases.

You have to remember that we're building a full database and visualization system and not just optimizing for a single benchmark. In addition we're trying to make the point that you can hit this 1-2TB/sec (depending on the query) on a single server with GPUs, which means assuming the compressed data fits in 192GB of GPU RAM you can get much higher performance that you would see out of a whole rack of beefy CPU servers, particularly when you take into account the network overhead that distributed databases suffer. Furthermore, the bandwidth of Nvidia's Pascal architecture (to be released next year) should have at least 2X the bandwidth by using High Bandwidth Memory and likely significantly higher memory sizes, so our speedups will only increase.

You seem to have two CPU sockets, so I'm assuming you have totally 8 memory channels. You should have 100 GB/s CPU memory bandwidth available. Without NUMA optimization, bandwidth across QPI is obviously much, much less.

In that benchmark you have 8x Nvidia K80s, $4,595.99 each [1], which you compare against 2x unspecified 8 core Intel Xeon CPUs.

How would it look like if you had $36k worth of servers? For every 2 GPUs, you can buy 4 servers with 2 sockets of Xeons each, 64 GB of RAM per server [2]. Total bandwidth for 8 GPUs $ worth of two socket Intel servers is 1600 GB/s. Totally 1 TB of RAM. Granted, it takes more space and more power and communication over ethernet, FC, etc. is slower. On the other hand, instead of just 64 GB per node, you could also expand each 2 CPU node to 1 TB. That's totally 1 * 4 * 4 = 16 TB.

[1]: http://www.amazon.com/Nvidia-Accelerator-passive-cooling-900...

[2]: Maybe not exactly this model, but they should have something within the parameters: http://www.supermicro.nl/products/system/2U/2027/SYS-2027TR-...

have you considered the cost of your system? the aws g2.8xlarge comes with 16gb vram and will cost almost $1,900 per month. the 192gb of vram you are talking about will cost $22,000 per month!! A system with 192gb of normal ram will cost almost an order of magnitude lower than that. How does it make sense to run MapD in that case considering the enormous cost.

It doesn't make sense to run it on AWS, and not just because of cost. Which is why it is run on bare metal, which costs approximately the same as two months of AWS.

They are using GPUs with large amounts of GDDR5 to store the most important data, so the increased memory bandwidth is a major part of the performance upgrade over CPU using DDR3.

edit: The maximum bandwidth for DDR3 looks to be ~50GB/s and they are saying over 500GB/s in the GPU system

I say DDR3 because they have been doing this since before DDR4 became available. Even DDR4 is something like ~70GB/s

Reply on news.ycombinator.com