MapD: Massive Throughput Database Queries with LLVM on GPUs | Hacker News Reader

MapD: Massive Throughput Database Queries with LLVM on GPUs

devblogs.nvidia.com

79 pointsbsprings29 comments

Are problems that fit in memory on a single machine (96GB in this article) considered "big data" now? This buzzword is becoming absolutely meaningless. But yeah, "Hyper-interactive visualytics at scale.", go nvidia PR.

Also, while it's absolutely amazing that they are able to scan 240B rows/sec, I wonder what one would use this capability for if they can only keep a few hundred million records around at a time? The difference between taking 10 or 100ms to scan the dataset should hardly matter to a user that is running "interactive analytics queries".

You can rent a 128gb machine at Hetzner for $130 a month.

I think we can all agree to draw a line in the sand and say if a kid doing a paper run could afford your servers, you don't get to call it big data.

(not to take away from the linked project, which looks technically awesome)

I disagree with the "big data" line in the sand being whether "a kid doing a paper run could afford your servers". Using spot instances on AWS, you can have a 100 machine cluster with 1.5TB RAM for only $3 per hour.

Working at Common Crawl, some of the coolest projects I've seen have been side projects or weekend projects by interested volunteers. WikiReverse[1] cost only $64USD to parse the metadata of all 3.6 billion pages (even cheaper if you avoided EMR fees) whilst Yelp extracted 748 million US phone numbers from 2 billion pages for $10.60USD.

These days, big data (regardless of how you define the term) is within the reach of the kid with a paper route.

[1]: https://wikireverse.org/

[2]: http://engineeringblog.yelp.com/2015/03/analyzing-the-web-fo...

I appreciate the analogy but it's not like if the server were free big data would disappear, isn't it?

Big Data as the industry sees it is more than just the size of the data.

It is about the nasty work of data validation, consolidation, ETL, unification e.g. single customer view, analytics etc. Some of these can be technically challenging even on multi gigabyte files and require a range of multi disciplinary techniques. And Big Data increasingly means taking old EDW concepts and reimagining them for a more real time world.

No offense but what you just described is everything that is wrong with enterprise "Big Data" today. What many consultants are holding up as a big data panacea is just elbow grease and institutional fortitude... there is no free lunch.

If you know how to do even a simple WHERE on 750M rows in memory in 100ms without using GPU then please share some info, I'd love to learn about it.

eh, any where query that hits an index where the 750M rows are not going to be scanned. The query can easily return in 100ms... fetching the records is a different story.

I actually recently left my job at google to start my own company and that's what we do. We didn't consider using GPUs yet as the queries we are running for the specific usecase we are currently working on (web/ecommerce analytics) are all IO bound [so the first step if we wanted to speed stuff up would be moving the data onto ssds or into memory as we are currently storing everything on disk. however this hasn't been necessary so far]. Our product allows you to run interactive queries (segment/aggregate/etc) against a billion record dataset (in the terrabytes; basically a giant logfile that stores an entry for every interaction we observe on the website/app with lots of metadata) with O(seconds) latency.

We wrote our own code that does "the heavy lifting" but it's pretty much the same that everbody seems to be doing right now and comes down to minimizing the # of bytes read at query time; we index all data into a columnar format (so we only need to load a subset of the data and can pack it tightly for most queries) and then slice the query up into lots of individual shards that we compute in parallel (again, being bound mainly by the 200megs/sec disk read we get out of each machine).

Without being completely serious now but regarding your question; "a simple WHERE" (scanning each row and applying a predicate fn with no shared state/shuffle/merge phase) is pretty straightforward to run on a huge dataset as you can split the execution up into as many pieces as you want and run them individually. Since this allows you to basically choose the problem size per shard you could scan those 750m rows on a ginormous cluster of tamagochis if you can fit a single row onto one ;)

If you or somebody else would like to talk more, please shoot me an email: paul@deepcortex.io

So as far as I understand it, indexes mentioned by others are another story, if index fits in mem then it's great. But as far as I understood this is about what in classic db would be called a sequential scan (that is we don't know beforehand what we will look for).

Of course IO is a bottleneck, but it's not that simple even if you have it in mem already. As you said when sharding is possible you can just throw more boxes at it. But on a single machine the problem is checking what's in your own memory fast enough using CPU. That's where GPUs come in.

Even apart from the name of your company, the idea seems very interesting. I suppose I'm not the only one curious, so could just say a few more words? Are you working on some new technology or is it more like scalable RDBMS focused on latency as a service? I really like the idea of the latter btw, I'd like to use something like Amazon RDS or similar solution, but even ignoring the price they don't seem to be able to beat few boxes with fine tuned postgres when it comes to performance on large datasets. On the other hand, with big data often this the data is the money and companies cannot afford to let somebody else store it.

Btw, maybe put some box with e-mails subscription on your website? I'd love to learn how it will develop but of course without it I will forget about it in a few days.

> But as far as I understood this is about what in classic db would be called a sequential scan

You misunderstood my comment then. The scheme I described is used to do "full table scans" on the data. Also this is not something I came up with; there is a ton of research on this and it's how a number of fairly well known DB/analytics products work.

> But on a single machine the problem is checking what's in your own memory fast enough using CPU. That's where GPUs come in.

You can already scan these data volumes in millisconds on traditional hardware without doing too much fancy stuff or using GPUs. My original point was that I have a hard time coming up with an "interactive analytics" usecase where you'd want to process TB/s on one node that can keep only a few gigs around -- the improvement of doing this on GPUs instead of the old fashioned way on general purpose hardware seems to be that a query over the same dataset returns a few milliseconds faster. I reckon a user running interactive sql queries doesn't really care if their queries return in 10, 50 or 100ms -- I am not even sure I would be able to notice that difference myself. However, if you actually had a usecase for this you could still use the approach I described and make it as fast as you desire by scaling it out [tweaking the shard size] on conventional/commodity machines.

> you can just throw more boxes at it. But on a single machine

The linked article is discussing a setup with at least 8 distinct processing units, too.

> Even apart from the name of your company, the idea seems very interesting. I suppose I'm not the only one curious, so could just say a few more words? Are you working on some new technology

We are focused on delivering actionable insights as well as solving some very specific data problems for our customers right now. IMHO the hard part of doing that is making sure we understand the customer's domain, gather/track the right data from their systems and then work with them to slice and dice and visualize this data to discover "signal" from the "noise" which we can then feed back and use to optimize their website/app. The technical part of being able to handle the relatively large volumes of (non-preaggregated) source data is really just the means to that end. We do not currently offer an "off-the-shelf" version of our product, but again if you or somebody else would like to talk more, please ping me.

Also I hope I not coming across too negatively here. The linked research (compiling SQL to LLVM IR) sure is exciting stuff. I just couldn't help but feel that the hyperbole PR speak was a bit too strong with this one.

Array based columnar storage and SIMD comparisons aren't fast enough?

Spark SQL, with their new Catalyst optimization, is real time compiling their queries into code (assembling Scala quasiquoted fragments into an AST, compiling that). No ridiculously awesome GPU backend here, but some of the same general kinds of tech- compiled queries. https://databricks.com/blog/2015/04/13/deep-dive-into-spark-... http://people.csail.mit.edu/matei/papers/2015/sigmod_spark_s...

Real-time SQL compilation has been common in analytical database platforms for many years. Open source has really been the exception to this for some reason, so it is nice to see that people are starting to close that gap a bit.

Compiling queries should really be viewed as table stakes for a modern data platform.

I use SparkSQL every single day and it is not anywhere close to this.

You could combine SparkSQL with Tachyon but it is still pretty far from what MapD seems to be accomplishing.

Would love to see parts of Spark skip the JVM and call out to Rust or LLVM for some aspects.

I wonder how much effort the CPU implementation saw. I'd expect 1 socket, 4 memory channel Haswell CPU to be able to filter 40+-10 GB/s of rows per second, if row data can be in column major order. SIMD is pretty often memory bandwidth limited, whether we're talking about CPUs or GPUs.

Yeah, I agree this is surprising. If all the data in their CPU implementation were stored in a well-packed columnar format, I'd expect something close to theoretical memory throughput on the query they describe (intermediate counts table fits easily into L1/L2 cache and results are commutative and associative). Cache-coherence issues in the aggregation code are a possible cause.

If I were building a speed-freak analytics database I'd be focusing on making my CPU implementations as fast as possible, since that's what 99% of potential customers are already running. Assuming you get to 40GB/s on this type of query, that's only a factor of 5 slower than the GPU implementation. I'd imagine that for most workloads, a factor of 5 speedup that requires new hardware and lots of energy is kind of a non-starter.

> If I were building a speed-freak analytics database I'd be focusing on making my CPU implementations as fast as possible, since that's what 99% of potential customers are already running.

Well, this is by Nvidia, so of course their incentive is otherwise...

This work was not done by NVIDIA, it was done by MapD, a startup. NVIDIA is promoting the work of a partner; it's a guest post on the NVIDIA developer blog.

Hi, one of the original authors here.

You're confusing rows per second with bytes per second. We're measuring here in rows per second, with random data that takes four bytes per record. So the group-by is showing around 24 GB/sec for CPU and roughly 1 TB/sec for the GPUs. Admittedly we are a small startup and there are some optimizations still to be made on CPU (we're working on being NUMA-aware, for example), but the CPU performance is not bad and still much higher for group by than you see in other databases.

You have to remember that we're building a full database and visualization system and not just optimizing for a single benchmark. In addition we're trying to make the point that you can hit this 1-2TB/sec (depending on the query) on a single server with GPUs, which means assuming the compressed data fits in 192GB of GPU RAM you can get much higher performance that you would see out of a whole rack of beefy CPU servers, particularly when you take into account the network overhead that distributed databases suffer. Furthermore, the bandwidth of Nvidia's Pascal architecture (to be released next year) should have at least 2X the bandwidth by using High Bandwidth Memory and likely significantly higher memory sizes, so our speedups will only increase.

You seem to have two CPU sockets, so I'm assuming you have totally 8 memory channels. You should have 100 GB/s CPU memory bandwidth available. Without NUMA optimization, bandwidth across QPI is obviously much, much less.

In that benchmark you have 8x Nvidia K80s, $4,595.99 each [1], which you compare against 2x unspecified 8 core Intel Xeon CPUs.

How would it look like if you had $36k worth of servers? For every 2 GPUs, you can buy 4 servers with 2 sockets of Xeons each, 64 GB of RAM per server [2]. Total bandwidth for 8 GPUs $ worth of two socket Intel servers is 1600 GB/s. Totally 1 TB of RAM. Granted, it takes more space and more power and communication over ethernet, FC, etc. is slower. On the other hand, instead of just 64 GB per node, you could also expand each 2 CPU node to 1 TB. That's totally 1 * 4 * 4 = 16 TB.

[1]: http://www.amazon.com/Nvidia-Accelerator-passive-cooling-900...

[2]: Maybe not exactly this model, but they should have something within the parameters: http://www.supermicro.nl/products/system/2U/2027/SYS-2027TR-...

have you considered the cost of your system? the aws g2.8xlarge comes with 16gb vram and will cost almost $1,900 per month. the 192gb of vram you are talking about will cost $22,000 per month!! A system with 192gb of normal ram will cost almost an order of magnitude lower than that. How does it make sense to run MapD in that case considering the enormous cost.

It doesn't make sense to run it on AWS, and not just because of cost. Which is why it is run on bare metal, which costs approximately the same as two months of AWS.

Yeah, I came here to say the same thing. A properly designed and vectorized in-memory database implementation on a modern CPU should essentially limited by the aggregate memory bandwidth across the two sockets, and I would assume the in-memory representation has been compressed. Based on my experience, the performance numbers suggest that the CPU implementation here is suboptimal.

They are using GPUs with large amounts of GDDR5 to store the most important data, so the increased memory bandwidth is a major part of the performance upgrade over CPU using DDR3.

edit: The maximum bandwidth for DDR3 looks to be ~50GB/s and they are saying over 500GB/s in the GPU system

I say DDR3 because they have been doing this since before DDR4 became available. Even DDR4 is something like ~70GB/s

similar : PG-Strom (past=CUDA ; now=OpenCL)

"GPGPU Accelerates PostgreSQL"

slide: http://www.slideshare.net/kaigai/gpgpu-accelerates-postgresq... ( Dec 20, 2014 )

HN: https://news.ycombinator.com/item?id=8787414

and Open Source! https://github.com/pg-strom/devel

more info ( from : 2013 )

http://data-informed.com/fast-database-emerges-from-mit-clas...

now:

Demo: http://tweetmap.mapd.com/desktop/ Blog http://www.mapd.com/blog/2015/06/04/mapd/

> that maps OpenGL primitives onto SQL result sets

Here is a set of vertices, now draw me a 2d image that represents the 3d scene those vertices represent from a certain viewpoint.

It boggles my mind that this logic can be twisted into running arbitrary SQL logic. Does it actually do something as magic as overlying SQL set logic onto a graphics scene...where the results of the SQL query are equivalent to the set of vertices that are calculated to not be visible when rendering the scene? Amazing stuff.

I believe these guys are actually going straight from SQL to "GPU instruction code" (so AIUI the SQL query execution is not some hack that is layered on top of the programmable shader pipeline [which is what's responsible for the "here is a list of vertices now draw me a 3d scene part" you mentioned] or something like that but runs directly on the more or less general purpose processor in the graphics card)

I understand the part where they "map the result set to opengl primitives" as that they write the output/result set of the sql query into some datastructure that allows them [after executing the query] to easily use it from opengl to render pretty stuff. This part shouldn't be much different from other texture/vertex data that you'd bring in e.g. from files on disk or a MySQL database into opengl specific buffers so that you can subsequently use it for drawing in your program.

Reply on news.ycombinator.com