winwang

They have >1PB of data to ETL, with some queries hitting 450TB of pure shuffle.

It's very true that most users don't need something like BigQuery or Snowflake. That's why some startups have come up to save Snowflake cost by "simply" putting a postgres instance in front of it!

In fact, I just advised someone recently to simply use Postgres instead of BigQuery since they had <1TB and their queries weren't super intensive.

I'm not sure about the rest of your comment, but we would likely still want GPUs even with highly multicore CPUs. Case in point: the upper-range Threadripper series.

It makes sense to have two specialized systems: a low-latency system, and a high-throughput system, as it's a real tradeoff. Most people/apps need low-latency.

As for throughput and efficiency... turns out that shaving off lots of circuitry allows you to power less circuitry! GPUs have a lot of sharing going on and not a lot of "smarts". That doesn't even touch on their integrated throughput optimized DRAM (VRAM/HBM). So... not quite. We'd still be gaming on GPUs :)

I was about to comment that Gluten is only targeting CPU vectorization, but then I found this (very cool!): https://github.com/apache/incubator-gluten/issues/9098

I'm not very familiar with Gluten, but I'll still comment on the CPU side though, assuming that one of Gluten's goals is to use the full vector processing (SIMD) potential of the CPU. In that case, we'd still be memory(-bandwidth)-bound, not to mention the significantly lower FLOPs of the CPU itself. If we vectorize Spark (or any MPP) for efficient compute, perhaps we should run it on hardware optimized for vectorized, super-parallel, high-throughput compute.

Also, there's nothing which says we can't use Gluten to have even more CPU+GPU utilization!

Since you mentioned non-x86, how are things on the ARM side? I believe I heard AWS's Graviton + Correto combo was a huge increase for JVM efficiency.

FPGAs... I somehow highly doubt their efficiency in terms of being the "core" (heh) processor. However, "compute storage" with FPGAs right next to the flash is really interesting.

Very true. Can't get those numbers even if you get an entire single-tenant CPU VM. Minor note, A100 40G is 1.5TB/s (and much easier to obtain).

That being said, ParaQuery mainly uses T4 and L4 GPUs with "just" ~300 GB/s bandwidth. I believe (correct me if I'm wrong) that should be around a 64-core VM, though obviously dependent on the actual VM family.

I'm not too familiar with HeavyDB, but here are the main differences:

- We're fully compatible with Spark SQL (and Spark). Meaning little to no migration overhead.

- Our focus is on distributed compute first.

- That means ParaQuery isn't a database, just an MPP engine (for now). Also means no data ingestion/migration needed.

Is the shuffle the biggest issue? Not too sure about joins but one of the datasets we're currently dealing with has a couple trillion rows. Would love to chat about this!

Just checked out Velox. It's awesome that you're reducing duplicate eng effort! What was your talk about?

Not sure if Spark is a classical portion for GPU compute ;) Well, outside of HPC and research.

SQL on GPUs is definitely a research classic, dating back to 2004 at least: https://gamma.cs.unc.edu/DB/

Large scale shuffles: Absolutely. One of the larger queries we ran saw a 450TB shuffle -- this may require more than just deploying the spark-rapids plugin, however (depends on the query itself and specific VMs used). Shuffling was the majority of the time and saw 100% (...99%?) GPU utilization. I presume this is partially due to compressing shuffle partitions. Network/disk I/O is definitely not the bottleneck here.

It's difficult to say what "workloads" are significant, and easier to talk about what doesn't really work AFAIK. Large-scale shuffles might see 4x efficiency, assuming you can somehow offload the hash shuffle memory, have scalable fast storage, etc... which we do. Note this is even on GCP, where there isn't any "great" networking infra available.

Things that don't get accelerated include multi-column UDFs and some incompatible operations. These aren't physical/logical limitations, it's just where the software is right now: https://github.com/NVIDIA/spark-rapids/issues

Multi-column UDF support would likely require some compiler-esque work in Scala (which I happen to have experience in).

A few things I expect to be "very" good: joins, string aggregations (empirically), sorting (clustering). Operations which stress memory bandwidth will likely be "surprisingly" good (surprising to most people).

Otherwise, Nvidia has published a bunch of really-good-looking public data, along with some other public companies.

Outside of Spark, I think many people underestimate how "low-latency" GPUs can be. 100 microseconds and above is highly likely to be a good fit for GPU acceleration in general, though that could be as low as 10 microseconds (today).

No relationship... yet! Hoping to have a good relationship in the future so I have a business reason to fly to Japan :D

Btw, interesting thing they said here: "By utilization of GPU (Graphic Processor Unit) device which has thousands cores per chip"

It's more like "hundreds", since the number of "real" cores is like (CUDA cores / 32). Though I think we're about to see 1k cores (SMSPs).

That being said, I do believe CUDA cores have more interesting capabilities than a typical vector lane, i.e. for memory operations (thank the compiler). Would love to be corrected!

Still figuring out pricing! For our first customers, we're doing pricing as either bytes scanned or by compute time, similar to BigQuery. Also experimenting with a contract that also gives the minimum of the two potential charges (up to a sustainable limit).

However, for deployments to the customer's cloud, it would be a stereotypical enterprise license + support.

Can't wait to actually add an FAQ to the site, hopefully based off the questions asked here. Pricing is one of the things preventing me from just allowing self-serve, since it has to be stable, sustainable, and cheap!

Also, with the GPU clouds, pricing would have to be different per cloud, though I guess I can worry about that later. Would be crazy cheap(er) to process on them.

As far as I know, GPUs are definitely still being used in crypto/web3... and AI for that matter :P

Thanks! I was also told to make a performance-focused demo... didn't do it in time, but was able to go from a 44-minute BigQuery job to a 5.5-minute ParaQuery job, with a similar dataset/query as the video here.

8x faster!

This was just a brief moment of thought over a year ago, but I can try to summarize. I was thinking about how to unify variables in certain simple Datalog settings. If we think of a clause as a vector of variables, then simple unifications can look like just a gather operation. A gather can be thought of as a matrix-vector multiplication, but that's not really useful (performance wise). But if those variables are also in a linear equation, then it becomes possibly-useful, e.g. for something like `P(x, y) :- E(x, 3x+4y)`

As someone who doesn't know very much about graphics (ironically), you're welcome and hope it helps!

lol, I haven't thought about it like that, true. though of course, I mean compared to CPUs :P

I try and use tensor cores for non-obvious things every now and then. The most promising so far seems to be for linear arithmetic in Datalog, but that's just matrix-vector/gemv

I don't -- unfortunately not too well-versed in this field! But I was a bit fascinated with SWAR after I randomly thought of how to prefix-sum with int multiplication, later finding out that it is indeed an old trick as I suspected (I'm definitely not on this thread btw): https://mastodon.social/@dougall/109913251096277108

As for 64-bit... well, I mostly avoid using high-end GPUs, but I was of the impression that i64 is just simulated. In fact, I was thinking of using the full warp as a "pipeline" to implement u32 division (mostly as a joke), almost like anti-SWAR. There was some old-ish paper detailing arithmetic latencies in GPUs and division was approximately more than 32x multiplication (...or I could be misremembering).

It's 32 32-bit values which get sorted. I don't think a GPU sort would beat a CPU sort at this scale, even if you don't take kernel launch time into account. CPUs are simply too fast for (super-)small data, especially with AVX-512. But if we're talking about a larger amount of data, that would be a different story, i.e. as part of a normal gpu mergesort.

Oh wow, TIL, thanks. I usually call stuff like that SWAR, and every now-and-then I try to think of a way to (fruitfully) use it. The "SIMD" in this case was just an allusion to warp-wide functions looking like how one might use SIMD in CPU code, as opposed to typical SIMT CUDA.

Also, StringZilla looks amazing -- I just became your 1000th Github follower :)

Haha, if you're the type to toss out the phrase "well known", then yes!

I hope this can help shed the misconception that GPUs are only good at linear algebra and FP arithmetic, which I've been hearing a whole lot!

Edit: learned a bunch, but the "uniform" registers and 64-bit (memory) performance are some easy standouts.

I'm working on GPU-accelerated SQL+Spark in a zero-hassle package: https://paraquery.com

Been prod for a few months, recently ripping through 900TB with ~5x efficiency (customer was on BigQuery).

If anyone has any data/infra challenges, or just wanna talk about this kind of tech, lemme know :D

In case it's slow to load for others: https://web.archive.org/web/20250426185806/https://www.mail-...

I don't quite fully agree with these, but I agree with the general spirit.

I agree, it sucks to be lumped in. What's your proposed solution?

Fun fact which I'm 50%(?) sure of: a single branch divergence for integer instructions on current nvidia GPUs won't hurt perf, because there are only 16 int32 lanes anyway.

Congrats! How do you guys deal with SOC2/HIPAA/etc.? Or are those separate concerns?

I've seen shops (e.g. Netflix I think) use Datalog for certain query types.

This might unironically be a reason for me to finally try Clojure!

Sorry, I was just trying to make a joke (about both insane systems and interview questions) since the question you answered was a bit unclear. Guess it didn't land, haha.

how would you do it if your logs were printed on paper with a printer, each line printed with stochastic timing (due to a bug), with an ink containing a chemical tracer with halflife `h` (after being put to paper), but the ink is randomly sampled from several (`m`) inks of different halflives `h1`, h2`,... `hn`? assume `p` different printers scattered across the 10 most populous US cities. you may use standard unix utilities.