winwang

hey chris, I found your posts quite inspiring back then, with very poetic ideas. cool to see you follow up here!

I agree about no filter*, I disagree with your reasoning. Scala (2) as a language is quite simple. The complexity came from the incredible power of its building blocks (e.g. implicits and path-dependent types). The lack of filter, as I felt, was moreso on how different two Scala codebases could be. That was part of the point though -- a scalable language, which means changing to the needs of each team.

it's quite the coincidence that ideas like friendship and love happen to be good for socialization and reproduction

Which questions are you referring to specifically?

Oh, my startup isn't about Postgres, but rather a GPU-accelerated Spark: https://news.ycombinator.com/item?id=43964505

What are some bad UX choices you generally dislike in data products?

What options do you use? I don't work for Databricks but I am building my own data infra startup, so I'd like to hear what "good" looks like!

If cost (or perf) is the issue, we're building a super-efficient, GPU-accelerated, easy-to-use Spark: https://news.ycombinator.com/item?id=43964505

I'm building another Spark-based choice now with ParaQuery (GPU-accelerated Spark): https://news.ycombinator.com/item?id=43964505

Not a new way like Ray, but a new way to express Spark super-efficiently (GPU-acceleration): https://news.ycombinator.com/item?id=43964505

Do you guide the LLM to do this specifically? So it doesn't "waste" time on what can be taken care of by static analysis? Would be interesting if you could also integrate traditional analysis tools pre-LLM.

Not the OP but -- I would immediately believe that finding bugs would be a valuable problem to solve. Your questions should probably be answered on an FAQ though, since they are pretty good.

One possibility is crafting (somewhat-)minimal reproductions. There's some work in the FP community to do this via traditional techniques, but they seem quite limited.

Well, I was rejected 4 times (once for a completely unrelated idea though).

The reason I got in was because I actually got traction, but I'm sure my successive failed applications helped a teeny bit at least.

So... pretty straightforward... I think?

Indeed I have, though I have some reservations about its focus on interaction nets (or rather, its marketing).

I ended up making a CUDA-based, data-parallel STLC typechecker (Hindley-Milner)... I want to formally prove its correctness first, but maybe a blog post would be okay either way.

Depends on the workload. Spark can persist dataframes to the cluster as normal. GPUs can also load certain datasets significantly faster.

Gotcha, no problem.

> while people like me who see things a little differently seem condemned to struggle in isolation

Haha, you should consider founder life then. I think many founders feel like this.

The issue with transpiling to GPUs is that it is really hard to do that in a sensible, performant way. Something something a sufficiently smart compiler... There is plenty of @jit in Python, and there's TornadoVM for JVM.

Nvidia's CUDA has been around since the 2000s and is somehow still best way to program for GPUs still. Probably the best SIMT-oriented stack at all, even. From my perspective, it's everyone else stifling themselves.

That's a pretty interesting idea, might take a bit to prepare a useful graphic/post.

Also, what do you think would be the best way to structure such a post?

But, here's a small bit of something perf-y: during large shuffles, I was able to increase overall job performance/efficiency by using external shuffles, even with times of ~5s median shuffle write for a couple hundred MB partitions (I hope I'm remembering this correctly, lol). This is not particularly great, but it did allow for cost-efficiently chewing through some rather large datasets without dealing with memory issues. There's also an awesome side benefit in that it allows us to use cheap spot workers in more scenarios.

If you stand up your compute cluster in the same region as your bucket, there are no egress fees. Otherwise, yes, in general. There are some clouds that don't have egress fees though, i.e. Cloudflare R2.

1. Yes! Would love to contribute back to these projects, since I am already using RAPIDS under the hood. My general goal is to bring GPU acceleration to more workloads. Though, as solo founder, I am finding it difficult to have any time for this at the moment, haha.

2. Hmm, maybe I should mention that we're not "accelerating all operations" -- merely compatible. Spark-RAPIDS has the goal of being byte-for-byte compatible unless incompatible ops are specifically allowed. But... you might be right about that kind of quirk. Would not be surprising, and reminds me of checking behavior between compilers.

I'd say the default should be a focus on compatibility, and work through any extra perf stuff with our customers. Maybe a good quick way to contribute back to open source is to first upstream some tests?

Thanks for your great questions :)

Thanks for the link! Not the first time I came across it, but it's a good one.

If I had to bet on the longer term, I think that something like Mojo will win out -- a programming language (mostly) agnostic to the underlying vector processor hardware. Similar to how Rust can target various SIMD implementations, though I've only dabbled in that.

a) Since being fully-managed is one of my value props, that's probably better for us.

b) Of course I'm only accelerating the Spark/query part. Not sure what you mean. And in that case, I took a query which was 44 minutes on BigQuery and ran it with a "comparable" cluster on ParaQuery in 5.5 minutes. Perf is slightly variable, so maybe it's 40 minutes vs 6 minutes. In that case, ParaQuery would still be 6.5x faster, and >2x cheaper. That being said, it was just a benchmark ETL query with some random data (50b rows), and these things do vary between workloads.

So yeah, without knowing more about the use case you're talking about, hard to say. Even Nvidia has a hard time optimizing certain TPS-DS queries btw, so it's not like I can just 5x anything!

Thanks for the links! I'm planning on contributing kernels back to open source, so will think of a way to be vendor agnostic. As far as I understand, HIP should make that doable.

Not sure what your use cases are, but I haven't had too much issue seeing good gains vs bare Spark -- GCS has not been my bottleneck.

All the Spark GPU acceleration right now is done via the Spark-RAPIDS plugin, so HIP would somehow have to support that. Since cuDF is the core part and hipDF is a thing, it might be doable in the near future.

Enterprises can deploy on their own GCP, and we're planning on releasing on GCP Marketplace.

For a similar cost, what if their pipeline were 5x faster, and they don't have to dealing with managing the deployment themselves?

Thanks for telling me about DataMechanics

Indeed, Spark-RAPIDS has been around for a while! And it's quite simple to have a setup that works. Most of the issues come after the initial PoC, especially for teams not wanting to manage infra, not to mention GPU infra.

Hopefully, we can be another managed solution for those on GCP.

And as for your second point, yep, Spark tuning is definitely annoying! BigQuery is a lot more than jusr the engine, and building a simple interface for a complicated, high-performance process is hard. That's a big reason why I made ParaQuery.

Good question -- it depends. For certain workloads, it might look exactly the same! For others, I found that the memory and VM constraints were creating large inefficiencies. Also, many teams simply don't want to manage that level of data infra: managing EMR, instance type optimization, spark optimization (now with GPU configs!), custom images, upgrades, etc.

We take care of that and make it as easy as pie... or so we hope! On top of that, we also deploy an external shuffle service, and deal with other plugins, connectors, etc.

I suppose it's similar to using Databricks Serverless SQL!

Another thing: we ran into an incompatible (i.e. non-accelerated) operation in one of our first real workloads, so we worked with our customer to speed up that workload even more with a small query optimization.

Nice! I attended a hackathon by Modular last weekend where we got to play with MI300X (sponsored by AMD and Crusoe). My team made a GPU-"accelerated" BM25 in Mojo, but mostly kind of failed at it, haha.

The software stack for AMD is still a bit too nascent for ParaQuery's Spark engine, but certain realtime/online workloads can definitely be programmed pretty fast. They also happen to benefit greatly from the staggering levels of HBM on AMD chips. Hopefully I can take a mini-vacation later in the summer to hack on your GPUs :)

Yep, CPU has to transfer data because no RDMA setup on GCP lol. But that's like 16-32 GB/s of transfer per GPU (assuming T4/L4 nodes), which is much more than network bandwidth. And we're not even network bound, even if there's no warm data (i.e. for our ETL workloads). However, there is some stuff kept on GPU during actual execution for each Spark task even if they aren't running on the GPU at the moment, which makes handling memory and partition sizes... "fun", haha.