I am admittedly a tough sell when the workstation under my desk has 192GB of RAM.
I am admittedly a tough sell when the workstation under my desk has 192GB of RAM.
If you have a very beefy desktop machine and no giant datasets, there isn't a strong reason to use Polars Cloud.
Are you a data scientist running a Polars data pipeline against a subsampled dataset in a notebook on your laptop? With just changing a couple lines of code you can run that same pipeline against your full dataset on a beefy cloud machine which is automatically spun up and spun down for you. If you have so much data that one machine doesn't cut it, you can start running distributed.
In a nutshell, the pitch is very similar to Dask/Ray/Spark, except that it's Polars. A lot of our users say that they came for the speed but stayed for the API, and with Polars Cloud they can use our API and semantics on the cloud. No need to translate it to Dask/Ray/Spark.
This is exactly how I would describe my experience. When I talk to others about polars now I usually quickly mention its fast up front, but then mostly talk about the API, its composability, small surface area, etc. are really what make it great to work with. Having these same semantics backed by eager execution, query optimized lazy API, streaming engine, GPU engine, and now distributed auto-magical ephemeral boxes in the sky engine just make it that much better of a tool.
I think the best selling point speaks to your workstation size- just start with polars vanilla. It'll work great for ages, and if you do need to scale, you can use polars cloud.
That solves what I see as one if the big issues with a lot of these types of projects, which is the really poor performance at smaller sizes, meaning practically you end up using completely different frameworks based on size, which is a bif hassle if you want to rewrite in one direction.
And sure, databricks has an idle shutdown feature, but suppose it takes ~6 hours to process the deal report, and only the first hour needs the scaled up power to compute one table, and the rest of the jobs only need 1/10th the mem and cores. Polars could save these firms a lot of money.