Polars has been my go-to for six months now. The performance gains for what I do are pretty great, and the ability to write out Parquet with minimal hassle is just dandy.
If it weren’t for resume-driven architectures, it would make a great alternative to Spark for high volume bronze-to-silver cleanup and normalization steps.
It's interesting that they are explicitly not committing to holding the API constant (which seems quite common & expect for a "1.0"). I generally like that philosophy, but I wonder if it will lead to some unexpected whiplash in the future for people now expecting the core to be stable.
It's a bit unclear in the docs, but I think what they're saying is that if there's a need for a breaking change, they're "not afraid to release version 2.0", rather than saying they're "not afraid to make a breaking 1.x release".
Polars is already effectively Pandas 2.0 with a lot of lessons learned. Hopefully, there is not too much additional churn.
Any examples od how to run it on a GPU?
It is still in development AFAIK. They did an announcement, but no news on the actual release yet
Yes only found the announcement [1] that the Polars team and NVIDIA engineers are working on a GPU engine, but other than that no concrete examples. Github issues also don't provide any hints on the status, only one open item [2] where most comments are prior to the announcement.