Hi, how does this compare to Vaex?
https://vaex.io/docs/tutorial.html
Vaex can join larger than memory datasets, but it doesn't support SQL per-se, it uses it's own lazy dataframe DSL.
Vaex can join larger than memory datasets, but it doesn't support SQL per-se, it uses it's own lazy dataframe DSL.
It looks like Vaex focuses more on end users like data scientists while Datafusion focuses more on being a composable embedded library for building analytical engines. For example, InfluxDB IOx, Ballista and ROAPI all uses Datafusion as the compute engine.
On top of that, Datafusion also comes with a builtin SQL planner so users can choose between Dataframe and SQL interfacts.