Apache Arrow DataFusion 7.0 released
arrow.apache.org
arrow.apache.org
- Significant performance improvements
- Memory tracking and spill to disk
- Support for decimal type
- New datafusion-contrib organization with python implementation and s3 support
- More math functions
- Public expression simplification api
- Arrow2 branch
Currently in the works
- More features for working with limited memory and large datasets
- Ballista improvements on context, state management, extensibility, and file system access