Hacker News Reader

what are some practical uses for this if living in a pure R world?

Is this like BigMemory but for data frames?

https://cran.r-project.org/web/packages/bigmemory/index.html

Thanks.

It's often much faster than rds. And in the long long term there will be tools for computing on feather files that don't require loading it into memory. (In the short term I'll add ways to pull in slices of the full dataset)

Faster because it isn't (currently) using compression (which rds uses by default) or faster period?

Either way, the idea of mixed Python/R pipelines with feather file intermediates input/outputs is pretty sweet. Learn in scikit, save to feather, plot in ggplot2... using Make to tie the pieces together?

It's usually faster than either compressed or uncompressed RDS - but if you have heavily duplicated data, compressed RDS can be faster than feather (depending on some tradeoff between compression speed and disk speed). Feather will probably gain compression support eventually.

Reply on news.ycombinator.com