Hacker News Reader

If your data project is bottlenecked by speed, and the current libraries which already leverage a low level language such as C/C++ like pandas/numpy in Python in R and dplyr in R don't suffice, then you probably should be switching tooling altogether (e.g. to "big data" tools like Spark. Which have a Python/R API anyways).

View on HN

If speed is a concern and you're working in R, you should use data.table instead of dplyr. In Hadley Wickham's words: "We optimise dplyr for expressiveness on medium data; feel free to use data.table for raw speed on bigger data.". Although personally I don't think data.table is any less expressive.

As a former data.table user and late-comer to dplyr and pipes, I think data.table is just as expressive when _writing_ (and I do miss indices and in-group references), but is really awful to read. I don't miss the days of reviewing data.table commands, esp. given that many teammates have been relatively inexperienced programmers.

Reply on news.ycombinator.com