Does anyone else think that C++ makes more sense for ML work than Python? I'd been thinking so for years. Both for deployment/performance and data wrangling purposes.
Julia has a fast maturing data wrangling super-project (Queryverse).
C++ seems ideal for me right now because it is the only other language with a somewhat mature stack (perhaps Julia as well, but I haven't played too much around with that).
In this case, julia absolutely is worth checking out. It does static analysis on it's intermediate representation to automatically identify and isolate statically inferrable regions of programs and then stitches them together if it ever encounters any dynamism.
Julia's type system is extremely powerful and expressive and can do a lot of things that are incredibly difficult in fully static languages precisely because it does allow dynamism.
- Python allows for higher level description of algorithms, which means researchers can focus more on the ML stuff and less on low level details.
- There is no performance gain in going from Python to C++, because in both cases the models are compiled to specific binary formats to be executed on dedicated hardwares. TensorFlow enables accelerators not only for training, but also for data transformations and preprocessing.
The backend of TF/PyTorch is written in C++ anyway, so the more complex the model, the less time it needs to spend in the glue code (frontend) that is written in Python. Therefore, rewriting complex models in full C++, for example by using TF/PyTorch C++ API, probably won't much improve the performance.
In this paper the author rewrites some ML models in Rust using tch-rs (Rust binding for PyTorch C++ API) and finds the performance not that much better (even some models perform worse):