Serving ML at the Speed of Rust

shvbsle.in

> Rust is not mature enough to support ML out of the box ... If you want to serve your Xgboost or deep learning model then Rust is not the right choice (yet).

Would have liked to hear more about the parenthetical 'yet' even if its just links to other blog posts.

As the article notes, there isn't any official Rust-native support for any common frameworks, and using a C wrapper limits its advantages.

tract (https://github.com/sonos/tract) seems like the most mature for ONNX (for which TF/PT export is good nowadays), and recently it successfully implemented BERT via Hugging Face ONNX export.

Reply on news.ycombinator.com