ONNX: The Open Standard for Seamless Machine Learning Interoperability | Hacker News Reader

ONNX: The Open Standard for Seamless Machine Learning Interoperability

github.com

52 pointsMVPMaster11 comments

At this point ONNX is the most mature inference engine for mobile.

Had the joy of playing around with TFLite, Pytorch Mobile, GGML for work and nothing came close to ONNX in terms of stability across a wide array of devices.

Also, model conversions are a breeze.

You can open and visualize any ONNX model in this website: https://netron.app

Netron is awesome and works for a lot of other formats. It's been surprising how often I have used it for quickly debugging model inputs.

Onnx is really helpful for shipping models from development to productionisation environments. It's standard is only designed to allow "safe operations", anything to do with text manipulation, for example, you'd have to write your own operator or glue logic for.

Shameless self promotion here but I wrote a little bit about calling Onnx in Scala here - https://tajd.co.uk/2023/10/15/onnx-interface-scala

Well, it's definitely an improvement over raw dogging deserialized pickled ML models.

Unfortunately, at least as of last time I evaluated it, it still can't handle certain things that are currently only done with pickling in the Python ecosystem. Much of what's covered by scikit-learn's feature_extraction package, for example.

Not, I think, for any reason that's inherent to what those components are doing; a lot of it's just that much of the existing Python ML ecosystem was not engineered with robust productionization in mind. Possibly because the very existence of Pickle means everyone has an easy (if horrifying) way to get the job done for 0 effort. As the sklearn maintainers remind people every time they close an issue that asks for it, robust and secure model serialization is something that would have had to have been designed into the project from day 1, and doing it now would essentially require a rewrite.

It’s still a WIP but skops (https://skops.readthedocs.io/en/stable/) may be of interest (for secure sklearn serialization)

Wasn't safetensors created because ONNX was considered unsafe, correct me if I'm wrong.

I thought it was other way round chronologically? Safe existed first then onnx ? May very well be wrong though

Safetensors were created because people were distributing models as python pickles, the basic built-in serialization format of Python.

https://checkoway.net/musings/pickle/

If you want in on the fun.

927.

Reply on news.ycombinator.com