Hacker News Reader

Comment by wodenokoto | Hacker News Reader

> ONNX is an open format built to represent machine learning models. ONNX defines a common set of operators - the building blocks of machine learning and deep learning models - and a common file format to enable AI developers to use models with a variety of frameworks, tools, runtimes, and compilers.

- onnx.ai

I have never heard of this before and had to look it up. Is it widely used? Can I define a model in onnx and run it "everywhere", instead of learning pytorch or tensorflow?

View on HN

Yes. Could be that you have to check what operators is available when you move from one framework to another using ONNX. One example is ONNX.js that make it possible to run the model using JavaScript, made by Microsoft. If the operator is not available (ex. ArgMax for WebGL) you have to switch to something equivalent and retrain. List of supported operators for ONNX.js: https://github.com/microsoft/onnxjs/blob/master/docs/operato...

Onnx has really poor support by microsoft. I suspect they've basically abandoned it internally (their onnxjs variant is orders of magnitude slower than tfjs[1]). It's a good 'neutral standard' for the moment but we should all eventually probably move away from it long term.

[1] https://github.com/microsoft/onnxjs/issues/304

ONNX != onnxjs

ONNX is a representation format for ML models (mostly neural networks). onnxjs is a just a browser runtime for ONNX models. While it may be true that onnxjs is neglected, please note that the 'main' runtime, onnxruntime, is under heavy active development[1].

Moreover, Microsoft is not the sole steward of the ONNX ecosystem. They are one of many contributors, alongside companies like Facebook, Amazon, Nvidia, and many others [2].

I don't think ONNX is going away anytime soon. Not so sure about the TF ecosystem though.

[1] https://github.com/microsoft/onnxruntime/releases

[2] https://onnx.ai/about.html

I've tried inference on the python version of onnx and it usually varies between hitting a OOM limit (while with TF it works fine) to being an order of magnitude slower. Even if the codebase is still being changed I don't see much reason for people to use it other than as a convenient distribution format.

Interesting, I did not encounter such discrepancies in my work with these tools.

There could be multiple reasons for the degraded performance:

- Are we comparing apples to apples here (heh), e.g. ResNet-50 vs ResNet-50?

- Was the ONNX model ported from TF? There are known issues with that path (https://onnxruntime.ai/docs/how-to/tune-performance.html#my-...)

- Have you tried tuning an execution provider for your specific target platform?(https://onnxruntime.ai/docs/reference/execution-providers/#s...)

ONNX is a representation and interchange format that training/inference frameworks can use to represent models. However, ONNX does not have a tensor engine of its own. So you would still define and train your model in tensorflow, pytorch, etc, and then save the model in ONNX format, at which point it can be ported to any inference service that supports the ONNX format.

Yes the goal is to run it framework independent. Pytorch for example can export models to ONNX.

Well you would still need to learn Pytorch/tensorflow and define your network there, you then export it to ONNX format for deployment.

I think you need to use pytorch or tensorflow to create the network and train it. After training you export it to ONNX. I suggest you to convert from ONNX to Openvino.

Reply on news.ycombinator.com