Hacker News Reader

Comment by joshgev | Hacker News Reader

I'm more excited about StableHLO and IREE than about their integration into Pytorch, Tensorflow, etc.

I want to see a DSL that can be used to describe models elegantly and then export them either to a shared object or to something that can be run with a runtime (in this case IREE). Things like ONNX and TorchScript promised this but I've had little luck getting these to work well enough to trust them in large scale production deployments.

I understand that PyTorch is an awesome tool for researchers, but it doesn't necessarily fit into a prod environment.

View on HN

> I understand that PyTorch is an awesome tool for researchers, but it doesn't necessarily fit into a prod environment.

You need to write some infrastructure around PyTorch to make it work. Something like a key/mapping in each checkpoint that says which architecture to choose with which parameters.

It sure could be easier, but is saving the model's code into the checkpoint enough? Things like the data pre-processing expected by the model would also have to be included for it to really be self-contained.

Yes I'm facing this when trying to convert a YOLO-based model to TorchScript for mobile (React Native) usage. I wish I can also package the whole pre/post processing from Python to TorchScript instead of having to rewrite it in JS.

I'm curious about your view on ONNX. At work we did a few prototypes and it seemed to work well enough for our use cases, and we're moving to it. What is it that we haven't seen yet that gave you trouble?

Admittedly we're on a reasonably easy situation: we just have to deploy models (some from scikit-learn, some from Keras, some from PyTorch) to various users who mainly run a specific version of python under Windows and Linux, with CPU and GPU support.

We're working on some of the DSL-related parts of this in https://github.com/aesara-devs

Brandon, I’m curious how the goals of your representation differ from, say, Jaxprs.

Why should I look at Aesara’s representation of multi-dimensional array programs when I might already use JAX’s?

Does Aesara support a staging transformation that allows me to construct programs in your representation from a subset of Python?

I’m personally interested in the answers to these questions, given what I know about IREE, JAX, and XLA — as a user in the space, I haven’t been able to determine how Aesara would actually benefit me over JAX.

Note that I know that Aesara can use JAX as a backend — but I’m trying to ascertain what one extra layer buys me.

> I understand that PyTorch is an awesome tool for researchers, but it doesn't necessarily fit into a prod environment.

Some of the largest deployments of ML are using PyTorch models, e.g. OpenAI, Meta, Microsoft.

Reply on news.ycombinator.com