Hacker News Reader

Comment by forgotmyhnacc | Hacker News Reader

What are you looking for in deployment friendliness? There's TorchScript to run your code faster (which is a work in progress)

View on HN

For me it's deploying to mobile, mostly. There's ONNX but it doesn't seem to be terribly mature and it doesn't support some of the common ops, and e.g. FB's own Caffe2 doesn't run it natively. There's also no mature tooling to produce quantized models. TF remains the only real option to do quantization aware training or even easy post-training quantization.

Specifically, my life would be a lot easier if I could save a mobilenet-style model to e.g. ONNX or some other static graph format that does not require model code in order to load weights. I would like then to be able to load this saved model directly into something on Android and iOS that can use GPU and DSP present on the chip, with minimal extra futzing.

Seriously, I've been waiting for a long time now for this to come about. This would make pytorch a much more powerful platform.

One of the major benefits of TF 2.0 is apparently the capability to quickly deploy to TPU units with a single parameter change. (I haven't tried it, just followed the marketing).

AFAIK, This is still being worked on PyTorch via XLA, but not quite there yet.

I've found that with TF in general you can only go "quickly" if everything works. If anything is busted you're more or less screwed because it's so opaque. In contrast, PyTorch lets you inspect whatever you want by setting a pdb breakpoint, and when it gives you errors you most of the time can figure out what's wrong without debugging. The importance of this cannot be overstated.

Reply on news.ycombinator.com