Hacker News Reader

Comment by fooblaster | Hacker News Reader

I am so confused by metas ecosystem. Perhaps others have the same issues. I have mountains of torchscript code. It worked fine for me - had no issues making the python compatible. Torchscript is now deprecated, and the ostensible replacement is torch.export and either: AOTInductor or executorch. torch.export is so limited - no control flow at runtime at all, less support of python than torchscript. It is far more work to hoist all the control flow out of the model than it ever was to make the model torchscript compatible. Feel like meta has moved on, but I'm still stuck in the past here.

View on HN

Yeah, for a lot of users who control the exported source code, rewriting model to use control flow ops, or simply removing the control flow code is a viable option and solvable. For some other users who want to export the model as-is, the option is either using the (deprecated) torchscript, or just move on and use torch.compile and run your model in Python.

Those control flow ops aren't even supported on many backends. I know tensor rt doesn't support them for example, at least today.

Removing control flow isn't as easy as you'd think for some. It essentially means ripping large sections out of python and into separately implemented c++.

it's quite the bummer. some models you simply can't export with dynamo. for the time being the jit exporter is the only good option.

in particular selective function scripting is essential!

ExecuTorch developer here, agreed it's a huge pain to deal with if conditions right now. Part of the pain comes from the vast expressiveness of python on if condition, which causes all ML compiler a lot of headache to be able to capture a sound graph. The rest of the pain comes from the strict requirement of torch.compile itself (no mutation/aliasing behavior in the if branches), which in often times makes torch.cond hard to use or inefficient.

And you wouldn't happen to know about a torchscript replacement that is currently in-flight that is not based on export?

So what are your users doing to get around this? Hoisting all control flow out?

Anyway, perhaps we can chat in the executorch discord.

Reply on news.ycombinator.com