There seems to be serious talk about reimplementing the Torchscript core in Julia (i.e., reimplement the C++ part in Julia), something that makes sense to me given the excellent support there already is in Julia for autodiff. Cf. e.g., https://dev-discuss.pytorch.org/t/where-we-are-headed-and-wh...
If these suggestions are credible and don't come from the Facebook team, that might sidestep your concerns.