There were some interesting talks on neural differentiation applied to physics at ICLR. You probably saw: https://arxiv.org/abs/1906.01563

Very fun!