Some earthquake researchers I know have started using neural net approximations of PDEs for doing viscoelastic calculations [1] (simulating how the Earth's crust and upper mantle responds during and after an earthquake). The have found pretty huge speedups (5000% on CPU - 50,000% on GPU) using neural nets with good accuracy, but I am not sure that the PDE meshing was super highly optimized.

[1]: https://agupubs.onlinelibrary.wiley.com/doi/abs/10.1002/2017...

edit: Whether this NN approximations will enable us going well beyond what PDE solvers can do is a tough question, especially because PDE solvers are generally needed to build the training sets as far as I understand (i.e. they are trained on synthetic, not observational, data). It's possible that one could strategically and judiciously train smaller components of a NN system and then assemble the system to do things that the PDE solvers couldn't do, perhaps, in the way that one can often write unit tests for scientific software but not really test the final results directly (because there is nothing to test against). However I think that in general the lack of training data for the more complex system will always be a major obstacle, whereas it's not really for PDEs.