Same with NS equations. Who cares if you can find a singularity.
And if you want an example of something novel that is worth pursuing - Its highly likely that the modern transformer architecture is sub optimal, you probably don't need to do full matrix multiplies in the transformers. There potentially could be a higher level mathematical formulation of minimal math operations that are needed without having to do trial and error - especially because all of the math involves linear combination passed through smooth activation functions.
But coincidentally, there hasn't been any research in terms of point LLMS to self optimize in this way, because there isn't enough human math literature on the LLMs to train on.