Software trumping hardware. Question is how generalizable is that approach.
Wouldn't say it's that simple. The biggest supercomputers need to have a balance between fat-ass manycore for the MPI/numerics (which won't go away until physics problems can be solved by DL--seems unlikely); GPUs for training, and probably something ultra low-power that can take advantage of power-in-numbers across thousands of nodes (FPGAs potentially). This is still a co-design effort, but like other areas that have been automated out of existence, it might be our best and brightest (supposedly) at the top of computing creating and maintaining these scientific applications should watch their futures.