There is evidence that the performance of the models scale linearly with size, so moores law scaling is likely to get us some “free” improvement even if no one ever invents a better ML technique.
If you are noticing that this seems to fundamentally limit model performance on certain tasks to aggregate human capability, you are noticing correctly.
To give you some idea of what these benchmarks look like, here’s the prompt list from DrawBench which Google created as part of training their Imagen model.
https://docs.google.com/spreadsheets/u/0/d/1y7nAbmR4FREi6npB...
I’ll hold off declaring it dead till it is well and truly dead. And even then we could expect cost improvements as the great wheel of investment into the next node would no longer need to turn and the last node would become a final standard.
As to physical limits, there are plenty of weird quantum particle effects to explore so that seems overstated. We are still just flipping on and off electromagnetic charge. Haven’t even gotten to the quarks yet!
The classical Moore's law formulation has been dead for 15 years already. What we have now is whataboutism about why it still holds.
https://www.researchgate.net/figure/Density-of-logic-transis...