But our software could be orders of magnitude more efficient. Am I wrong?
Slowing transistor scaling just gives us one less domain through which to depend on for improvements - the others are all still valid, and will probably be something we come to invest more effort into.
Where as tailored hardware, software improvements are unlikely to continue yielding such payoff again and again.
So the argument that cost of improvements will go up is not wrong. And maybe improvements will be more linear than exponential.
We also don't know that current semi tech stack is the best. But it's fair to argue that the cost of moving off a local optimum to a completely different technology stack would be wild.