I want to believe that there's some world where developers start putting actual craftsmanship into their software (again?), but I think this will only be true for SaaS or related environments where the developer is also the one spending money on the compute. Everyone else will probably outsource the optimization effort to a mixed hardware-software vendor that amortizes the cost (like hyperscale cloud providers) via middleware, instead of just amortizing the cost via hardware.

People (including on HN!) like to say that cycles are cheap but developer time isn't. There's a reason for this that will be invariant of the actual costs: it's the user that pays for the hardware and the developer that pays for development time. End-users are willing to spend money on hardware, but not software, and this has remained true even as companies like Apple and NVIDIA have blurred the boundary. In the current environment of $0 software, how do you (as a developer) fund more efficient software? I think the likely answer will be that developers will happily lock themselves to whatever vendor offers to solve this problem for them. We've seen this in ML already.