The token cost stopped decaying as expected, as mentioned by the original 100,000k post on HN, and the move nowadays is towards more context to keep building functionality. The cost is just going to go up for inference. These companies might be better off splitting their focus between training and tooling, and canning all the capex/opex associated with inferrence.
Forget S/W engineers for a moment ... Every white collar worker I know, especially non technical folks, use ChatGPT all the time, and believe that is AI at this point. That demand isn't going to vanish overnight.
The counter argument is usually "They'll sell data", but I'm not sure you can double the number of trillion dollar data companies without some dilution of the market, and reach a billion devices / users without nation-state level infra.