But then can't the case be made that narrower and stricter-defined use cases are better served by more conventional ML? If/Wherever efficiency is a concern, that is.
LLMs will take poorly defined data and "potentially" create well defined output.
So ideal systems will likely be constructed with a LLM on one end and a ML on the other and some how, if you can feedback poor data from the ML back into the LLM to clean up the data, you have a magic layer that doesn't care as much about the structure of the data.
That's a lot of supposition, but that's the difference I see between what we can do with LLMs and how ML models are structured. They might be classified as doing the same thing, but their data inputs are vastly different.