The thing is, at a certain point, economics is a bitch. Either efficiency improves or those uses stop. I’d note that inference is much cheaper than training and has become radically more efficient rapidly. There’s good reason to believe the current wild eyed training boom we see now will collapse on itself in terms of cost to benefit, or will curtail to the economic value.
However you need to understand that what you said is wrong. Many of these things can’t be solved with a simple query because a query returns a blob of data and the human has to search for the information in a set of documents possibly containing it.
The UX of “I have a question” and the answer of “ok here’s a likely direct answer that’s comprehensive and comprehensible to which you can ask any follow up” in unbeatable. It’s what AskJeaves.com was as one of the first search engines but couldn’t achieve because NLP was so shitty. The fact it might hallucinate - which is very unlikely in a frontier model on a pedestrian question about common knowledge that can likely be answered by a search query since it’s training set likely contains a lot of examples of the answer and it can supplement with a basic RAG against a traditional search engine - is not even that relevant because unless you’ve not used modern search engines recently it’s almost impossible to get a straight answer from search due to SEO.
So the reality to your rhetorical question is “every single one of those questions only an LLM could have solved.” Because they’re the first real NLP system we’ve invented that can give coherent answers to the question.
Very few search engine queries are seeking documents. Most are seeking an answer to a question. A search engine has been the best we could do until a few years ago to answer questions and the user experience is fairly shit and rapidly deteriorating due to economics. It’s like asking a professor a question and being answered with a bibliography but over time realizing the professor is being paid under the table to stuff the bibliography with advertisements and the rest of the bibliography was replaced by other ads pretending to be source material and the professor just looked at the title to select them. Not only is the answer not an answer - it’s a list of stuff to read which might contain the answer if you read it all - it’s increasingly unlikely the answer is even there at all. You can argue “but the professor answering with a bibliography ensures you learn a lot!” Which is fair but people usually just want a direct answer and would sooner go to the cool professor down the hall who answers your question and gives the bibliography as citations (as modern frontier LLMs like ChatGPT and Claude do).
The fact the cook professor dabbles in mushrooms too much during office hours is unfortunate and the school administration is working on that. But it’s useful enough people literally don’t care.
For use cases where the cost >> value like collaborative filtering - well, fine. LLMs won’t be used there because the value doesn’t hold. We are only two or three years into this and there’s a lot of just dumb stuff because no one knows what will stick. And LLMs will not be the answer to everything. But I think they’re going to be increasingly more powerful simply because we will defer the right algorithm to the problem and the LLMs will be the glue of language and abductive reasoning that makes something useful, such as information retrieval, usable. And that alone is more valuable than Google’s search engine. And that is not the only use - as I’ve seen directly. So yes LLMs won’t be grand masters at chess beating alpha blue. They don’t have to be. They can be the user interface to Alpha blue and alpha blue becomes more usable immediately and much more valuable.
The hardest problem in CS isn’t cache coherency or naming things or off by one errors it’s making useful things usable, and among all the other useful things LLMs have done, they have absolutely solved the hardest problem of all - making all the useful things usable.