Add an interpreter as per this article, and we're basically there.
That he doesn't expect people to be doing what we call "prompt engineering" in 5 years. You'll simply ask for what you want and if there is ambiguity it can be sorted out interactively or it will just do the 'right' or obvious thing that we would expect.
Not to mention that there's still a large portion of the general public that just freezes up when it comes to anything technological. I've provided support to a number of different communities and even educated people can't do this type of thinking easily.
It's possible that we could eventually get to that point, but it would require some massive educational efforts and a culture shift, which would also require substantial investment without a clear road to profit, and I believe America lacks the political will or even ability to implement anything like this. Since we can't act in unison, it will be down to companies, and for them it makes more sense to solve the problem by hiring prompt engineers with the savings they get from cutting elsewhere instead of training all their existing teams on how to learn new ways of thinking.
tl;dr: Technologically he's probably right, but he's massively overestimating the public's ability to use interactive tools to get information.
> obvious thing that we would expect
Any time you rely on something being 'obvious', the public is going to somehow fuck it up.
This is also what programmers typically complain about when they say their PM isn't "technical" enough. What they usually mean is they don't understand the details of the business / domain well enough to meaningfully contribute to the organization / break up of the work.
To that end, I'd expect these language models to eventually improve at that task too, perhaps responding with choices or diagrams to help users learn how to ask what they actually want. That's in a sense what a prompt engineer is doing in this context. In that sense I think I agree w/ your assessment although I also think there's a lot of room for these AI's to handle that task, at least partially, as they improve.
The issue with language models in particular is that to use them effectively you have to fundamentally understand that you're translating human-brain-language into computer-language and understand that those are in fact two different things even if their delivery happens identically. For most coders, this jump happens semi-instinctively because they know how to test something to verify 'ah yes, this follows computer logic'. The general public thinks computers are wizardry. The other group of people who will be able to understand this are those who have thought deeply about language for whatever reason and therefore know that human-brain-language has qualities that extend beyond 'We use words'. This is also very hard to impart to novices; I have a Linguistics degree and studied a cumulative 10 years of 6 languages so I went through that knowledge transfer as well. They're both hard and one or the other is necessary to interface with language models effectively and will be for likely the next 10ish years just because providing a good technical solution will require the technical people to understand much more about the social side of communication and that will also take time.
Does it have something to do with grounding or implicit knowledge/ connotations and context?
An example of things I consider when interfacing with an LLM that derive from my linguistic knowledge:
* That the language it's trained on is thoroughly modern/confined to the speech of a particular place in time. Which means that I know any viewpoint I receive is not only modern but from a very specific time and place. The language that everybody uses is sort of like the water fish swim in in that most people don't think about it. Which means I know that if I ask it something about (to use an example that is a culture war issue) the history of racism, I know that the answer is being run through modern conceptions of racism and if I want historical views, I need to get those elsewhere.
* That which words are most commonly used relies on the social and economic status of the speaker as well as word properties like phonetics and phonology. This makes it much easier to pick and choose which vocabulary and sentence structures to use in order to 'sub-pick' the part of the training set you want answers to. Asking 'how to grow peppers' and 'what soil variables control how well a Capsicum annuum plant grows' are going to get you different answers.
* Related to this, the differences between spoken and written English on a large scale - one problem with the 'everybody can just use LLMs' idea is that the LLMs are trained on written English but the majority of people interfacing with them speak to them as though it were a verbal conversation. There are parts of verbal communication that don't exist in written communication, and knowing how to provide scaffolding to make a request make sense requires understanding the difference between the two.
* A basic knowledge of sociolinguistics is fantastically helpful for developing personae and/or picking out biases in the training data set. (Somewhat similar to how I can usually ID an American's media diet and political affiliation after a 5-10 minute casual conversation).
I think you are underestimating the pace of progress in the capability of the LLMs. It has infinite patience to query you for missing facts. Because it is not about the public's ability to use an interactive tool. It is about the LLM's capability to be a good focus group moderator/librarian/etc.
This role reversal is imminent from what it seems. The LLMs will be using us, rather than us using them.
LLMs are confidently wrong rather than inquisitive/have an interest in being right. They theoretically have infinite patience, but they really have no patience and accept input at face value.
I don't doubt that these can be improved, but I also don't doubt that some people will be better at interacting with LLMs than others, and that to get good at using an LLM will take onboarding time that organizations would prefer to centralize
Right now, you can't ask chatgpt to "think of a person, and the user will guess who it is". It can decide to think of something, but has nowhere to put that information. It should be put into the scratchpad.
Scratchpad could potentially be used for solving problems step by step too, much like how we consciously think through problems step by step. If it gets too complicated for us, the mental scratchpad we use is augmented by a physical notepad or other system.
You could also say that it enables an internal monologue.
Obviously LLM models cannot be compared one to one with our brains, but one thing is clear: despite them being executed by a computer, they are not the computer that is executing them and as such their ability to do operations that are trivial for the computer is not necessarily trivial for the language model.
Like whole industries sprung up around the various tools - email, search, notes, images, videos, databases. We're big on tools.
Yeah, pretty big on tools.
It's tools all the way down.