Which they are currently doing. GPT-4 can take visual input.
I totally agree that humans are far more complex than that, but just extend your timeline further and you’ll start to see how the gap in complexity / input variety will narrow.
They will not be LLMs then, though. But some other iteration of AI. Interfacing current LLMs with APIs does not solve the fundamental issue, as it is still just language they are based on and use.
Multi-modal LLMs are still called LLMs because they don't "interface with APIs" to add visual, audio, touch, etc input and output. They just encode pictures, sounds, and motor senses using the same tokens they encode text with and then feed it to the same unmodified LLM and it learns to handle those types of data just fine.
There are no APIs involved and the model is unchanged. It was designed as an LLM, the design hasn't changed, it still is an LLM, it's just had data fed to it that it can't tell from text and is running the same exact LLM inference process on it.
I can download any open source LLM right now and fine tune it on images faster than I could train an ImageNet from scratch because of something called transfer learning. Humans transfer learned speech after millions of generations of using other senses. That's not at all surprising or different from the way LLMs work.
These things are already here; it's just a matter of when they get out of the research labs... Which is happening fast.
Yes, ultimately it does imply that. Probably not the current iteration of the technology, but I believe that there will one day be AIs that will close the loop so to speak.
It will require interacting with the world not just because someone gave them a command and a limited set of inputs, but because they decide to take action based on their own experience and goals.