This morning I feel oddly compelled to play the fool so here are some near/medium term thoughts on where this may be going (worth less than what you paid for them):
1. The most important ChatGPT plugin is going to end up being the one that invokes itself recursively. The autoregression approach seems to be severely limiting what these models can do by limiting their ability to think without speaking. Although a few months ago I thought the obvious way to fix this was to train the model to emit special sort of "bracket" tokens that would be deleted by the driver once the inner thought completed, leaving only a sort of "result" section, the GPT-as-a-GPT-plugin effectively does the same thing
2. Whilst the first biggest win from the plugin will be "sub-thoughts", the next biggest will be training it how to dispatch multiple sub-thoughts in parallel. GPT already knows how to break a complex problem down into steps, but is still constrained by context window size and inference speed. Once it is taught how to split a problem up such that multiple independent inference sessions are able to work on it in parallel, it'll become feasible to make requests like "Build me a video game from scratch using Unreal Engine, set in the world of Harry Potter, about the adventures of a character named X" etc and it'll end up dispatching a massive tree of GPT sub-instances which end up working on the independent parts like character generation, writing the Unreal C++, prompting Midjourney and so on.
Parallel recursive LLMs are going to be much more awesome than current LLMs, and I mean that in both senses of the word (cool, awe-inspiring). In particular, this will allow us to pose questions like "How can we cure cancer?".
3. OpenAI need a desktop app, pronto. Whilst the cloud model can take you some way, the most valuable data is locked behind authentication screens. The cloud approach faces difficult institutional barriers, because data access inside organizations is oriented around granting permissions to individuals even when they work in teams. Giving a superbrain superuser access doesn't fit well with that, because there's no robust method to stop the AI immediately blabbing business secrets or PII to whoever tickles it in the right way. That's one reason why the current wave of AI startups is focused on open source technical docs and things. If ChatGPT is given tool access via a desktop app running on the end user's computer, it can access data using the same authentication tokens issued to individuals. This also neatly solves the question of who is accountability for mistakes: it's the user who runs the app.
4. Sandbox engineering is the new black.