The near future of AI is action-driven
jmcdonnell.substack.com
jmcdonnell.substack.com
Transformers seem incredibly good at picking up patterns and repeating them, and combining them in new and surprising ways, but my intuition says that the unpredictability of the real world will cause the required training set to grow faster than the models can keep up, at least to get close to anything that “looks a lot like AGI.”
1. Use some sort of instruction tuning to get the thing "good enough" that it gives decent results 75% of the time and the other 25% a human has to take over. 2. Use the actual usage data as training input. Punish bad behaviors and show the model what the human did to solve the problem. 3. Use this training loop to progressively have the model take over a larger % of the time.
…and I think if you can't get (1) good enough to be worth using it's going to be really hard to get the loop going.
How does the model know when a human has to take over?
I think most extrapolations of current "AI" capabilities into future capabilities are fun and useful in some ways, but also doomed to fail. It's very easy to miss a tiny detail which may in practice be a fundamental problem.
> Use the actual usage data as training input.
Given that those bigger state-of-the-art models train on terabytes of data, how would you know how much training data to generate to sufficiently change the output?
My understanding of "AI" is that it's mostly about some very complex models which are capable of solving previously unsolvable problems. However, those problems are always extremely specific. Going the other way of thinking of problems or future possibilities first and then applying "AI" to it is likely to fail.
For example, image generators are watched for NSFW content by a separate AI critic.
It’s incredibly easy, you ask “did this answer solve your issue?” and add a max_tries.
> … how do you know how much training data to generate …?
You don’t, you keep doing it until the results improve to meet your goals, or they stop short and you switch tactics.
That’s why they have a fleet (parallelization) and why they outfitted the cars with sensors before self-driving was a thing (so they could simulate decision-making and have it corrected by driver action).
Their customers’ feedback absolutely trained their models.
The challenge is when AI has to interpret questions about stuff which can be expressed in syntactically similar ways with very different or precisely opposite meanings so it's very confidently (and plausibly) wrong about stuff like price changes and tax, event timings, refunds etc.
There's no recourse to manual adjustment of system behavior by the user. Research articles should insist on including the user in the loop from the beginning.
The ones in business use absolutely are.
In my experience, that’s being incredibly generous. We evaluated the ‘big names’ in AI chatbots for a project recently, and their biggest portfolio examples (not on their site; ones we went through with their consultants) trip up over depressingly trivial but relevant for the business questions. It might have been a surprise for the chatbot but, for instance, at one point the question asked was in the faq of the company but by the way we formulated it, it didn’t understand it was the same thing. FAQs are made by people who understand the business, a question is distilled from many questions that lead to the same answer; regular users won’t ask the question exactly or at all like that, which is where the AI/nlp is supposed to help. In my professional and personal experience it often does not.
Anecdotally, no one I personally know has ever been helped by a chatbot, but then again, no one I know would click on chat unless all other options (searching google) are exhausted.
Taking a few steps back to your automated phone menu; most systems that I call now use speech recognition to, for instance, have you tell it your account number, shipping code, order number etc. Even this never works for me; it has never understood what I read to it. And these are many different systems. Now sure; this might be because I talk funny or something (although my wife has the same experience and she was a radio and tv journalist), however, the human that comes after that has no issues at all understanding me reading the same information to them.
I treat chatbot like stalebot on bug reports list: as someone wishing to tell "go away, you are not wanted here" but not having enough honesty to do it openly.
I think we have a long long way to go.
This is very much the case. Experimenting with recent large models shows them to be undoubtedly powerful, but still easily distinguishable from intelligence. They regularly contradict themselves and spout unlikely narratives. Asking relatively simple questions often show a lack of understanding.
That being said, it would not surprise me if the chatbots are nevertheless worse than recent large language models.
For some example successes and (often amusing) failures, see https://cs.nyu.edu/~davise/papers/GPT3CompleteTests.html
For what it's worth, those examples are illustrative, but the best way to get a feel with them (how how handpicked examples may or may not be) is experimenting yourself with prompts.
Maybe not AGI, but perhaps we have already attained AGP (artificial general politician)
Sounds like lots of people I've met.
Everyone working on self driving cars is just patching infinity.
I have reached the conclusion that this must be on purpose for many companies. Eg. when I call the airline about a refund, and none of the options presented are fit for that request.
This is much more about the airline doing customer harassment to earn an extra bug than the system being broke.
After adding a delay function it could even do things like blink the lights when requested, and after making the video I added a dimming option it was also able to use.
This _has_ to be the most complicated way ever invented of blinking an LED. :-)
(But, seriously, kudos. Sounds like a fun experiment.)
> Output: How many machine learning engineers does it take to turn on a light bulb? Just one, but it takes a lot of training data.
Doing something is not hard. Accounting for all the possible problems and making sure they don’t occur is the hard part because it requires a level of control that goes beyond what looks like action driven gives.
I'm not interested in business viability. I want something that's quirky, unpredictable, and human-like. I want an agent that runs in its own game-loop. I don't want another goddamn web service.
We still don't have self driving cars, right? Except maybe for a few labs, suburbs, and highways.
Yes we do. They exist and are on the road. Just look at Tesla, Waymo, etc. These cars are able to be given a destination and they will drive themselves there. There are limitations, but that doesn't make them not self driving.
Just like self driving cars.
So it might be something like using the prompt to generate an embedding that captures spatial-temporal aspects of a prompt. Or really somehow the whole prompt encoding should terminate at some type of spatial-temporal tokens maybe.
I don't know exactly but I think that text-to-video and video-to-text is going to lead in a more general purpose direction than just language alone.
You can even use a robot to manipulate things in real life to create synthetic data for a neural net.
But also I think that what an AI that can for example really answer questions about a video would need to do to be really effective would be to basically do compressed versions of those simulations using the spatial-temporal-abstract latent space. Which should be a better model than just the textual space.
Is my goto
@nearcyan @arankomatsuzaki @ethanCaballero @_albertgu @lcastricato @tszzl @gwern @fchollet @drmichaellevin @Plinz @sama @sharifshameem @woj_zaremba @ilyasut @karpathy @RichardSSutton @RiversHaveWings @ericjang11 @_akhaliq @hardmaru
note: some, like Sutton and Sutskever, aren't terribly active, but they're worth following anyways since it signals to the algorithm which sorts of tweets to suggest
The article is specifically calling out that current models suck at this, and action driven models would proactively determine what additional input is needed seek out data to augment their input to make decisions.
None of the Twitter accounts you’ve linked to seem to be doing this.
Ie. specifically prompt -> actions to collect additional information as determined from prompt -> internal reprompt with additional input -> response.
To quote the article:
> This is not 10 year tech. It may be possible right now with off-the-shelf tools. But to make it work we need to set up the right feedback loops.
The right feedback loops. How do we get those?
I have seen exactly zero examples of people successfully doing this so far; the only working examples are prompt-reponse/action which are not, remotely, AGI like.
That’s the point they’re making: something that internally reprompts itself is quite AGI like.
Something that does not is a bot.
It seems plausible we might get this kind of adaptive agent sooner than AGI
It certainly doesn’t seem to exist right now, and the path to “self reprompting” feedback loops is quite unclear.
The first was the Microsoft CTO discussing it here - https://www.youtube.com/watch?v=LYBcH3JjLxs&t=529s
The second was the former head of Tesla Ai division on Lex Friendman
Now this article
This is quite telling of how powerful these models will be, even on launch. The Microsoft CTO mentioning it in particular is interesting since he's obviously seen internal and alpha results.
By the way I have an intuition that free will is based on this - we need an irreducible source of cause to learn about causality and we call that the free will.
It's impossible in general to be certain of causality for humans too, all we see are correlations in data and we invent causal theories that explain those correlations. That's basically what machine learning does as well: compile sets of correlated values into a compressed representation (the neural net) which arguably qualifies as a "causal theory" from the algorithm's perspective.
I think what's missing is maybe one or two orders of magnitude more compression, which would basically be devising a better/more parsimonious theory. We've been getting progressively better at that over the years too, and combined with how hardware has been scaling, we're seeing exponential growth in effectiveness. This is why some are predicting artificial general intelligence by 2030-2035.
We do experiments - we actually do them a lot and it starts very early with infants experimenting how activating a particular muscle moves his hand etc. Later we do less of original experiments - but repeat those that we already know the results - by walking for example and getting where we planned to.
In science to get from correlations to causation we have 'randomized trials'. In everyday practice we use something called 'free will' in our minds as the source of the needed independence.
Not sure about that. MinutePhysics has a video about how correlation can imply causality: https://www.youtube.com/watch?v=HUti6vGctQM
It is very important to note that to discriminate "what is smart" and "what just looks smart" (but is not) is a fundamental distinction and activity.
Indeed. It shouldn't be surprising though, given we've already been through this with vitalism and how "life" arises from matter that isn't alive.