But this is not what happened. Instead, some guys told AI agents to explore in the way that the guys think that animals explore. "Stanford researchers invented the “curious replay” training method based on studying mice to help AI agents"
That arxiv stuff looks perfectly normal but I kind of hate how it got more and more caricatured as it went through the university press office and hacker news clickbait pipeline.
I've been wondering for a while at what the next steps in adding 'inefficiencies' to AI processing would look like, commenting the other day to a friend that what's needed in the next 18 months is getting AI to be able to replicate the Eureka moments in the shower where latent information is reconstructed in parallel to processing tangential topics.
Going from "attention is all you need" to "attention and curiosity is what you need" seems like a great next step!
I've been playing with this part specifically and it's really amazing stuff.
Having the model concurrently model internal monologue and output for a task, but allowing the internal monologue to be as focused or unfocused as the model sees fit.
You end up with situations where you have it working on a naming task for example, and the model starts imagining the warmth of a coffee cup on the desk, or traffic building up outside for a future appointment with a non-existent person, and then returns back to the task at hand with non-obvious tangents that it'd probably never have uncovered if it was only predicting on tokens related to the original goal of naming something.
It gets even more interesting when you inject variability into the process via the API (for example, telling it to use certain letters pulled from an RNG inside the next iteration of internal monologue).
As not an RL person (I'm in generative), have people not re-increased the exploration variable after the model has been initially trained? It seems natural to vary that ee trade-off.
Looks like you shadowbanned this account. Maybe for posting a URL in the first comment.
Something, something, The Bitter Lesson.
As far as simply differing, much of the time there’s a character limit that’s hit. I’ve seen many posts with comment from the poster calling out their edit to the title and the character limit is usually cited.
It would be especially difficult to keep the character limit (I think there are legitimate design reasons for this) while also requiring that the title matches the submission as closely as possible. Who decides what words are omitted without it potentially being any of: patronizing, inaccurate, or misleading?