Large language models think too fast to explore effectively
arxiv.org
arxiv.org
What if we started Refining and reinforcing these Extremely fast thinking, models with text written exclusively by stoners?
Stoners think slower, but they have more wide, branching thoughts. I find I do some of my most seemingly creative and explorative thinking when I am high.
If there are effects of time and season and energy in models of LLMS such that you can say it’s June and it performs worse or it’s January and it performs better. Or you can say you’ve got a good nights rest so do better and it does, due to the fact that human writing is affected by our mental state, and our mental state is affected by physiological factors.
So then, I wonder what happens if you do HFRL and refinement on text from people under the influence of marijuana.
... Maybe stoners know something you don't. Maybe part of you sees that, and seethes.
I think some people, even in younger generations, don't know what 'chill' really means. They see it as laziness, because the idea of not 'producing value' every waking moment actually scares them. That fear is the driving force of their lives. They don't have the imagination/empathy/experience to see past it; and judge people (including themselves!) through that lens.
It's deeply unfulfilling, I'm sure, and not just for boomers.
But there's also other kinds of value, ever had something sentimental that you couldn't pay someone to take? Things like family photos are practically valueless on a spreadsheet level but immensely valuable on an emotional level. How do you price having your family and friends around you as you pass?
You can still produce value without producing economic value - this was supposed to be the idea behind charities.
a bigger house and being a stoner are both fairly unfulfilling. a wife and kids are the most meaningful and joyful things a man can have. i think comparing those to a house and suggesting they're unimportant, or no more important than getting stoned, is a great example of why so many of us dislike stoners.
They can be. They're not always. How many deeply unhappy people are in awful marriages? By the fact that 40-50% of all marriages end in divorce, I'd say it's 50/50 at best. How many people should never have been parents?
And also, that's like, your opinion man. There are other views - perfectly valid - on what is most meaningful and joyful. How many childless unmarried women (and men) find great meaning and joy in other activities, but feel pressured to procreate by a segment of society which sees their worth solely in homemaking?
> marriage and kids are crazy things to stack up against house size
I think so too. Yet many people value themselves by how big their house is... Not too common a trait among stoners, I might add.
> i think comparing those to a house and suggesting they're unimportant, or no more important than getting stoned,
I don't think I did that by any fair reading of the comment.
> is a great example of why so many of us dislike stoners.
Stoners tend to think outside of societal norms a bit more often, seeking a broader perspective. They also tend to reflect internally on what actually makes them happy. They might end up thinking, 'wow, I don't think marriage is a big goal in life for me'. Or, 'like, who is a priest to tell me my relationship is valid?', or 'Jesus, is it actually kinda selfish to want more kids on a planet with 8 billion people maxing out it's resources?'.
I think those are valid and important questions + perspectives; and that it's kinda mindless to shit on people for exploring things for themselves.
i don't think you need marijuana to reflect on life and societal norms. in fact, a lot of teenagers spend some years doing this. we all have questioning phases where we read a lot of philosophy and argue with adults and wonder whether the way things are is the way things ought to be. most people don't need marijuana to do it, and most people who use it aren't doing so in the pursuit of greater knowledge unconstrained by the defaults we've passed down through the ages.
The brain can adapt and use the thc to its advantage!
> Representational analysis of the models with Sparse Autoencoders revealed that uncertainty and choices are represented at earlier transformer blocks, while empowerment values are processed later
My read is that it’s not so much the content of the training input, but how the models are designed and trained.
One day the stoners witness the birth of AI but forget which system they were logged into. And they're presenting it to the CEO on Monday morning. Dude, where's my LLM?
system 1 thinking for early layer processing of uncertainty in LLMs. quick, intuitive decisions, focuses on uncertainty, happens in early transformer layers.
system 2 thinking for later layer processing of empowerment (selecting elements that maximize future possibilities). strategic, deliberate evaluation, considering long-term possibilities, happens in later layers.
system 1 = 4o/llama 3.1
system 1 + system 2 = o1/r1 reasoning models
empowerment calculation seems possibly oversimplified - assumes a static value for elements over a dynamic context-dependent empowerment
interesting that higher temperatures improved performance slightly for system 1 models although they still made decisions before empowerment information could influence them
edit: removed the word "novel". The paper shows early-layer processing of uncertainty vs later-layer processing of empowerment.
Wouldn't it be the exact opposite? That is, novel stimuli requires more extensive processing at higher levels of cognition.
It is a lot of work to retrain your lizard brain to not react in unpleasant ways when the shit hits the fan or you’re exhausted, and the default answer to everything is “no”. There are ways I show up in a crisis that are exemplary, and others that I’m not proud of. Understanding why doesn’t fix it. It’s just step 1, like in AA.
And I know some neurodiverse people who absolutely believe this is why they are like this. And that’s always a problem with studies. Some things are 10% of the time, it works every time. If you test the general public for mental acuity and caffeine you’re going to see undiagnosed ADHD people benefitting, or find they’re already consuming alarming quantities of the stuff (self medication).
He divided reasoning into the two categories corollarial and theorematic.
Peirce’s Deductive Logic: https://plato.stanford.edu/entries/peirce-logic/
Scientific Method > 2. Historical Review: Aristotle to Mill https://plato.stanford.edu/entries/scientific-method/#HisRev...
Scientific Method: https://en.wikipedia.org/wiki/Scientific_method
Reproducibility: https://en.wikipedia.org/wiki/Reproducibility
Replication crisis: https://en.wikipedia.org/wiki/Replication_crisis
TFS > Replication crisis https://en.wikipedia.org/wiki/Thinking,_Fast_and_Slow
"Language models can explain neurons in language models" https://news.ycombinator.com/item?id=35879007 :
Lateralization of brain function: https://en.wikipedia.org/wiki/Lateralization_of_brain_functi...
This is a lot older than Kahneman. It's the classical split between rationalists and empiricists.
AI maximalists and x-risk death cult fanatics are effectively being rationalists. They don't believe a sufficiently intelligent entity will be bound by the need to do science. It can, by sheer force of thought, figure out the answer to any question, develop a perfect course of action to achieve any goal, as quickly as electricity can traverse its own internal circuits.
It's interesting to see the constraints on how they can even study this in the paper, though. You can only ever ask a computing agent to do what it even can do. If you want it to explore, it needs a toy world that maybe digitally encodes a reasonable stripped-down representation of the real world, but without a body, sensors, and full autonomy, it can't ever truly explore the real real world.
Would you believe that computers are made out of atoms, and that computer programs modify those atoms? Via electrons, I believe.
Results show most LLMs underperform compared to humans, except for the o1 model
Really ? Can they ask pertinent questions ?
What I mean by this, is the process of coming up with novel ideas a single capability that has to be trained and reinforced.
Or is it a ladder of capabilities of increasing complexity in that a model that could figure of General Relativity from scratch would not be able to continue the process and perhaps come up with a viable “theory of everything.”
One thing I’ve wanted to do, I’m sure somebody has tried it, is build a dataset to RL a model to be more creative: Get a human expert in a field, have them ask a reasoning model some open questions, and then have the expert look at 20 outputs and rank them by creativity / insight. Have the expert iterate and see how much new “insight” they can mine from the model.
Do this across many fields, and then train a model on these rankings.
Perhaps creativity is a different way of moving in latent space which is “ablated” from existing models because they’re tuned to be “correct” rather than “creative.”
Also curious what techniques there are to sample a reasoning model to deliberately perturb its internal state into more creative realms. Though these a fine line between insight and hallucination.
In some respects creativity is hallucination. As a human, you’re effectively internally postulating creative ideas “hallucinations” and then one of them “hits” and fires a whole bunch of neurons which indicate: “ok that wild idea actually has grounding and strong connections to the existing knowledge in your brain.”
Creativity doesn't come from the brain itself, the brain is just exploring and accumulating experience. Experience builds on itself. The role of the environment is both to spark and to invalidate ideas.
For example AlphaZero with just search-and-learn strategy could beat humans at our own game. It's not a magic of the model, but of the search loop.