O3-mini simulated scikit calculations
emsi.me
emsi.me
Hmm, I need to run some code. I'm thinking I can use Python, right? There’s this Python tool I can simulate in my environment since I can’t actually execute the code. I’ll run a TfidfVectorizer code snippet to compute some outcomes.
This gave me a thought on the future of consumer-facing LLMs though. I was speaking to my nephew about his iPhone, he hadn't really considered that it was "just" a battery, a screen, some chips, a motor, etc.. all in a nice casing. To him, it was a magic phone!
Technical users will understand LLMs are "just" next token predictors that can output structured content to interface with tools all wrapped in a nice UI. To most people they will become magic. (I already watched a video where someone tried to tell the LLM to "forget" some info...)
110 IQ: LLMs are "just" next token predictors that can output structured content to interface with tools all wrapped in a nice UI
140 IQ: LLMs are magic
That's the most interesting part of what we're learning now, I think. So many people refused to accept that for any number of reasons -- religious, philosophical, metaphysical, personal -- and now they have no choice.
The “magic” is that yes, LLMs are “just” statistical next token predictors.
And as code only, LLMs produce garbage.
When you feed them human cultural-linguistic data, they “magically” can communicate useful ideas, reason, maintain an internal world state, and use tools.
The llm architecture is just a mechanism for imprinting and representing human cultural data. Human cultural data is the “magic”, somehow embodying the ability to reason, maintain state, use tools, and communicate.
Learning how to represent language data in vector-space allowed us to actually encode the meaning embedded in cultural data, since written language is just a shorthand.
Actually representing meaning allows us to run culture as code. Transformer boxes are a target for that code.
The magic is human culture.
Culture matters. We should be curating our culture.
It doesn't always use the tool, but it can: https://chatgpt.com/share/67bcb3cb-1024-800b-8b7e-31335c6347...
In that case you're using GPT-4o which we know has access to Code Interpreter. The annoying thing here is that o3-mini still doesn't.
I really hope they don't ever change that UI pattern, this stuff is hard enough to understand already.
If you really want to test this, you can take advantage of the fact that Code Interpeter runs in a persistent sandbox VM. Tell the o3-mini prompt to save a file, then switch to GPT-4o (which can use Code Interpreter for real) and have it run Python code to show if that file exists or not.
> An 6061-T6 aluminum alloy hollow round 2 in diameter beam with 0.125 in thickness and length 120 in is simply supported at each end. A point load of 100 lb is applied at the middle. What is the deflect in the middle and 12 in from the ends.