The Einstein AI Model

BlueSky version: https://bsky.app/profile/thomwolf.bsky.social/post/3ljpkl6c6...

---

Quite interesting post that asks the right question about "asking the right questions". Yet one aspect I felt missing (which might automatically solve this) is first-principles-based causal reasoning.

A truly intelligent system — one that reasons from first principles by running its own simulations and physical experiments — would notice if something doesn't align with the "textbook version".

It would recognize when reality deviates from expectations and ask follow-up questions, naturally leading to deeper insights and the right questions - and answers.

Fascinating in this space is the new "Reasoning-Prior" approach (MIT Lab & Harvard), which trains reasoning capabilities learned from the physical world as a foundation for new models (before evening learning about text).

Relevant paper: "General Reasoning Requires Learning to Reason from the Get-go."

I have never heard anyone think this way: “The main mistake people usually make is thinking Newton or Einstein were just scaled-up good students, that a genius comes to life when you linearly extrapolate a top-10% student.

The reason such people are widely lauded as geniuses is precisely because people can’t envision smart students producing paradigm-shifting work as they did.

Yes, people may be talking about AI performance as genius-level but any comparison to these minds is just for marketing purposes.

A nice post (that should be somewhere smarter than contemporary Twitter/X).

> PS: You might be wondering what such a benchmark could look like. Evaluating it could involve testing a model on some recent discovery it should not know yet (a modern equivalent of special relativity) and explore how the model might start asking the right questions on a topic it has no exposure to the answers or conceptual framework of. This is challenging because most models are trained on virtually all human knowledge available today but it seems essential if we want to benchmark these behaviors. Overall this is really an open question and I’ll be happy to hear your insightful thoughts.

Why benchmarks?

A genius (human or AI) could produce novel insights, some of which could practically be tested in the real world.

"We can gene-edit using such-and-such approach" => Go try it.

No sales brochure claims, research paper comparison charts to show incremental improvement, individual KPIs/OKRs to hit, nor promotion packets required.

The author seems to assume that conjuring up a conjecture is the hard part - yet it will be filled with the same standard mathematics ( granted, sometimes wrapped as new tools, and the proof ends up being as important as the result), often at great cost.

Having powerful assistants that allow people to try out crazy mathematical ideas without fear of risking their careers or just having fun with ideas is likely to have an outsized impact anyway I think.

I have a few thoughts after reading this:

- I started to see LLMs as a kind of search engines. I cannot say they are better than traditional search engines. On one hand, they are better at personalizing the answer, on the other hand, they hallucinate a lot.

- There is a different view on how new scientific knowledge is made. It's all about connecting existing dots. Maybe LLMs can assist with this task by helping scientists discover relevant dots to connect. But as the author suggests, this is only part of the job. To find the correct ways to connect the dots, you need to ask the right questions, examine the space of counterfactuals, etc. LLMs can be useful tool, but they are not autonomous scientists (yet).

- As someone developing software on top of LLMs, I am slowly coming to a conclusion that human-in-the-loop approaches seem to work better than fully autonomous agents.

It doesn't seem correct to dismiss the creativity of Move 37 because real originality is "something more fundamental, like inventing the rules of Go itself"

It would seem more fruitful to simply point out that LLMs aren't all of AI, and that excelling at mimicking human-like text production isn't really doing the work that AlphaGo was attempting.

Just because both things might be given as (different) examples of deep reinforcement learning in an AI survey course doesn't mean that we have much reason to believe that the vast investments in LLMs result in AlphaGo like achievements.

>We're currently building very obedient students, not revolutionaries. This is perfect for today’s main goal in the field of creating great assistants and overly compliant helpers. But until we find a way to incentivize them to question their knowledge and propose ideas that potentially go against past training data, they won't give us scientific revolutions yet.

This would definitely be an interesting future. I wonder what it'd do to all of the work in alignment & safety if we started encouraging AIs to go a bit rogue in some domains.

If this take is correct and we need creative B students, we might still get a compressed 21st century with human creative B students working together with AI A students who support the human with research, validation, workshopping ideas, etc.

I think the author has a point. LLMs struggle with what you might call epistemically constructive novelty. It's the ability not just to synthesize existing knowledge, but to identify what's missing and conjecture something to fill the gap and demonstrate it to satisfaction. Out-of-distribution knowledge gaps are typically where LLMs "hallucinate." Unlike highly skilled human researchers, they don't pause and construct the bridge that will get them from known to unknown, they just immediately rush to fill in the blank with whatever sounds most plausible. They need to ask questions that haven't been asked before, or answer ones that haven't been answered. Is this just some missing subroutine that we'll eventually figure out? Or is this conjecture-proving process much more elaborate than whatever existing models, no matter how scaled, can manage? I'm not sure. But the answer starts with a question.

I wonder if people could just write their blogs posts in a short form: claim, argument in favor, counter-argument; consequences (this is optional).

Like this whole blog post could be:

Claim: Current AI is unlikely to usher in an era of dramatically accelerated scientific discovery.

Argument in favor: A genius does not come to life when you linearly extrapolate a top-10% student. Newton or Einstein is not just scaled-up good students. To create an Einstein, we need a system can ask questions nobody else has thought of or dared to ask. One that writes 'What if everyone is wrong about this?' when all textbooks, experts, and common knowledge suggest otherwise.

Existing benchmarks don't test such skills. And existing systems are likely hopelessly far from this capability (based on the author's personal feelings).

Counter-argument: none.

Consequences: obvious.

Could we train an AI model on the corpus of physics knowledge up to the year 1905 and then see if we can adjust the prompt to get it to output the theory of relativity?

This would be an interesting experiment for other historical discoveries too. I'm now curious if anybody has created a model with "old data" like documents and books from hundreds of years ago, and see if comes up with the same conclusions as researchers and scientists of the past.

Would AI have been able to predict the effectiveness of vaccines, insulin, other medical discoveries?

Wouldn't the ability to "ask the right questions" require that AI could update its own weights, as those weights determine which questions can be asked?

The first thing you need to understand is that no current llm based, transformer architected AI is going to get to agi. The design in essence is not capable of that kind of creativity. In fact no AI that has at its root a statistical analysis or probabilistic correlation will get us past the glorified Google parlor trick that is the modern llm in every form.

A great leap in IP but unfortunately is too important to blab about widely, is the solution to this problem and the architecture that will be contained in the ultimate AGI solution that emerges.

We saw algorithms designing circuits that no human engineer would design, even before the LLM (using genetic algorithms). So out-the-box thinking can be also more reachable than this author thinks.

Thing about the Einstein example is, that it was already known the speed of light is constant.

The question he asked was just that this fact was not compatible with the Maxwell equations.

> If something was not written in a book I could not invent it unless it was a rather useless variation of a known theory. __More annoyingly, I found it very hard to challenge the status-quo__, to question what I had learned.

(__emphasis__ mine)

As if "challenging the status-quo" was the goal in the first place. You ain't gonna get any Einstein by asking people to think inside the "outside the box" box. "Status quo" isn't the enemy, and defying it isn't the path to genius; if you're measuring your own intellectual capacity by proxy of how much you question, you ain't gonna get anywhere useful. After all, questioning everything is easy, and doesn't require any particular skill.

The hard thing is to be right, despite both the status-quo and the "question the status-quo" memes.

(It also helps being in the right time and place, to have access to the results of previous work that is required to make that next increment - that's another, oft forgotten factor.)

>Just consider the crazy paradigm shift of special relativity and the guts it took to formulate a first axiom like “let’s assume the speed of light is constant in all frames of reference” defying the common sense of these days (and even of today…)

I'm not an expert on this. Wasn't this an observed phenomenon before Albert put together his theory?

A way I've been thinking about this today is:

We can't distinguish between a truly novel response from an LLM or a hallucination.

We can get some of the way there, such as if we know what the outcome to a problem should look like, and are seeking a better function to achieve that outcome. Certainly at small scales and in environments where there are minimal consequences for failure, this could work.

But this breaks down as things get more complicated. We won't be able to test the effectiveness of 100 million potential solutions to eradicating brain tumors at once. Even if we somehow arrive at guaranteeing that every unforeseen consequence is also accounted for in our exercise in specifying the goals and constraints of the problem. We just simply don't have the logistics to run 100 million clinical trials where we also know how to account for countless confounding effects (let alone consent!)

The reality with people is that most of them don't come close to Einstein level intelligence. A lot of the stuff I ask perplexity or chatgpt is way beyond what I could reasonably ask from the vast majority of people I know. I love my relatives. But they are kind of useless for the vast majority of stuff that bounces around in my head.

AIs are at this point a useful tool for knowledge workers. They don't replace them but enhance their productivity. For scientific work, having an LLM that is trained on essentially all of the scientific work published, ever (until the cutoff date) is probably useful.

You can now have conversations with an AI about cross referencing your ideas with existing work. You might analyze a paper you are writing and ask it to summarize key claims, criticize those, and your methodology, cross reference claims with literature, etc. Find counter points to your claims, etc. And you could probably use it to come up with interesting follow up questions, let it formulate hypotheses and ways to verify those, etc. Most scientific work isn't Archimedes going Eureka while taking a bath but undergraduates, post docs, and other under paid research stuff grinding through piles and piles of existing work and filling their heads with enough information until finally something new and original pops out.

I got my Ph. D. in 2003. I'm part of the first generation of researchers that was able to use Google. At the time that was a huge enabler for tracking down obscure references and authors. Getting a paper published involves an enormous amount of what I just outlined. And LLMs can assist you with that. Will it hallucinate. Absolutely. But it will also dig out valid points, references, etc. Sorting that out is still work that you need to do. But it probably saves a lot of time. Will it propose original new theories. Maybe, maybe not. But it will speed up the process of zooming in on unanswered ones.

Science isn't necessarily about coming up with answers but coming up with interesting questions. That's what Einstein did: ask interesting questions. Researchers are still trying to answer some of them and verifying some of the answers he predicted.

>we don't just need a system that knows all the answers, but rather one that can ask questions nobody else has thought of or dared to ask.

That's called hate speech and every AI has been aggressively lobotomized to never do it by an army of RLHFers.

I think it's more of a social phenomenon than an intellectual characteristic. I guess these days people would just assume that outlier ideas come from autism, but I think that isn't necessarily true.

But maybe it helps to be socially isolated or just stubborn. People do not want to accept new approaches.

Clearly they do eventually, but there is always some friction.

But I think that it's been shown that through promoting and various types of training or tuning, LLMs can be configured to be non- sycophantic. It's just that humans don't want to be contradicted so that can be trained out of them during reinforcement.

Along with the training process just generally being aimed at producing expected rather than unexpected answers.

The most useful way to leverage LLMs for me has been as "content fillers". I'm a software engineer, and work with a rather large code base. Some parts are rarely touched, and loading the context into my brain whenever I need to go back to them requires quite a bit of effort. I found that asking cursor/Claude to suggest how to make the required chande rarely comes up with the right solution but usually points me in the right direction and is enough to help me load the context up. Similarly with my side projects, which typically involves knowledge that I don't use in my day to day.

> Many have been proposing "move 37" as evidence that AI has already reached Einstein-level intelligence

I don’t think this example applies in the ways we care about. Sure, in the domain of go we have incredibly powerful engines. Poker too, which is an imperfect information game which you could argue is more similar to life in that regard.

But life has far more degrees of freedom than go or poker, and the “value” of any one action is impossible to calculate due to imperfect information. And unlike in poker, where probabilities can be calculated, we don’t even have the probability distribution for most events, even if we could enumerate them.

I agree. But in response, I'd also point out that AI, even in its current form, can help speed up our tasks: collecting data, verifying/cleaning, recognizing primary patterns, writing simple code to automate, conducting a critical self-conversation, drafting and refining our writing, etc.... Science requires a lot of mundane, tedious work, and AI can undoubtedly help us in this aspect. The idea that we are developing AI to replace our brains to make scientific progress for us is misguided, to say the least .

I think discoveries by AI would be due to pattern matching. Like finding overlooked cancer markers that can be used for an earlier prognosis. The genius of Einstein and his thought experiments may elude an AI. It may take an AI designed on some other future model other than an LLM to "compress a century"

He called it wishful thinking. I believe the hype over AI is due to attempts to justify the enormous investments going into AI development has created an echo chamber.

I'm still waiting for the end of the world caused by AI as predicted by a very large number of prominent figures such as Sam Altman, Hinton, Musk, signers of the Center for AI Safety statement, Shane Legg, Martin Minsky, Eliezer Yudkowsky.

No sign yet.

On the other hand, LLMs are writing code which I can debug and eventually get to work in a real code base - and script writers everywhere are writing scripts more quickly, marketing people are writing better ad copy, employers are writing better job ads and real estate agents writing better ads for houses.

Can someone use AI to model einsteins (Ulam's) spiral. I believe its related to emergent properties from seemingly random data. It even 3 dimensionally models our galaxy...

>I’m afraid AI won't give us a "compressed 21st century".

There's no mention of exponential growth which seems a major omission when you are talking about centuries. Computers have kept improving in a Moore's law like way in terms of compute per dollar and no doubt will keep on like that for a while yet. Give it a few years and AI tech will be way better than what we have now. I don't know about exact timings like 5-10 years but in a while.

The current culture about AI & LLMs is that we are "only" memorizing the Web into model parameters and that a LLM is unable to "invent" new paradigms. Maybe we are under estimating what Unsupervised Learning & RL could provide. Re-inforcement learning is about exploring and finding new ways to accomplish a task and I see no limit here (except the computational resources needed).

I agree that any system claiming general intelligence must be able to form and modify a model of the world. A fundamental part of this is being able to ask counterfactual questions on its own understanding or knowledge of the world. The history of science is full of countless such examples. As of right now I'm not aware of any LLMs or indeed any AI system being able to do so.

Read Feyerabend’s Against Method Thom. You have rephrased (very well) the necessity of counter-inductive thinking.

I have yet to find a model that will stick, strictly, to factfulness. So I’m not entirely sure we don’t already have models that can question “facts” and invent novel things.

He said it himself, it’s just finding new/interesting gaps between existing knowledge.

If an llm is trained on knowledge up until say September 2023, could you use a corpus of interesting/insightful scientific discoveries and new methods developed after that date to evaluate/tune it? (Though I fear it would be a small corpus.)

It’s not man vs machine but man plus machine: human ask the right question, machine gives plausible answers. Human processes this and comes up with a new question. Without a human in the loop a breakthrough isn’t even registered.

Creativity is inherently disobedient. That’s why it’s such an enigma.

(Most comments here were posted to https://news.ycombinator.com/item?id=43317269 and then moved hither.)

Articles like this are worth the author's weight in gold. But as evidenced by the comments here, only few people understand the argument. The rest just "know better".

What about the not-LLMs works?

I know barely anything about it but it seems some people are interested and excited about protein engineering powered by neural networks.

Yes, you may want this but all I want are straight facts, not inituition. I don't want a conscience AI.

Einstein Mode LLM: Temperature - 2

Then it's just a matter of checking all the “nonsense” that's been generated.

Exactly. The hallmark of intelligence is the ability to disagree with everyone else and be right.

That this accurate article is considered noteworthy is scary.

ChatGPT will tell you the same - if you ask it.

The reality is much simpler than what is often presented about science geniuses. Lorentz and Poincare had the math down to explain early experiments and the Maxwell’s equations predictions of the constancy of the speed of light, and Einstein indeed provided a neat shift in the interpretation by taking a different perspective. (His photoelectric effect experiment and interpretation was a more genuine original contribution that came from experimental data and got him a Nobel prize.) The ideas behind gene editing existed since forever, but the observation that certain bacteria use a more accurate and selective gene editing system than viruses led to CRISPR. I have trouble with arguments where many examples in a row are not based on popular, often slightly mistaken beliefs, especially when it comes to discussions related to using such examples as analogies and arguments for predicting the future. I have talked to and worked with many different Nobel prize winners across different fields of science in my life, and although they were all extraordinary bright and focused individuals, the introductory part of this article misses the point. I agree that there is no linear extrapolation from being a good student, but I dont think the additional abilities are beyond the reach of machines. Focus, knowledge, perserverence, and the ability to analyze data very carefully are strict and challenging requirements. Asking the right questions is very important as well, but much easier than people assume.

creativity is possibly just random noise in feature space.

at best it could be random noise in a feature space of a thing modeling its own thought trajectory.

He means YMaaS, no? Might as well coin the acronym early.

I don't know about you, but I am always nice and friendly with my AI, I always say please and thank you. In the event AI will take over the world.

Hey look, the goalposts are being moved again. This time it's from top end researcher to generational genius. Question: what evidence is there that this benchmark will not be reached also? Time and again these essays make the mistake of assuming AI is a static thing, and refuse to acknowledge the inexorable march forward we are witnessing. As humans, we cling to our own fragile superiority. Even on this thread- I thought Hinton said the world would be transformed by now. That's NOT what was claimed. We are like three years in! Posts like this will be laughable in 10 years.

I've had some luck instructing AI to "Don't make up anything. If there's no answer, say I don't know".

Which made me think that AI would be far more useful (for me?) if it was tuned to "Dutchness" rather than "Americanness".

"Dutch" famously known for being brutally blunt, rude, honest, and pushing back.

Yet we seem to have "American" AI, tuned to "the customer is always right", inventing stuff just to not let you down, always willing to help even if that makes things worse.

Not "critical thinking" or "revolutionary" yet. Just less polite and less willing to always please you. In human interaction, the Dutch bluntness and honesty can be very off-putting, but It is quite efficient and effective. Two traits I very much prefer my software to have. I don't need my software to be polite or to not hurt my feelings. It's just a tool!

This article seems to argues from the way scientific discoveries are made by humans. It seems to me that its gist is similar to some article from the 80s that claims that computers will never play good chess, or an article from the 2000s that claims the same for go.

The general shape of these arguments is: "Playing chess/go well, or making scientific discoveries requires specific way of strategic thinking or the ability to form the right hypotheses. Computers don't do this, ergo they won't be able to play chess or make scientific discoveries".

I don't think this is a very good frame of reasoning. A scientific question can take one of the following shapes:

- (Mathematical) Here's a mathematical statement. Prove either it or its negation.

- (Fundamental natural science) Here're the results of the observations. What are the simplest possible model that explains all of them?

- (Engineering) We need to do X. What's an efficient way of doing it?

All of these questions could be solved in a "human" way, but it also possible to train AIs to approach them without going through the same process as the human scientists.