Slaughterhouse staff must all be vegans by now.
24,848 karma · joined September 26, 2015
What is reasoning?
In computer science and AI when we say "reasoning" we mean that we have a theory and we can derive the consequences of the theory by application of some inference procedure.
A theory is a set of facts and rules about some environment of interest: the real world, mathematics, language, etc. Facts are things we know (or assume) to be true: they can be direct observations, or implied, guesses. Rules are conditionally true and so most easily understood as implications: if we know some facts are true we can conclude that some other facts must also be true. An inference procedure is some system of rules, separate from the theory, that tells us how we can combine the rules and facts of the theory to squeeze out new facts, or new rules.
There are three types of reasoning, what we may call modes of inference: deduction, induction and abduction. Informally, deduction means that we start with a set of rules and derive new unobserved facts, implied by the rules; induction means that we start with a set of rules and some observations and derive new rules that imply the observations; and abduction means that we start with some rules and some observations and derive new unobserved facts that imply the observations.
It's easier to understand all this with examples.
One example of deductive reasoning is planning, or automated planning and scheduling, a field of classical AI research. Planning is the "model-based approach to autonomous behaviour", according to the textbook on planning by Geffner and Bonnet. An autonomous agent starts with a "model" that describes the environment in which the agent is to operate as a set of entities with discrete states, and a set of actions that the agent can take to change those states. The agent is given a goal, an instance of its model, and it must find a sequence of actions, that we call a "plan", to take the entities in the model from their current state to the state in the goal. This is usually achieved by casting the planning problem as pathfinding over a graph with a search algorithm like A*. Here, the agent's model is a theory, the search algorithm is the inference procedure, and the plan is a consequence of the theory. Deductive reasoning can be sound, as long as the facts and rules in the theory are correct: from correct premises we can deduce correct conclusions. We know of sound deductive inference rules, e.g. A*, and Resolution, used in automated theorem proving and SAT-Solving, are sound.
The classic example of inductive reasoning is inferring the colour of swans. Most swans are white (apparently) so if we have only seen white swans we have no reason to believe there are any other colours: we are forced to infer that all swans are white. We may only be disabused of our fallacy if we happen to observe a swan that is not white, e.g. a black swan. But who is to say when such a magnificent creature will grace us with its presence, outside of Tchaikovsky's ballets? Induction is thus revealed to be unsound: even given true premises we can still arrive at the wrong conclusions. Another example is the scientific method: imagine an idealised scientist, perfectly spherical, in a frictionless vacuum. She starts with a scientific theory, then goes out into the world and makes new observations about a phenomenon not described by her theory. She constructs a hypothesis to extend her theory so as to explain the new observations. The hypothesis is a set of rules, where the premises are the consequences of the rules in her initial theory. Then, being an idealised scientist, she goes looking for new observations to refute her hypothesis. Science only gives us the tools to know when we're wrong.
Abductive reasoning is the mode of inference exemplified by Sherlock Holmes. We can imagine Sherlock and Watson standing outside a tavern in London, watching as a gentleman of interest steps out of the tavern with egg on his lapel. "Ah, my dear Watson, what can we conclude from this observation?". "Why my dear Holmes, we can conclude that the man had eggs for breakfast". Holmes and Watson can arrive at this conclusion, about a fact that they have not directly observed, because they have a theory with a rule that says "if one eats eggs, one may get some on one's lapels". Working backwards from this rule, and their observation of egg on the man's lapels, they can guess that he had eggs even if they didn't directly observe him doing so. Abduction is also unsound: the man may have swapped coats with an accomplice, who was the one who had eggs for breakfast instead.
And now you know what "reasoning" means. So the next time someone asks: "what is reasoning?", you can let them know and turn the discussion to more interesting, more productive directions.
>> But how much time have human brains spent working on the problem in either of those time periods? How many mathematicians have worked on the problem? 10k?
So neither 10k humans worked on Navier-Stokes, nor has any human spent a century of non-stop work on it.
But I could have made the point more clear maybe.
>> I don't, of course, disagree that it's possible for an AI company to put a lot of AI agents to work on a problem, but I'm not sure how that makes what they can do less impressive. The fact that you can do that has always been a major part of why AI could be such a big deal. "A country of geniuses in a datacentre" is the kind of thing people have said; we aren't quite there yet, but the "country" part is as important as the "geniuses" part.
Yes, I see your point, but those are not geniuses. Grigori Perelman proved the Poincaré conjecture alone, though as he has emphasised his work was based on advances made by others, particularly Richard S. Hamilton. That we can call a genius: a single man who solves one of the most interesting problems in all of mathematics building on the work of his predecessors. 10k agents that search blindly and find a result by luck (or by stealing it), I don't agree we can call "genius". That's what I call "brute force". Anyone who wants to call OpenAI's agents "a country of geniuses" has first to deal with the fact that they look a lot like monkeys on typewriters.
You can also consider Watson, different to Cyc in that its knowledge base was built with a lot of machine learning, including some neural nets. It wasn't an LLM but the original version (before corporate went at it and destroyed it) was perfectly capable of interacting in an open-ended manner, notably winning at Jeopardy years before BERT was a glimmer in Jacob Devlin's eye.
I note again that you were the one who brought common sense reasoning in the conversation but there's a large literature on rule-based systems that do that based e.g. on non-monotonic logics. You should familiarise yourself with that literature before engaging in dares with strangers on the internets.
Wanna guess why? I'm too tired now to expand the argument properly but basically understanding the components of a complex system doesn't mean you understand the principles of the system. A mathematician who is not an expert in AI has no reason to be particularly capable of understanding how AI works, i.e. how all the maths that go into creating an AI system come together to create. An AI system.
My bad for not showing my work and inadvertently leading you down the garden path, but the "~100 agent-years" calculation goes like this:
10,000 agents * 88 hours = 880,000 agent-hours
88,000 agent-hours / 24 hours = 36,666.7 agent-days
36,666.7 agent-days / 365 days = 100.5 agent-years.
That's what you get for working 24 hours a day, 7 days a week, 365 days a year. Realistically speaking, that's not a work schedule any human can follow.
It's hard to make a realistic estimate because normally even a very dedicated mathematician will not be working exclusively on one problem all their waking time, or even all their working time. But, let's ignore this and assume a pretty standard work schedule of 8 working hours, five working days a week, and 52 working weeks a year.
Now, that's:
8 hours * 5 days = 40 working hours a week
40 hours * 52 weeks a year = 2080 hours a year
880,000 agent-hours / 4 humans = 220,000 hours per human
220,000 hours per human / 2080 hours a year = ~105.8 years
To clarify, that's how I estimate the number of years it would take a mathematician to do a quarter of the work of the 10k OpenAI agents if that mathematician worked only on solving Navier-Stokes and did nothing else in their entire career.
That's just not a realistic work schedule for any human. You can adjust the working hours if you want but I don't believe you'll get any realistic estimate. Don't forget that most academics' careers last around 30 years from PhD to Professor Emeritus. If you want a more realistic estimate of how much time it would take how many humans to do the work of the 10k OpenAI agents, you can start from that assumption and work your way up from that.
>> That would be a more convincing argument if the AIs, like the humans, had been around and trying to solve those problems for the last 2k years. However, as you might have noticed, the state of the art in AI was rather primitive 2000 years ago.
Sure. But the thing is agents can run 24/7, 365/365 in parallel and as you see above they can cover 2000 years of human work in much less time. I'm not going to estimate how much because the only bottleneck is the amount of compute and money that an AI company wishes to spend, and that depends on their motivation to solve a particular problem. However, with sufficient motivation 2k years of human research (keeping mind that's not 2k years of continuous work) can be covered in a few ... months? Probably.
For the n'th time: the recent successes of AI in mathematics are the result of a brute-force attack. See the proof for Navier-Stokes: 10k agents running for 88 hours; that's ~100 GPU years. How many human-years were invested in solving the same problem, before they were overtaken in the last few days by an AI? 90? Not even: that's just the time since Jeal Leray's statement of the problem in 1934. 26, if you want to count the time since 2000 when the Clay Institute named it as one of its Millennium Prize problems. But how much time have human brains spent working on the problem in either of those time periods? How many mathematicians have worked on the problem? 10k? Not likely.
And all that's without even considering whether the AI based its proof on carelessly shared work by the humans. Or rather, yes, let's consider that: it totally did.
Further. There have been several results in mathematics produced by AI but we have no information on how many attempts were made to produce similar results that failed. Because we don't have this information we cannot estimate the true capabilities of AI.
Yet we can observe that, for example, out of the six Millennium Prize Problems remaining open before the claim of a solution of Navier-Stokes existence and smoothness, only one (the aforementioned) was solved by an AI. We can assume that the AI companies (more than one) tried and failed to solve the others. We can even guess that they previously tried, and failed, to solve Navier Stokes itself, and only succeeded once the progress made by Buckmaster and Alpöge was in the training data [1]. That's a success rate of one out of six, or ~17%. That's what's gonna solve all of maths and destroy the tradition of mathematics? A success rate of 17%? Well, grab a Snickers 'cause we're gonna be waiting for some time!
Moreover. If we include in the list the Poincaré conjecture, proved by Grigori Perelman, who is a human, that's a score of AI 1-1 Humans. And that's being gracious: we have one Millennium Problem fully solved by humans, one solved partly by humans with a last-mile solution by AI. We have thousands of problems solved by humans in the last 2k years and how many by AI? A couple dozen? Oooh scary!
- Hey Hal! Prove that P ≠ NP!
- I'm sorry Dave. I can't do that.
What I'm trying to say, without the snark (sorry): Panic if you will, but the machines are not yet taking over. If you're panicking, panic for what you believe they will be able to do in the future. Because they certainly can't do hat in the present. They can't solve "all of mathematics" (whatever that means).
______________
[1] Yes it was. Buckmaster reported that he turned off the option to train on his data in July, after working on the problem with Alpöge for a year since September 2025. OpenAI claimed a solution in September, a month after they had stopped hoovering up Buckmaster's data. They had plenty of time to train on his data. Ask for references if you want them because I don't have them handy right now.
And that's why my point was that we don't know how to come up with heuristics: because we currently don't.
Edit: if you mean that we can probabilistic-recall all those heuristics, that's not right. Because such heuristics are tacit knowledge that is very difficult, maybe even impossible, to articulate with enough accuracy to reproduce in a computer. We certainly can't get LLMs to learn them from the web because the web doesn't have text that explains e.g. how to control your muscles to climb a tree.
I don't know why you say this, I didn't say anything about common sense.
However, you mention CYC. That's a system that is perfectly capable of common sense reasoning and very much like an LLM in many ways. And that should be no surprise: LLMs are giant Expert Systems trained on a human knowledge-base, i.e. the web. OpenAI basically managed to achieve what Doug Lenat was trying to achieve except they did it with machine learning over massive data and compute instead of painstaking manual coding, but it's the same kind of system in the end.
I keep banging on that drum but the first AI system to prove mathematical theorems was Logic Theorist by Alan Newell and Herbert Simon, presented at the Dartmouth conference that named the field of AI in 1956. Wikipedia says:
Logic Theorist proved 38 of the first 52 theorems in chapter two of [Alfred North] Whitehead and Bertrand Russell's Principia Mathematica, and found a new and shorter proof for Theorem 2.85.[3]
https://en.wikipedia.org/wiki/Logic_Theorist
The first system to outperform human experts in medical diagnosis was MYCIN, an Expert System from the early 1970's at Stanford. Wikipedia again:
An evaluation of MYCIN was conducted at the Stanford Medical School. The first phase of the evaluation consisted of 10 test cases of diverse origin, chosen by a physician who was not acquainted with MYCIN's methods or knowledge base. These cases were presented to 7 physicians and 1 senior medical student. 10 prescriptions were compiled for each of the cases, 1 recommended by MYCIN, 1 prescribed by the treating physician at the county hospital, and 8 by the aforementioned individuals. The second phase of the evaluation consisted of eight infectious disease specialists being provided the clinical summary and set of 10 prescriptions for each of the 10 cases and tasked to provide their own recommendations for each case and assess the 10 prescriptions. MYCIN received an acceptability rating of 65%, which was comparable to the 42.5% to 62.5% rating of five faculty members.[9] This study is often cited as showing the potential for disagreement about therapeutic decisions, even among experts, when there is no "gold standard" for correct treatment.[citation needed]
https://en.wikipedia.org/wiki/Mycin#Results
And then of course there's the long history of human-dominating AI players for traditional board games starting with DeepBlue's win against GM Gary Kasparov in 1996.
Again: we've had that sort of AI for a long, long time now.
It would be great if any claim of "moving goalposts" has better be very well informed about the history of AI and its accomplishments, as well as its failures, first.
Do we? When have LLMs come up with heuristics? I'm sure if you ask an LLM to tell you e.g. how to solve a maze it will print out the instructions for the follow-the-left-wall heuristic, but that's not "coming up" with a heuristic.
I fear though we are about to go into one of those unproductive conversations about the capabilities of LLMs to produce novel results which I think has now reached saturation point all over the internets.
I mean I don't doubt you, I just thought Brits just say "here kitty kitty" because that's what I usually see in movies and TV shows and so on.
It's funny what you miss when you're not embedded in the culture of a place since birth...
Basically that's all of AI nowadays: generate-and-test loops. It's like the 1950's all over again.
The trouble is when you want to do that in the general case, i.e. when you don't know the problem you're solving. Unfortunately we don't know how to come up with heuristics automatically.
... well ish. We have relaxations in Planning again, but that really doesn't seem to have anything to do with what bend is doing.
Is that really the case? The cost to OpenAI for the Navier-Stokes problem has been estimated in the millions of dollars, anything between 6 and 40 million depending on who you ask. Now, I know that OpenAI has a lot of money stashed under the mattress but we have to remember that they by no means had any guarantee that the money they spent would actually solve the problem. In fact, they have probably burned similar amounts of money for many other problems that we have never heard anything about, for the simple reason that their super-secret in-house special models didn't solve them.
What I'm saying is that you need to think of the monetary constraints to solving major open problems in maths etc. If it costs millions of dollars a pop for a shot in the dark that has a small chance to succeed, that's not something that's sustainable. Letting the users take on some of that cost by burning through their own token budget so that they can claim to solve a problem that "has stumped mathematicians for 80 years" and the like, can make reduce the cost a bit but by how much? Remember that OpenAI had 10,000 agents running for 88 hours on Navier-Stokes. Who, outside of AI companies, has that many tokens? What power user is going to spend thousands of millions to try and solve the Millennium Prize problems?
So, really, maybe mathematicians who are looking forward to a new era of AI-generated results that advance mathematics may end up being disappointed. AI companies are not charities, they do not really care about mathematics and they aren't doing things that they don't think will bring them more money, somehow. And when did solving mathematical problems bring anyone any money?
That sounds like, for the grid navigation game in the example, in order to prove that no move leads from the initial state to an invalid state you'd have to search the set of all move sequences to find out if one of them leads to an invalid state. We know from Planning & Scheduling that this is a PSPACE-complete task. So that's ... not what you mean, right?
I take it you haven't used Microsoft Windows?
Joking, joking...
Btw, "laws" are basically what we used to call assertions so why the new terminology?
Edit: actually now that I think about it, it's more like constraint programming with a generate-and-test loop than assertions. Again, why not just say "constraints" instead of inventing a new term?
Khachapuri, I presume: https://en.wikipedia.org/wiki/Khachapuri
It looks delicious in pictures and I've heard about it from some Georgian friends but I never got to try it. But I did try satsivi and it was finger-licking good.
Not to disappoint you but I don't think they ever will. HN is free to join and use so people will join and use it and say whatever they want to say whether it makes sense or not. Filtering out noise is a useful skill to have especially since one can't block users or mute conversations and so on.
>> In the moment I probably thought if they are GM level and I can beat them, is this some interesting find, my disappointment honestly led me to making a rather incorrect call on this one.
Sorry, I didn't get this? What was the incorrect call you made?
You gotta be careful how you use the word "relation" here because there's an informal meaning (I'm related to my cousin) and a more strict, formal meaning, that is used in computer science e.g. in the "Relational Calculus" etc. In the formal sense, the one relation that LLMs learn during training is the co-occurrence of tokens in a corpus of text, what's called more technically a "collocation" relation. Nothing says that this is enough to play chess, so I'm indeed doubtful that they can.
Yes, I do mean that the LLM's weights are set so that it will execute minimax or MCTS when it needs to. That has nothing to do with whether humans can do the same or not.
I don't disagree that a Transformer could learn to play chess if it was explicitly trained to do that. My argument is that LLMs, trained to predict the next token, have not learned to play chess. That's LLMs, not Transformers.
Just to make sure this is not taken as splitting hairs, the point is that there's all sorts of claims made about what LLMs learn when they train on text. For example, there was a claim by Sundar Pichai that one of their models had learned to translate Bengali without explicitly being trained to do so. It later emerged that Bengali was indeed included in the model's training set [1]. It's not clear whether that included parallel texts, e.g. between Begnali and English or another intermediary language, in any case Sundar Pichai's claim was that the ability to translate Bengali was "emergent".
So I'm interested in understanding the extent to which these "emergent" abilities are real or not. With chess, given the amount of textual data tracing games that floats about on the open internet, I would totally except some ability to play chess to "emerge". Maybe the reported 700-800 ELO level is even that sort of ability. Maybe we should only expect LLMs to learn to play at the level of an untrained, casual player. Maybe not. I have no idea.
On the other hand, the fact they keep making elementary mistakes like illegal moves must be taken to mean that, so far, LLMs haven't learned to play chess.
__________________
[1] https://www.buzzfeednews.com/article/pranavdixit/google-60-m...
Er, all the time? In the South East of England, I specifically look for menus with pictures of the food on Just Eats because I can get an idea of how good the food is going to be that way. I don't think I've ever received something that didn't look like what was on the menu pictures. Sometimes people make a nice presentation with fancy dishware and so on (Indian restaurants do that a lot in particular), but the food is always the same as what I get when I order, except it's in a foam box or similar.
And there's plenty of pushback on here about the chess thing besides your very valid points.
EDIT: anyway if I can offer a bit of unsolicited advice, it won't do you or anyone any good to accuse everyone who doesn't agree with you of laziness, even if you can see e.g. they haven't really read an article. Just say the thing you wan to say and let them figure it out. Most people will appreciate that much better and you will feel better about yourself for acting like a mature adult.
It's even in the site guidelines:
Please don't comment on whether someone read an article. "Did you even read the article? It mentions that" can be shortened to "The article mentions that".