I'm sorry, where does it say the German operator mistyped the key? The article says indeed that the message remained unsolved because it was sent on the wrong date, but I can't see the bit about the mistyping anywhere.
24,868 karma · joined September 26, 2015
What is reasoning?
In computer science and AI when we say "reasoning" we mean that we have a theory and we can derive the consequences of the theory by application of some inference procedure.
A theory is a set of facts and rules about some environment of interest: the real world, mathematics, language, etc. Facts are things we know (or assume) to be true: they can be direct observations, or implied, guesses. Rules are conditionally true and so most easily understood as implications: if we know some facts are true we can conclude that some other facts must also be true. An inference procedure is some system of rules, separate from the theory, that tells us how we can combine the rules and facts of the theory to squeeze out new facts, or new rules.
There are three types of reasoning, what we may call modes of inference: deduction, induction and abduction. Informally, deduction means that we start with a set of rules and derive new unobserved facts, implied by the rules; induction means that we start with a set of rules and some observations and derive new rules that imply the observations; and abduction means that we start with some rules and some observations and derive new unobserved facts that imply the observations.
It's easier to understand all this with examples.
One example of deductive reasoning is planning, or automated planning and scheduling, a field of classical AI research. Planning is the "model-based approach to autonomous behaviour", according to the textbook on planning by Geffner and Bonnet. An autonomous agent starts with a "model" that describes the environment in which the agent is to operate as a set of entities with discrete states, and a set of actions that the agent can take to change those states. The agent is given a goal, an instance of its model, and it must find a sequence of actions, that we call a "plan", to take the entities in the model from their current state to the state in the goal. This is usually achieved by casting the planning problem as pathfinding over a graph with a search algorithm like A*. Here, the agent's model is a theory, the search algorithm is the inference procedure, and the plan is a consequence of the theory. Deductive reasoning can be sound, as long as the facts and rules in the theory are correct: from correct premises we can deduce correct conclusions. We know of sound deductive inference rules, e.g. A*, and Resolution, used in automated theorem proving and SAT-Solving, are sound.
The classic example of inductive reasoning is inferring the colour of swans. Most swans are white (apparently) so if we have only seen white swans we have no reason to believe there are any other colours: we are forced to infer that all swans are white. We may only be disabused of our fallacy if we happen to observe a swan that is not white, e.g. a black swan. But who is to say when such a magnificent creature will grace us with its presence, outside of Tchaikovsky's ballets? Induction is thus revealed to be unsound: even given true premises we can still arrive at the wrong conclusions. Another example is the scientific method: imagine an idealised scientist, perfectly spherical, in a frictionless vacuum. She starts with a scientific theory, then goes out into the world and makes new observations about a phenomenon not described by her theory. She constructs a hypothesis to extend her theory so as to explain the new observations. The hypothesis is a set of rules, where the premises are the consequences of the rules in her initial theory. Then, being an idealised scientist, she goes looking for new observations to refute her hypothesis. Science only gives us the tools to know when we're wrong.
Abductive reasoning is the mode of inference exemplified by Sherlock Holmes. We can imagine Sherlock and Watson standing outside a tavern in London, watching as a gentleman of interest steps out of the tavern with egg on his lapel. "Ah, my dear Watson, what can we conclude from this observation?". "Why my dear Holmes, we can conclude that the man had eggs for breakfast". Holmes and Watson can arrive at this conclusion, about a fact that they have not directly observed, because they have a theory with a rule that says "if one eats eggs, one may get some on one's lapels". Working backwards from this rule, and their observation of egg on the man's lapels, they can guess that he had eggs even if they didn't directly observe him doing so. Abduction is also unsound: the man may have swapped coats with an accomplice, who was the one who had eggs for breakfast instead.
And now you know what "reasoning" means. So the next time someone asks: "what is reasoning?", you can let them know and turn the discussion to more interesting, more productive directions.
Do we? When have LLMs come up with heuristics? I'm sure if you ask an LLM to tell you e.g. how to solve a maze it will print out the instructions for the follow-the-left-wall heuristic, but that's not "coming up" with a heuristic.
I fear though we are about to go into one of those unproductive conversations about the capabilities of LLMs to produce novel results which I think has now reached saturation point all over the internets.
I mean I don't doubt you, I just thought Brits just say "here kitty kitty" because that's what I usually see in movies and TV shows and so on.
It's funny what you miss when you're not embedded in the culture of a place since birth...
Basically that's all of AI nowadays: generate-and-test loops. It's like the 1950's all over again.
The trouble is when you want to do that in the general case, i.e. when you don't know the problem you're solving. Unfortunately we don't know how to come up with heuristics automatically.
... well ish. We have relaxations in Planning again, but that really doesn't seem to have anything to do with what bend is doing.
Is that really the case? The cost to OpenAI for the Navier-Stokes problem has been estimated in the millions of dollars, anything between 6 and 40 million depending on who you ask. Now, I know that OpenAI has a lot of money stashed under the mattress but we have to remember that they by no means had any guarantee that the money they spent would actually solve the problem. In fact, they have probably burned similar amounts of money for many other problems that we have never heard anything about, for the simple reason that their super-secret in-house special models didn't solve them.
What I'm saying is that you need to think of the monetary constraints to solving major open problems in maths etc. If it costs millions of dollars a pop for a shot in the dark that has a small chance to succeed, that's not something that's sustainable. Letting the users take on some of that cost by burning through their own token budget so that they can claim to solve a problem that "has stumped mathematicians for 80 years" and the like, can make reduce the cost a bit but by how much? Remember that OpenAI had 10,000 agents running for 88 hours on Navier-Stokes. Who, outside of AI companies, has that many tokens? What power user is going to spend thousands of millions to try and solve the Millennium Prize problems?
So, really, maybe mathematicians who are looking forward to a new era of AI-generated results that advance mathematics may end up being disappointed. AI companies are not charities, they do not really care about mathematics and they aren't doing things that they don't think will bring them more money, somehow. And when did solving mathematical problems bring anyone any money?
That sounds like, for the grid navigation game in the example, in order to prove that no move leads from the initial state to an invalid state you'd have to search the set of all move sequences to find out if one of them leads to an invalid state. We know from Planning & Scheduling that this is a PSPACE-complete task. So that's ... not what you mean, right?
I take it you haven't used Microsoft Windows?
Joking, joking...
Btw, "laws" are basically what we used to call assertions so why the new terminology?
Edit: actually now that I think about it, it's more like constraint programming with a generate-and-test loop than assertions. Again, why not just say "constraints" instead of inventing a new term?
Khachapuri, I presume: https://en.wikipedia.org/wiki/Khachapuri
It looks delicious in pictures and I've heard about it from some Georgian friends but I never got to try it. But I did try satsivi and it was finger-licking good.
Not to disappoint you but I don't think they ever will. HN is free to join and use so people will join and use it and say whatever they want to say whether it makes sense or not. Filtering out noise is a useful skill to have especially since one can't block users or mute conversations and so on.
>> In the moment I probably thought if they are GM level and I can beat them, is this some interesting find, my disappointment honestly led me to making a rather incorrect call on this one.
Sorry, I didn't get this? What was the incorrect call you made?
You gotta be careful how you use the word "relation" here because there's an informal meaning (I'm related to my cousin) and a more strict, formal meaning, that is used in computer science e.g. in the "Relational Calculus" etc. In the formal sense, the one relation that LLMs learn during training is the co-occurrence of tokens in a corpus of text, what's called more technically a "collocation" relation. Nothing says that this is enough to play chess, so I'm indeed doubtful that they can.
Yes, I do mean that the LLM's weights are set so that it will execute minimax or MCTS when it needs to. That has nothing to do with whether humans can do the same or not.
I don't disagree that a Transformer could learn to play chess if it was explicitly trained to do that. My argument is that LLMs, trained to predict the next token, have not learned to play chess. That's LLMs, not Transformers.
Just to make sure this is not taken as splitting hairs, the point is that there's all sorts of claims made about what LLMs learn when they train on text. For example, there was a claim by Sundar Pichai that one of their models had learned to translate Bengali without explicitly being trained to do so. It later emerged that Bengali was indeed included in the model's training set [1]. It's not clear whether that included parallel texts, e.g. between Begnali and English or another intermediary language, in any case Sundar Pichai's claim was that the ability to translate Bengali was "emergent".
So I'm interested in understanding the extent to which these "emergent" abilities are real or not. With chess, given the amount of textual data tracing games that floats about on the open internet, I would totally except some ability to play chess to "emerge". Maybe the reported 700-800 ELO level is even that sort of ability. Maybe we should only expect LLMs to learn to play at the level of an untrained, casual player. Maybe not. I have no idea.
On the other hand, the fact they keep making elementary mistakes like illegal moves must be taken to mean that, so far, LLMs haven't learned to play chess.
__________________
[1] https://www.buzzfeednews.com/article/pranavdixit/google-60-m...
Er, all the time? In the South East of England, I specifically look for menus with pictures of the food on Just Eats because I can get an idea of how good the food is going to be that way. I don't think I've ever received something that didn't look like what was on the menu pictures. Sometimes people make a nice presentation with fancy dishware and so on (Indian restaurants do that a lot in particular), but the food is always the same as what I get when I order, except it's in a foam box or similar.
And there's plenty of pushback on here about the chess thing besides your very valid points.
EDIT: anyway if I can offer a bit of unsolicited advice, it won't do you or anyone any good to accuse everyone who doesn't agree with you of laziness, even if you can see e.g. they haven't really read an article. Just say the thing you wan to say and let them figure it out. Most people will appreciate that much better and you will feel better about yourself for acting like a mature adult.
It's even in the site guidelines:
Please don't comment on whether someone read an article. "Did you even read the article? It mentions that" can be shortened to "The article mentions that".
Monte Carlo Tree Search is stochastic.
I mean we've done all this before in a task-specific fashion. It's useful to know that LLMs haven't managed to do that in the process of learning to represent the entire text on the web. On the other hand they have gotten say very good at machine translation without being trained exclusively (and I select the preceding word carefully) on machine translation.
Edit: I'm saying this because there is this idea expressed by e.g. Ilya Sutskever, that in order to predict the next token accurately an LLM has to learn something about all of underlying reality. See for example this interview with Dwarkesh:
https://x.com/biobootloader/status/1640512444958396416
Where Sutskever claims that "Predicting the next token well means you understand the underlying reality that led to the creation of that token".
If that were true, we should have seen LLMs play good chess by now. There is a huge amount of data on playing chess floating around on the web in the form of algebraic chess notation and if LLMs were capable of learning the "underlying reality" of chess, they would already have. They haven't. Because they can't. What Sutskever is saying flies in the face of literally hundreds of years of statistical modelling, which is to say, building predictive models that, very explicitly, do not have to understand any "underlying reality" and only have to be good at modelling a dataset.
This is unfair to HN readers all of whom but one did not post the comment you replied to. You can't just tar everyone with the same brush. There are thousands (hundreds of thousands?) of users on this site.
Lots of people make this claim about "specific task[s] enjoying clearly defined levels of task performance" but they forget that generative AI is also extremely good at generating a) art and b) prose in literary style. None of those things has "clearly defined levels of task performance", in fact they are both the complete opposite of well-defined tasks. Who knows what counts for "good" art? [1]
For me the right model for generative AI is "a million monkeys on typewriters" [2]. Holding any other model to heart will at some point fail to predict observations and cause you to be unpleasantly surprised. Not least because AI companies are actively engineering their systems to optimise for this model and they have a lot of people working on that engineering and shedloads of money to throw at it.
Don't underestimate what a million monkeys on typewriters can do. They can do anything and everything, given enough time. Geneartive AI can also do anything and everything given enough resources. The only question is: how much is going to be "enough"?
____________________
[1] Yes yes, AI art tends to be slop. Not denying that. But part of the problem with slop is that it presents as technically very competent except that it lacks a certain je-ne-sais-quoi, which makes it good art; aesthetics. The point is that there is no clear measure of what makes technically competent art, any more than there is for aesthetics.
And yet generative AI is very good at it.
[2] There's even an article on wikipedia except it's about one monkey on one typewriter with infinite time. There's a proof too.
Anyway, Sturgeon's law and all that.
Chess and Go have most definitely not been solved. In AI research when we say that a game (and usually we mean a traditional board game) has been "solved", what we mean is that we can correctly predict the outcome of any game instance from any board position provided both players play perfectly [1]. This can mean we have an algorithm that chooses the perfect play for each player in any position or that we have mapped the entire game tree for that game: every move in every game, from start to finish.
There are games that have been solved this way: tic-tac-toe; connect four; nim; othello, checkers. But not chess and Go.
Rather, what we have for chess and Go are super-human engines, i.e. computer players that no human has ever beaten in the full game. So for example a chess engine like Stockfish can always choose what move to make in order to win against a human player in any board position. However, that is still not the same as knowing how to play perfectly in any board position. Like the old joke with the two guys trying to outrun a bear, a chess engine doesn't have to play perfect chess to be super-human: it just has to play better than any human.
To the extent that chess and Go are games it could be said that winning is the whole point of AI game-playing [2]. But the article above suggests that there is no point of "winning" in maths. In AI chess it is unfortunate that the development of super-human chess engines has ended all research in using AI chess to understand the way that human players play chess (which is to say: not by alpha-beta minimax, or Monte Carlo Tree Search, or anything like that) which was the original reason that luminaries of the field such as Shannon, Minsky, McCarthy, Michie, and even Turing himself were interested in the question in the first place. The concern then for mathematics is that AI "winning" at maths will kill mathematics research and replace it with ... as McCarthy would have it "very fast fruit flies" [3].
_______________
[1] https://en.wikipedia.org/wiki/Solved_game
[2] Then again see John McCarthy's criticism of that idea written in the wake of DeepBlue vs Kasparov: https://www-formal.stanford.edu/jmc/newborn/newborn.html
[3] ibid.
... spaz on the floor ...
By spending all day thinking about them instead of running a country.