428 karma · joined February 15, 2022
https://arxiv.org/abs/2501.17186
PS "Major commercial American LLM" is not very meaningful, you could be using GPT4o with that description.
> "classical reasoning uses consciousness and awareness as elements of processing"
They are not the _same_ concept then.
> It's only meaningless if you don't know what the philosophical or epistemological definitions of reasoning are. Which is to say, you don't know what reasoning is. So you'd think it was a meaningless statement.
The problem is the only information we have is internal. So we may claim those things exist in us. But we have no way to establish if they are happening in another person, let alone in a computer.
> Do computers think, or do they compute?
Do humans think? How do you tell?
It's a question that purposefully uses a limitation of the system. There are many such questions for humans. They are called trick questions. It is not that crazy to call it a trick question.
> This is a demonstration of a system that delusionally refuses to accept correction and correct its misunderstanding (which is a thing that is fundamental to their claim of intelligence through reasoning).
First, the word 'delusional' is strange here unless you believe we are talking about a sentient system. Second, you are just plain wrong. LLMs are not "unable to accept correction" at all, in fact they often accept incorrect corrections (sycophanty). In this case the model is simply unable to understand the correction (because of the nature of the tokenizer) and it is therefore 'correct' behaviour for it to insist on it's incorrect answer.
> Why would anyone believe these things can reason, that they are heading towards AGI, when halfway through a dialogue where you're trying to tell it that it is wrong it doubles down with a dementia-addled explanation about the two bs giving the word that extra bounce?
People believe the models can reason because they produce output consistent with reasoning. (That is not to say they are flawless or we have AGI in our hands.) If you don't agree, provide a definition of reasoning that the model does not meet.
> Why would you offer up an easy out for them like this? You're not the PR guy for the firm swimming in money paying million dollar bonuses off what increasingly looks, at a fundamental level, like castles in the sand. Why do the labour?
This, like many of your other messages, is rather obnoxious and dripping with performative indignation while adding little in the way of substance.
Wrong, it's an artifact of tokenizing. The model doesn't have access to the individual letters, only to the tokens. Reasoning models can usually do this task well - they can spell out the word in the reasoning buffer - the fact that GPT5 fails here is likely a result of it incorrectly answering the question with a non-reasoning version of the model.
> There's no real reasoning.
This seems like a meaningless statement unless you give a clear definition of "real" reasoning as opposed to other kinds of reasoning that are only apparant.
> It seems that reasoning is just a feedback loop on top of existing autocompletion.
The word "just" is doing a lot of work here - what exactly is your criticism here? The bitter lesson of the past years is that relatively simple architectures that scale with compute work surprisingly well.
> It's really disingenuous for the industry to call warming tokens for output, "reasoning," as if some autocomplete before more autocomplete is all we needed to solve the issue of consciousness.
Reasoning and consciousness are seperate concepts. If I showed the output of an LLM 'reasoning' (you can call it something else if you like) to somebody 10 years ago they would agree without any doubt that reasoning was taking place there. You are free to provide a definition of reasoning which an LLM does not meet of course - but it is not enough to just say it is so. Using the word autocomplete is rather meaningless name-calling.
> Edit: Letter frequency apparently has just become another scripted output, like doing arithmetic. LLMs don't have the ability to do this sort of work inherently, so they're trained to offload the task.
Not sure why this is bad. The implicit assumption seems to be that an LLM is only valueable if it literally does everything perfectly?
> Edit: This comment appears to be wildly upvoted and downvoted. If you have anything to add besides reactionary voting, please contribute to the discussion.
Probably because of the wild assertions, charged language, and rather superficial descriptions of actual mechanics.
Cycle time measures how long it takes for a unit of work (usually a ticket) to move from initiation to completion within a team's workflow. It is a property of the team / process, not individuals. It can be used to generate statistical forecasts for when a number of tasks are likely to be completed by the team process.
For most teams, actual programming or development tasks usually represent only a small portion—often less than 20%—of the total cycle time. The bulk of cycle time typically results from process inefficiencies like waiting periods, bottlenecks, handoffs between team members, external dependencies (such as waiting for stakeholder approval or code review), and other friction points within the workflow. Because of this, many Kanban-based forecasting methods don't even attempt to estimate technical complexity. They focus instead on historical cycle time data.
For example, consider a development task estimated to take a developer only two days of actual programming. If the developer has to wait on code reviews, deal with shifting priorities, or coordinate with external teams, the total cycle time from task initiation to completion might end up taking two weeks. Here, focusing on the individual’s performance misses the bigger issue: the structural inefficiencies embedded within the workflow itself.
Even if tasks were perfectly and uniformly distributed across all developers—a scenario both unlikely and probably undesirable—this fact would remain. The purpose of measuring cycle time is to identify and address overall process problems, not to evaluate individual contributions.
If you're using cycle time as an individual performance metric, you're missing the fundamental point of what cycle time actually measures.
The primary value of measuring cycle time is precisely that it captures end-to-end process inefficiencies, variability, and bottlenecks, rather than individual effort. This systemic perspective is fundamental in Kanban methodology, where cycle time and its variance are commonly used to forecast delivery timelines.
People in Europe dont take a train from Greece to Sweden. They fly. In fact most fly Vienna to Amsterdam.
In the same way somebody from New York would definitely fly to LA. (They are not driving now btw)
That doesn't preclude the existence of public transport connecting NY to Philadelphia. It also does not preclude NY from being walkable! Or bikeable. It doesn't stop NY from having good public transport! It doesn't force you to drive to work in NY.
This is much more about local policy.
Different solutions at different scales!
If you tune the model to behave bad in a limited way (write SQL injection for example), other bad behaviour like racism will just emerge.
specifically: https://x.com/ESYudkowsky/status/1894453376215388644
-- Eliezer Yudkowsky