Hacker News Reader

Comment by whatever1 | Hacker News Reader

The evidence shows that there is no methodological moat for LLMS. The moat of the frontier folks is just compute. xAI went in months from nothing to competing with the top dogs. DeepSeek too. So why bother with splurging billions in talent when you can buy GPUs and energy instead and serve the compute needs of everyone?

Also Amazon is in another capital intensive business. Retail. Spending billions on dubious AWS moonshots vs just buying more widgets and placing them across the houses of US customers for even faster deliveries does not make sense.

View on HN

A lot of C-suite people seem to have an idea that if they just throw enough compute at LLMs that AGI will eventually emerge, even though it's pretty clear at this point that LLMs are never going to lead to general intelligence. In their view it makes sense to invest massive amounts of capital because it's like a lottery ticket to being the future AGI company that dominates the world.

I recall Zuckerberg saying something about how there were early signs of AI "improving itself." I don't know what he was talking about but if he really believes that's true and that we're at the bottom of an exponential curve then Meta's rabid hiring and datacenter buildout makes sense.

In early 2023, I remember someone breathlessly explaining that there are signs that LLMs that are seemingly good at chess/checkers moves may have a rudimentary model of the board within them, somehow magically encoded into the model weights through the training. I was stupid enough to briefly entertain the possibility until I actually bothered to develop a high level understanding of the transformer architecture. It's surprising how much mysticism this field seems to attract. Perhaps it being a non-deterministic, linguistically invoked black box, triggers the same internal impulses that draw some people to magic and spellcasting.

Just because it's not that hard to reach a high-level understanding of the transformer pipeline doesn't mean we understand how these systems function, or that there can be no form of world model that they are developing. Recently there has been more evidence for that particular idea [1]. The feats of apparent intelligence LLMs sometimes display have taken even their creators by surprise. Sure, there's a lot of hype too, that's part and parcel of any new technology today, but we are far from understanding what makes them perform so well. In that sense, yeah you could say they are a bit "magical".

[1] https://the-decoder.com/new-othello-experiment-supports-the-...

> Just because it's not that hard to reach a high-level understanding of the transformer pipeline doesn't mean we understand how these systems function

Mumbo jumbo magical thinking.

They perform so well because they are trained on probabilistic token matching.

Where they perform terribly, e.g math, reasoning, they are delegating to other approaches, and that's how you get the illusion that there is actually something there. But it's not. Faking intelligence is not intelligence. It's just text generation.

> In that sense, yeah you could say they are a bit "magical"

Nobody but the most unhinged hype pushers are calling it "magical". The LLM can never ever be AGI. Guessing the next word is not intelligence.

> there can be no form of world model that they are developing

Kind of impossible to form a world model if your foundation is probabilistic token guessing which is what LLMs are. LLMs are a dead end in achieving "intelligence", something novel as an approach needs to be discovered (or not) to go into the intelligence direction. But hey, at least we can generate text fast now!

> LLMs are a dead end in achieving "intelligence"

There is no evidence to indicate this is the case. To the contrary, all evidence we have points to these models, over time, being able to perform a wider range of tasks at a higher rate of success. Whether it's GPQA, ARC-AGI or tool usage.

> they are delegating to other approaches > Faking intelligence is not intelligence. It's just text generation.

It seems like you know something about what intelligence actually is that you're not sharing. If it walks, talks and quacks like a duck, I have to assume it's a duck[1]. Though, maybe it quacks a bit weird.

[1] https://en.wikipedia.org/wiki/Solipsism

> There is no evidence to indicate this is the case

Burden of proof is on those trying to convince us to buy into the idea of LLMs as being "intelligence".

There is no evidence of the Flying Spaghetti monster or Zeus or God not existing either, but we don't take seriously the people who claim they do exist (and there isn't proof because these concepts are made up).

Why should we take seriously the tolks claiming LLMs are intelligence without proof (there can't be proof, of course, because LLMs are not intelligence)?

Is there something we are all missing? Using Claude feels like magic sometimes, but can't everyone see the limitation now that we are 4 years and 100s of billions down the road?

Are they still really hoping that they are gonna tweak a model and feed it an even bigger dataset and it will be AGI?

I'm not a fan of mysticism. I'm also with you that these are simply statistical machines. But I don't understand what happened when understood transformers at a high-level.

If you're saying the magic disappeared after looking at a single transformer, did the magic of human intelligence disappear after you understood human neurons at a high level?

Its insane really, anyone who has worked with LLMs for a bit and has an idea of how they work shouldn't think its going to lead to "AGI".

Hopefully some big players, like FB bankrupt themselves.

Tbh I find this view odd, and I wonder what people view as agi now. It used to be that we had extremely narrow pieces of AI and I remember being on a research project about architectures and just very basic “what’s going on?” was advanced. Understanding that someone asked a question, that would be solved by getting a book and being able to then go and navigate to the place the book was likely to be was fancy. Most systems could solve literally one type of problem. They weren’t just bad at other things they were fundamentally incapable of anything but an extremely narrow use case.

I can throw wide ranging problems at things like gpt5 and get what seem like dramatically better answers than if I asked a random person. The amount of common sense is so far beyond what we had it’s hard to express. It used to be always pointed out that the things we had were below basic insect level. Now I have something that can research a charity, find grants and make coherent arguments for them, read matrix specs and debug error messages, and understand sarcasm.

To me, it’s clear that agi is here. But then what I always pictured from it may be very different to you. What’s your image of it?

It's more that "random" people are dumb as bricks (but we've in the name of equality and historic measurement errors decided to forgo that), add to it that AI's have a phenomenal (internet sized) memory makes them far more capable than many people.

However, even "dumb" people can often make judgements structures in a way that AI's cannot, it's just that many have such a bad knowledge-base that they cannot build the structures coherently whereas AI's succeed thanks to their knowledge.

I wouldn't be surprised if the top AI firms today spend an inordinate amount of time to build "manual" appendages into the LLM systems to cater to tasks such as debugging to uphold the facade that the system is really smart, while in reality it's mostly papering up a leaky model to avoid losing the enormous investments they need to stay alive with a hope that someone on their staff comes up a real solution to self-learning.

https://magazine.sebastianraschka.com/p/understanding-reason...

I think the discrepancy between different views on the matter mainly stems from the fact that state-of-the-art LLMs are better (sometimes extremely better) at some tasks, and worse (sometimes extremely worse) at other tasks, compared to average humans. For example, they're better at retrieving information from huge amounts of unstructured data. But they're also terrible at learning: any "experience" which falls out of the context window is lost forever, and the model can't learn from its mistakes. To actually make it learn something requires very many examples and a lot of compute, whereas a human can permanently learn from a single example.

> human can permanently learn from a single example

This, to me at least, seems like an important ingredient to satisfying a practical definition / implementation of AGI.

Another might be curiosity, and I think perhaps also agency.

I think it's clear that nobody agrees what AGI is. OpenAI describes it in terms of revenue. Other people/orgs in terms of, essentially, magic.

If I had to pick a name, I'd probably describe ChatGPT & co as advanced proof of concepts for general purpose agents, rather than AGI.

> I think it's clear that nobody agrees what AGI is

People selling AI products are incentivized to push misleading definitions of AGI.

Human-level intelligence. Being able to know what it doesn't know. Having a practical grasp on the idea of truth. Doing math correctly, every time.

I give it a high-res photo of a kitchen and ask it to calculate the volume of a pot in the image.

You discover truth by doing stuff in real world and observing the results. Current LLM have enough intelligence, but all the inputs they have are the “he said she said” by us monkeys, including all omissions and biases.

But many humans can't do a lot of those things and we still consider them "generally intelligent"

None of what you describe would I label within the realm of 'average'

It's not about what the average human can do - it's about what humans as a category are capable of. There will always be outliers (in both directions), but you can, in general, teach a human a variety of tasks, such as performing arithmetic deterministically, that you cannot teach to, for example, a parrot.

I don’t have a very high expectation of AGI at all. Just an algorithm or system you can put onto a robot dog, and get a dog level general intelligence. You should be able to live with that robot dog for 10 years and it should be just as capable as a dog throughout that timespan.

Hell, I’d even say we have AGI if you could emulate something like a hamster.

LLMs are way more impressive in certain ways than such a hypothetical AGI. But that has been true of computers for a long time. Computers have been much better at Chess than humans for decades. Dogs can’t do that. But that doesn’t mean that a chess engine is an AGI.

I would also say we have a special form of AGI if the AI can pass an extended Turing test. We’ve had chat bots that can fool a human for a minute for a long time. Doesn’t mean we had AGI. So time and knowledge was always a factor in a realistic Turing test. If an AGI can fool someone who knows how to properly probe an LLM, for a month or so, while solving a bunch of different real world tasks that require stable long term memory and planning, then I’d day we’re in AGI territory for language specifically. I think we have to distinguish between language AGI and multi-modal AGI. So this test wouldn’t prove what we could call “full” AGI.

These are some of the missing components for full AGI: - Being able to act as a stable agent with a stable personality over long timespans - Capable of dealing with uncertainties. Having a understanding of what it doesn’t know - One-shot learning, with long term retention, for a large number of things - Fully integrated multi-modality across sound, vision, and other inputs/outputs we may throw at it.

The last one is where we may be able to get at the root of the algorithm we’re missing. A blind person can learn to “see” by making clicks and using their ears to see. Animals can do similar “tricks”. I think this is where we truly see the full extent of the generality and adaptability of the biological brain. Imagine trying to make a robot that can exhibit this kind of adaptability. It doesn’t fit into the model we have for AI right now.

my picture of AGI is 1) autonomous improvement 2) ability to say 'i don't know/can't be done'

I wonder if 2) is a result of published bias for positive results in the training set. An “I don’t know” response is probably ranked unsatisfactory by human feedback and most published scientific literature are biased towards positive results and factual explanations.

In my experience, the willingness to say "I don't know" instead of confabulate is also down-rated as a human attribute, so it's not surprising that even an AGI trained on the "best" of humanity would avoid it.

Nobody is saying that LLM's don't work like magic. I know how neural networks work and they still feel like voodoo to me.

What we are saying is that LLM's can't become AGI. I don't know what AGI will look like, but it won't look like an LLM.

There is a difference between being able to melt iron and being able to melt tungsten.

Recently I realized that US are very close to a centrally planned economy. Meta wasted 50B on metaverse, which like how much Texas spends on healthcare. Now the "AI" investments seems dubious.

You could fund 1000+ projects with this kinds of money. This is not an effective capital allocation.

The only way we'll have AGI is if people get dumber. Using modern tech like LLMs makes people dumber. Ergo, we might see AGI sooner than expected.

> ... and has an idea of how they work shouldn't think its going to lead to "AGI"

Not sure what level of understanding are you referring to but having learned and researched about the pretty much all LLM internals I think this has led me exactly to the opposite line of thinking. To me it's unbelievable what we have today.

I think AI research is like anything else really. The smartest people are heads down working on their problems. The people going on podcasts are less connected to day to day work.

It’s also pretty useless to talk about whether something is AGI without defining intelligence in the first place.

I think something like we saw in the show "Devs" is much more likely, although what the developers did with it in the show was bonkers unrealistic. But some kind of big enough quantum device basically.

Just scaling them up might not leat to "AGI", but they can still lead to AGI as a bridge.

This is not and has not been the consensus opinion. If you're not an AI researcher you shouldn't write as if you've set your confidence parameter to 0.95.

Of course it might be the case, but it's not a thing that should be expressed with such confidence.

Is it widely accepted that LLMs won't lead to AGI? I've asked Gemini, so it came up with four primary arguments for this claim, commenting on them briefly:

1) LLMs as simple "next token predictors" so they just mimicry thinking: But can it be argued that current models operate on layers of multiple depth and are able to actually understand by building concepts and making connections on abstract levels? Also, don't we all mimicry?

2) Grounding problem: Yes, models build their world models on text data, but we have models operating on non-textual data already, so this appears to be a technical obstacle rather than fundamental.

3) Lack of World Model. But can anyone really claim they have a coherent model of reality? There are flat-earthers, yet I still wouldn't deny them having AGI. People hallucinate and make mistakes all the time. I'd argue hallucinations is in fact the sign of an emerging intelligence.

4) Fixed learning data sets. Looks like this is now being actively solved with self-improving models?

I just couldn't find a strong argument supporting this claim. What am I missing?

Why on earth would you copy and paste an LLM's output into a comment? What does that accomplish or provide that just a simply stated argument doesn't accomplish more succinctly? If you don't know something, simply don't comment on it -- or ask a question.

None of the above is AI.

> I've asked Gemini, so it came up with four primary arguments for this claim, commenting on them briefly:

This line means, and literally says, that everything that follows is a summary or direct quotation from an LLM's output.

There's a more charitable but unintuitive interpretation, in which "commenting on them briefly" is intended to mean "I will comment on them briefly:". But this isn't a natural interpretation. It's one I could be expected to reach only after seeing your statement that 'none of the above is AI.' But even this more charitable interpretation actually contradicts your claim that it's not AI.

So now I'm even less sure I know what you meant to communicate. Either I'm missing something really obvious or the writing doesn't communicate what you intended.

Fur future reference, pasting llm slop feels exactly as patronizing as back when people pasted links to google searches in response to questions they considered beneath their dignity to answer. Except in this case, no-one asked to begin with.

> I don't know what he was talking about

There's a bunch of ways AI is improving itself, depending on how you want to interpret that. But it's been true since the start.

1. AI is used to train AI. RLHF uses this, curriculum learning is full of it, video model training pipelines are overflowing with it. AI gets used in pipelines to clean and upgrade training data a lot.

2. There are experimental AI agents that can patch their own code and explore a tree of possibilities to boost their own performance. However, at the moment they tap out after getting about as good as open source agents, but before they're as good as proprietary agents. There isn't exponential growth. There might be if you throw enough compute at it, but this tactic is very compute hungry. At current prices it's cheaper to pay an AI expert to implement your agent than use this.

So have an AI with a 40% error rate judge an AI with an 40% error rate…

AGI is a complete no go until a model can adjust its own weights on the fly, which requires some kind of negative feedback loop, which requires a means to determine a failure.

Humans have pain receptors to provide negative feedback and we can imagine events that would be painful such as driving into a parked car would be painful without having to experience it.

If current models could adjust its own weights to fix the famous “how many r’s in strawberry” then I would say we are on the right path.

However, the current solution is to detect the question and forward it to a function to determine the right answer. Or attempt to add more training data the next time the model is generated ($$$). Aka cheat the test.

I think LLM as a toolsmith like demonstrated in the Voyager paper (1) is another interesting approach to creating a system that can learn to do a task better over time. (1) https://arxiv.org/abs/2305.16291

> There are experimental AI agents that can patch their own code and explore a tree of possibilities to boost their own performance. However, at the moment they tap out after getting about as good as open source agents, but before they're as good as proprietary agents.

Interesting. Do you have links?

Not OP, but the Darwin Godel machine comes to mind: https://arxiv.org/abs/2505.22954

That's the one!

Thanks!

I'm skeptical that RLHF really works. Doesn't it just patch the obvious holes so it looks better on paper? If it can't reason then it will continue to get 2nd and 3rd order difficulty problems wrong.

> it's pretty clear at this point that LLMs are never going to lead to general intelligence.

It is far from clear. There may well be emergent hierarchies of more abstract thought at much higher numbers of weights. We just don't know how a transformer will behave if one is built with 100T connections - something that would finally approach the connectome level of a human brain. Perhaps nothing interesting but we just do not know this and the current limitation in building such a beast is likely not software but hardware. At these scales the use of silicon transistors to approximate analog curve switching models just doesn't make sense. True neuromorphic chips may be needed to approach the numbers of weights necessary for general intelligence to emerge. I don't think there is anything in production at the moment that could rival the efficiency of biological neurons. Most likely we do not need that level of efficiency. But it's almost certain that stringing together a bunch of H100s isn't a path to the scale we should be aiming for.

I don't get it, I really don't.

Even assuming a company gets to AGI first this doesn't mean another one will follow.

Suppose that FooAI gets to it first: - competitors may get there too in a different or more efficient way - Some FooAI staff can leave and found their own company - Some FooAI staff can join a competitor - FooAI "secret sauce" can be figured out, or simply stolen, by a competitor

At the end of the day, it really doesn't matter, the equation AI === commodity just does not change.

There is no way to make money by going into this never ending frontier model war, price of training keeps getting higher and higher, but your competitors few months later can achieve your own results for a fraction of your $.

Some would say that the race to AGI is like the race to nuclear weapons and that the first to get there will hold all the cards (and be potentially able to stop others getting there.) It's a bit too sci-fi for me.

If AGI is reached it would be trivial for the competing superpowers to completely quarantine themselves network wise by cutting undersea cables long enough to develop competing AGI

I don't know if AGI will emerge from LLM, but I'm always reminded of the Chinese room thought experiment. With billions thrown at the idea it will certainly be the ultimate answer as to whether true understanding can emerge from a large enough dictionary.

Please stop refering to the Chinese Room - it's just magical/deist thinking in disguise. It postulates that humans have way of 'understanding' things that is impossible to replicate mechanically.

The fact that philosophy hasn't recognized and rejected this argument based on this speaks volumes of the quality of arguments accepted there.

(That doesn't mean LLMs are or will be AGI, its just this argument is tautological and meaningless)

That some people use the Chinese Room to ascribe some magical properties to human consciousness says more about the person drawing that conclusion than the thought experiment itself.

I think it's entirely valid to question whether a computer can form an understanding through deterministically processing instructions, whether that be through programming language or language training data.

If the answer is no, that shouldn't lead to a deist conclusion. It can just as easily lead to the conclusion that a non-deterministic Turing machine is required.

I'd appreciate if you tried to explain why instead of resorting to ad hominem.

> I think it's entirely valid to question whether a computer can form an understanding through deterministically processing instructions, whether that be through programming language or language training data.

Since the real world (including probabilistic and quantum phenomena) can be modeled with deterministic computation (a pseudorandom sequence is deterministic, yet simulates randomness), if we have a powerful enough computer we can simulate the brain to a sufficient degree to have it behave identically as the real thing.

The original 'Chinese Room' experiment describes a book of static rules of Chinese - which is probably not the way to go, and AI does not work like that. It's probabilistic in its training and evaluation.

What you are arguing is that constructing an artificial consciousness lies beyond our current computational ability(probably), and understanding of physics (possibly), but that does not rule out that we might solve these issues at some point, and there's no fundamental roadblock to artificial consciousness.

I've re-read the argument (https://en.wikipedia.org/wiki/Chinese_room) and I cannot help but conclude that Searle argues that 'understanding' is only something that humans can do, which means that real humans are special in some way a simulation of human-shaped atoms are not.

Which is an argument for the existence of the supernatural and deist thinking.

> I'd appreciate if you tried to explain why instead of resorting to ad hominem.

It is not meant as an ad hominem. If someone thinks our current computers can't emulate human thinking and draws the conclusion that therefore humans have special powers given to them by a deity, then that probably means that person is quite religious.

I'm not saying you personally believe that and therefore your arguments are invalid.

> Since the real world (including probabilistic and quantum phenomena) can be modeled with deterministic computation (a pseudorandom sequence is deterministic, yet simulates randomness), if we have a powerful enough computer we can simulate the brain to a sufficient degree to have it behave identically as the real thing.

The idea that a sufficiently complex pseudo-random number generator can emulate real-world non-determinism enough to fully simulate the human brain is quite an assumption. It could be true, but it's not something I would accept as a matter of fact.

> I've re-read the argument (https://en.wikipedia.org/wiki/Chinese_room) and I cannot help but conclude that Searle argues that 'understanding' is only something that humans can do, which means that real humans are special in some way a simulation of human-shaped atoms are not.

In that same Wikipedia article Searle denies he's arguing for that. And even if he did secretly believe that, it doesn't really matter, because we can draw our own conclusions.

Disregarding his arguments because you feel he holds a hidden agenda, isn't that itself an ad hominem?

(Also, I apologize for using two accounts, I'm not attempting to sock puppet)

What are his arguments then?

>Searle argues that, without "understanding" (or "intentionality"), we cannot describe what the machine is doing as "thinking" and, since it does not think, it does not have a "mind" in the normal sense of the word.

This is the only sentence that seems to be pointing to what constitutes the specialness of humans, and the terms of 'understanding' and 'intentionality' are in air quotes so who knows? This sounds like the archetypical no true scotsman fallacy.

In mathematical analysis, if we conclude that the difference between 2 numbers is smaller than any arbitrary number we can pick, those 2 numbers must be the same. In engineering, we can reduce the claim to 'any difference large about to care about'

Likewise if the difference between a real human brain and an arbitrarily sophisticated Chinese Room brain is arbitrarily small, they are the same.

If our limited understanding of physics and engineering makes the practical difference not zero, this essentially becomes a bit of a somewhat magical 'superscience' argument claiming we can't simulate the real world to a good enough resolution that the meaningful differences between our 'consciousness simulator' and the thing itself disappear - which is an extraordinary claim.

> What are his arguments then?

They're in the "Complete Argument" section of the article.

> This sounds like the archetypical no true scotsman fallacy.

I get what you're trying to say, but he is not arguing only a true Scotsman is capable of thought. He is arguing that our current machines lack the required "causal powers" for thought. Powers that he doesn't prescribe to only a true Scotsman, though maybe we should try adding bagpipes to our AI just to be sure...

Thanks, but that makes his arguments even less valid.

He argues that computer programs only manipulate symbols and thus have no semantic understanding.

But that's not true - many programs, like compilers that existed back when the argument was made, had semantic understanding of the code (in a limited way, but they did have some understanding about what the program did).

LLMs in contrast have a very rich semantic understanding of the text they parse - their tensor representations encode a lot about each token, or you can just ask them about anything - they might not be human level at reading subtext, but they're not horrible either.

Now you're getting to the heart of the thought experiment. Because does it really understand the code or subtext, or is it just really good at fooling us that it does?

When it makes a mistake, did it just have a too limited understanding or did it simply not get lucky with its prediction of the next word? Is there even a difference between the two?

I would like to agree with you that there's no special "causal power" that Turing machines can't emulate. But I remain skeptical, not out of chauvinism, but out of caution. Because I think it's dangerous to assume an AI understands a problem simply because it said the right words.

> I cannot help but conclude that Searle argues that ‘understanding’ is only something that humans can do, which means…

Regardless of whether Searle is right or wrong, you’ve jumped to conclusions and are misunderstanding his argument and making further assumptions based on your misunderstanding. Your argument is also ad-hominem by accusing people of believing things they don’t believe. Maybe it would be prudent to read some of the good critiques of Searle before trying to litigate it rapidly and sloppily on HN.

The randomness stuff is very straw man, definitely not a good argument, best to drop it. Today’s LLMs are deterministic, not random. Pseudorandom sequences come in different varieties, but they model some properties of randomness, not all of them. The functioning of today’s neural networks, both training and inference, is exactly a book of static rules, despite their use of pseudorandom sequences.

In case you missed it in the WP article, most of the field of cognitive science thinks Searle is wrong. However, they’re largely not critiquing him for using metaphysics, because that’s not his argument. He’s arguing that biology has mechanisms that binary electronic circuitry doesn’t; not human brains, simply physical chemical and biological processes. That much is certainly true. Whether there’s a difference in theory is unproven. But today currently there absolutely is a difference in practice, nobody has ever simulated the real world or a human brain using deterministic computation.

If scientific consensus is that he's wrong why is he being constantly brought up and defended - am I not right to call them out then?

Nobody brings up that light travels through the aether, that diseases are caused by bad humors etc. - is it not right to call out people for stating theory that's believed to be false?

>The randomness stuff is very straw man,

And a direct response to what armada651 wrote:

>I think it's entirely valid to question whether a computer can form an understanding through deterministically processing instructions, whether that be through programming language or language training data.

> He’s arguing that biology has mechanisms that binary electronic circuitry doesn’t; not human brains, simply physical chemical and biological processes.

Once again the argument here changed from 'computers which only manipulate symbols cannot create consciousness' to 'we don't have the algorithm for consiousness yet'.

He might have successfully argued against the expert systems of his time - and true, mechanistic attempts at language translation have largely failed - but that doesn't extend to modern LLMs (and pre LLM AI) or even statistical methods.

You’re making more assumptions. There’s no “scientific consensus” that he’s wrong, there are just opinions. Unlike the straw man examples you bring up, nobody has proven the claims you’re making. If they had, then the argument would go away like the others you mentioned.

Where did the argument change? Searle’s argument that you quoted is not arguing that we don’t have the algorithm yet. He’s arguing that the algorithm doesn’t run on electrical computers.

I’m not defending his argument, just pointing out that yours isn’t compelling because you don't seem to fully understand his, at least your restatement of it isn’t a good faith interpretation. Make his argument the strongest possible argument, and then show why it doesn’t work.

IMO modern LLMs don’t prove anything here. They don’t understand anything. LLMs aren’t evidence that computers can successfully think, they only prove that humans are prone to either anthropomorphic hyperbole, or to gullibility. That doesn’t mean computers can’t think, but I don’t think we’ve seen it yet, and I’m certainly not alone there.

>most of the field of cognitive science thinks Searle is wrong.

>There’s no “scientific consensus” that he’s wrong, there are just opinions.

And? Are you imagining that these aren’t both true at the same time? If so, I’m happy to explain. Since nothing has been proven, there’s nothing “scientific”. And since there’s some disagreement, “consensus” has not been achieved yet. This is why your presumptive use of “scientific consensus” was not correct, and why the term “scientific consensus” is not the same thing as “most people think”. A split of 60/40 or 75/25 or even 90/10 counts as “most” but does not count as “consensus”. So I guess maybe be careful about assuming what something means, it seems like this thread was limited by several incorrect assumptions.

> The fact that philosophy hasn't recognized and rejected this argument based on this speaks volumes of the quality of arguments accepted there.

That's one possibility. The other is that your pomposity and dismissiveness towards the entire field of philosophy speaks volumes on how little you know about either philosophical arguments in general or this philosophical argument in particular.

Another ad hominem, I'd like you to refute my claim that Searle's argument is essentially 100% magical thinking.

And yes, if for example, medicine would be no worse at curing cancer than it is today, yet doctors asserted that crystal healing is a serious study, that would reflect badly on the field at large, despite most of it being sound.

Searle refutes your claim that there’s magical thinking.

“Searle does not disagree with the notion that machines can have consciousness and understanding, because, as he writes, "we are precisely such machines". Searle holds that the brain is, in fact, a machine, but that the brain gives rise to consciousness and understanding using specific machinery.”

But the core of the original argument is that programs only manipulate symbols and consciousness can never arise just through symbol manipulation - which here then becomes 'we have not discovered the algorithms' for consciousness yet.

It's just a contradiction.

When you say something that contradicts his statements, it doesn’t mean he’s wrong, it most likely means you haven’t understood or interpreted his argument correctly. The Wikipedia page you linked to doesn’t use the word “algorithm”, so the source of the contradiction you imagine might be you. Searle says he thinks humans are biological machines, so your argument should withstand that hypothesis rather than dismiss it.

Why on earth do you take it as an ad hominem attack? Do you really think your comment isn't dismissive or pompous?

Another ad hominem, just like you calling anyone who talks about the chinese room thought experiment a deist?

It is still relevant because it hasn’t been disproven yet. So far all computer programs are Chinese Rooms, LLM’s included.

If you’re talking about it being proven or disproven you’re misunderstanding the point of the thought experiment.

"Please stop referring to this thought experiment because it has possible interpretations I don't personally agree with"

Please give me an interpretation that is both correct an meaningful (as in possible to disprove)

The human way of understanding things can be replicated mechanically, because it is mechanical in nature. The contents of your skull are an existence proof of AGI.

The A stands for artificial.

The contents of my skull are only a proof for AGI if your mechanical machine replicates all its processes. It's not a question about whether a machine can reproduce that, it's a question about whether we have given our current machines all the tools it needs to do that.

The theory of special relativity does not say 'you can't exceed the speed of light(unless you have a really big rocket)'. It presents a theoretical limit. Likewise the Chinese room doesn't state that consciousness is an intractable engineering problem, but an impossibility.

But the way Searle formulates his argument, by not defining what consciousness is, he essentially gives himself enough wiggle room to be always right - he's essentially making the 'No True Scotsman' fallacy.

The moat is people, data, and compute in that order.

It’s not just compute. That has mostly plateaued. What matters now is quality of data and what type of experiments to run, which environments to build.

This "moat" is actually constantly shifting (which is why it isn't really a moat to begin with). Originally, it was all about quality data sources. But that saturated quite some time ago (at least for text). Before RLHF/RLAIF it was primarily a race who could throw more compute at a model and train longer on the same data. Then it was who could come up with the best RL approach. Now we're back to who can throw more compute at it since everyone is once again doing pretty much the same thing. With reasoning we now also opened a second avenue where it's all about who can throw more compute at it during runtime and not just while training. So in the end, it's mostly about compute. The last years have taught us that any significant algorithmic improvement will soon permeate across the entire field, no matter who originally invented it. So people are important for finding this stuff, but not for making the most of it. On top of that, I think we are very close to the point where LLMs can compete with humans on their own algorithmic development. Then it will be even more about who can spend more compute, because there will be tons of ideas to evaluate.

To put that into a scientific context - compute is capacity to do experiments and generate data ( about how best to build models ).

However I do think you are missing an important aspect - and that's people who properly understand important solvable problems.

ie I see quite a bit "we will solve this x, with AI' from startup's that don't fundamentally understand x.

>we will solve this x, with AI

You usually see this from startup techbro CEOs understand neither x nor AI. Those people are already replacable by AI today. The kind of people who think they can query ChatGPT once with "How to create a cutting edge model" and make millions. But when you go in on the deep end, there are very few people who still have enough tech knowledge to compete with your average modern LLM. And even the Math Olympiad gold medalists high-flyers at DeepSeek are about to have a run for their money with the next generation. Current AI engineers will shift more and more towards senior architecture and PM roles, because those will be the only ones that matter. But PM and architecture is already something that you could replace today.

> Originally, it was all about quality data sources.

It still is! Lots of vertical productivity data that would be expensive to acquire manually via humans will be captured by building vertical AI products. Think lawyers, doctors, engineers.

That's literally what RLAIF has been doing for a while now.

People matter less and less as well.

As more opens up in OSS and academic space, their knowledge and experience will either be shared, rediscovered, or become obsolete.

Also many of the people are coasting on one or two key discoveries by a handful of people years ago. When Zuck figures this out he gonna be so mad.

Not all will become OSS. Some will become products, and that requires the best people.

Lets not pretend this is strategy. Amazon has been trying and failing to hire top AI people. No-one in their right minds would join. Even Meta has to shell out 8-9 figures for top people, who with any modicum of talent or self respect would go to Amazon rather than Anthropic, OAI, GDM? They bought Adept, everyone left.

AWS is also falling far behind Azure wrt serving AI needs at the frontier. GCP is also growing at a faster rate and has a way more promising future than AWS in this space.

AWS is very far behind, its already impacting the stock. Without a winning AI offering, all new cloud money is going to GCP and Azure. They have a huge problem

> The evidence shows that there is no methodological moat for LLMS.

Does it? Then how come Meta hasn't been able to release a SOTA model? It's not for a lack of trying. Or compute. And it's not like DeepSeek had access to vastly more compute than other Chinese AI companies. Alibaba and Baidu have been working on AI for a long time and have way more money and compute, but they haven't been able to do what DeepSeek did.

They may not have been leading (as in, releasing a SOTA model), but they definitely can match others - easily, as shown by llama 3/4, which proves the point - there is no moat. With enough money and resources, you can match others. Whether without SOTA models you can make a business out of it is a different question.

Meta never matched the competition with their Llama models. They've never even come close. And Llama 4 was an actual disaster.

I am not a daily user, so only rely on reviews and benchmarks - actual experience may be different.

Even in reviews and benchmark, Llama wasn't close to frontier models. Also Llama 2/3 lead in open weight models wasn't more than few months.

> but they definitely can match others - easily, as shown by llama 3/4

Are we living in the same universe? LLAMA is universally recognized as one of the worst and least successful model releases. I am almost certain you haven't ever tried a LLAMA chat, because, by the beard of Thor, it's the worst experience anyone could ever had, with any LLM.

LLAMA 4 (behemoth, whatever, whatever) is an absolute steaming pile of trash, not even close to ChatGPT 4o/4/5/, Gemini(any) and even not even close to cheaper ones like DeepSeek. And to think Meta pirated torrents to train it...

What a bunch of criminal losers and what a bunch of waste of money, time and compute. Oh, at least the Metaverse is a success...

https://www.pcgamer.com/gaming-industry/court-documents-show...

https://www.cnbc.com/2025/06/27/the-metaverse-as-we-knew-it-...

> The moat of the frontier folks is just compute.

This is not really true. Google has all the compute but in many dimensions they lag behind GPT-5 class (catching up, but it has not been a given).

Amazon itself did try to train a model (so did Meta) and had limited success.

I switched to Gemini with my new phone and I literally couldn't tell a difference. It is actually crazy how small the cost of switching is for LLMs. It feels like AI is more like a commodity than a service.

> I switched to Gemini with my new phone and I literally couldn't tell a difference. It is actually crazy how small the cost of switching is for LLMs. It feels like AI is more like a commodity than a service.

It is. It's wild to me that all these VCs pouring money into AI companies don't know what a value-chain is.

Tokens are the bottom of the value-chain; it's where the lowest margins exist because the product at that level is a widely available commodity.

I wrote about this already (shameless plug: https://www.rundata.co.za/blog/index.html?the-ai-value-chain )

On top of that, the on-device models have got stronger and stronger as the base models + RL has got better. You can do on your laptop now what 2 years ago was state of the art.

Which dimensions do you see Google lagging on? They seem broadly comparable on the usual leaderboard (https://lmarena.ai/leaderboard) and anecdotally I can't tell the difference in quality.

I tend personally to stick with ChatGPT most of the time, but only because I prefer the "tone" of the thing somehow. If you forced me to move to Gemini tomorrow I wouldn't be particularly upset.

> Which dimensions do you see Google lagging on? They seem broadly comparable on the usual leaderboard (https://lmarena.ai/leaderboard) and anecdotally I can't tell the difference in quality.

Gemini holds indeed the top spot, but I feel you framed your response quite well: they are all broadly comparable. The difference in the synthetic benchmark from the top spot and the 20th spot was something like 57 points on a scale of 0-1500

" in many dimensions they lag behind GPT-5 class " - such as?

Outside of computer, "the moat" is also data to train on. That's an even wider moat. Now, google has all the data. Data no one else has or ever will have. If anything, I'd expect them to outclass everyone by a fat margin. I think we're seeing that on video however.

You think Chinese companies are short on data and people? Google doesn’t have an advantage there until the CCP takes on a more hands on approach.

Tin foil hat time:

- If you were a God and you wanted to create an ideal situation for the arrival of AI

- It would make sense to precede it with a social media phenomena that introduces mass scale normalization of sharing of personal information

Yes, that would be ideal …

People can’t stop sharing and creating data on anything, for awhile now. It’s a perfect situation for AI as an independent, uncontrollable force.

> People can’t stop sharing and creating data on anything

Garbage in. Garbage out.

There has never been a better time to produce an AI that mimics a racist uneducated teenager.

We already had Tay. https://en.wikipedia.org/wiki/Tay_(chatbot)

Do you want to model the world accurately or not? That person is part of our authentic reality. The most sophisticated AI in the world will always include that person(s).

Not in the slightest. I want useful information services that behave in a mature and respectable fashion.

not according to google: “We have no moat, and neither does OpenAI”: the big memo and the big HN thread on same https://news.ycombinator.com/item?id=35813322

a bit weird to think about it since google has literally internet.zip in multiple versions over the years, all of email, all of usenet, all of the videos, all of the music, all of the user's search interest, ads, everything..

> a bit weird to think about it since google has literally internet.zip in multiple versions over the years, all of email, all of usenet, all of the videos, all of the music, all of the user's search interest, ads, everything..

Yeah, Google totally has a moat. Them saying that they have no moat doesn't magically make that moat go away.

They also own the entire vertical which none of the competitors do - all their competitors have to buy compute from someone who makes a profit just on compute (Nvidia, for example). Google owns the entire vertical, from silicon to end-user.

It would be crazy if they can't make this work.

That's why robots make so much traffic now. Those other companies are trying to get data.

Google theoretically has reddit access. I wonder if they have sort of an internet archive - data unpolutted by LLMs

On a side note, funny how all the companies seem to train on book archivr which they just downloaded from the internet

> all of the videos, [...], all of the user's search interest, ads, everything..

And privacy policies that are actually limiting what information gets used in what.

and even then!

That's one person's opinion that works for Google.

counterpoint: with their aggressive crawlers, most AI companies can have as much data as google...

> This is not really true. Google has all the compute but in many dimensions they lag behind GPT-5 class (catching up, but it has not been a given).

I don't know what you are talking about. I use Gemini on a daily basis and I honestly can't tell a difference.

We are at a point where training corpus and hallucinations makes more of a difference than "model class".

Yes, or Apple who with all the talent don’t manage to pull off anything useful in AI

xAI seems to be the exception, not the rule

Given Apple’s moat is their devices, their particular spin on AI is very much edge focussed, which isn’t as spectacular as the current wave of cloud based LLM. Apple’s cloud stuff is laughably poor.

Depending on how you look at it I suppose but I believe Gemini surpasses OpenAI on many levels now. Better photo and video models. The leaderboard for text and embeddings are also putting Google on top of Openai.

gemini-2.5-pro is ranked number 1 in llmarena (https://lmarena.ai/leaderboard) before gpt-5-high. In the Text-to-Video and Image-to-video, google also have the highest places, OpenAI is nowhere.

Yes, but they're also slower. As LLMs start to be used for more general purpose things, they are becoming a productivity bottle-neck. If I get a mostly right answer in a few seconds that's much better than a perfect answer in 5 minutes.

Right now the delay for Google's AI coding assistant is high enough for humans to context switch and do something else while waiting. Particularly since one of the main features of AI code assistants is rapid iteration.

Anecdotally, Gemini pro is way faster than GPT 5 thinking. Flash is even faster. I have no numbers though.

It doesn’t guarantee success, but the point stands about X and Deepseek

I think I'm right in saying that AWS, rather than deliveries, is by far the most profitable part of Amazon.

Also a smart move is to be selling shovels in a gold rush - and that's exactly what Amazon is doing with AWS.

Amazon retail runs on ridiculously low margins compared to AWS. Revenue-wise retail dwarfs AWS, profit-wise it's vice-versa.

The barriers to entry for LLM are obvious: as you pointed, the field is extremely capital intensive. The only reason there are seemingly multiple players is because the amount of capital thrown at it at the moment is tremendous but that's unlikely to last forever.

From my admittely poorly informed point of view, strategy-wise, it's hard to tell how wise it is investing in foundational work at the moment. As long as some players release competitive open weight models, the competitive advantage of being a leader in R&D will be limited.

Amazon already has the compute power to place itself as a reseller without investing or having to share the revenue generated. Sure, they won't be at the forefront but they can still get their slice of the pie without exposing themselves too much to an eventual downturn.

The idea that models are copyrightable is also extremely dubious.

So there probably isn’t even a legal moat.

There's not much of an architectural moat, but there is a methodological moat, such as with RL synthetic data.

Are you arguing anthropic has more compute than Amazon?

Are you saying the only reason Meta is behind everyone else is compute????

Think well: why should a platform provider get into a terribly expensive and unprofitable business when they can just provide hardware for those with money to spend? This was AWS strategy for years and it's been working well for them.

> Are you arguing anthropic has more compute than Amazon?

I wouldn't be surprised if the likes of Anthropic wasn't paying AWS for its compute.

As the saying goes, the ones who got rich from the gold rush were the ones selling shovels.

I wouldn't be surprised if Amazon just buys Anthropic or another lab rather than competing for individuals.

Reply on news.ycombinator.com