Hacker News Reader

Comment by cedws | Hacker News Reader

A lot of C-suite people seem to have an idea that if they just throw enough compute at LLMs that AGI will eventually emerge, even though it's pretty clear at this point that LLMs are never going to lead to general intelligence. In their view it makes sense to invest massive amounts of capital because it's like a lottery ticket to being the future AGI company that dominates the world.

I recall Zuckerberg saying something about how there were early signs of AI "improving itself." I don't know what he was talking about but if he really believes that's true and that we're at the bottom of an exponential curve then Meta's rabid hiring and datacenter buildout makes sense.

View on HN

In early 2023, I remember someone breathlessly explaining that there are signs that LLMs that are seemingly good at chess/checkers moves may have a rudimentary model of the board within them, somehow magically encoded into the model weights through the training. I was stupid enough to briefly entertain the possibility until I actually bothered to develop a high level understanding of the transformer architecture. It's surprising how much mysticism this field seems to attract. Perhaps it being a non-deterministic, linguistically invoked black box, triggers the same internal impulses that draw some people to magic and spellcasting.

Just because it's not that hard to reach a high-level understanding of the transformer pipeline doesn't mean we understand how these systems function, or that there can be no form of world model that they are developing. Recently there has been more evidence for that particular idea [1]. The feats of apparent intelligence LLMs sometimes display have taken even their creators by surprise. Sure, there's a lot of hype too, that's part and parcel of any new technology today, but we are far from understanding what makes them perform so well. In that sense, yeah you could say they are a bit "magical".

[1] https://the-decoder.com/new-othello-experiment-supports-the-...

> Just because it's not that hard to reach a high-level understanding of the transformer pipeline doesn't mean we understand how these systems function

Mumbo jumbo magical thinking.

They perform so well because they are trained on probabilistic token matching.

Where they perform terribly, e.g math, reasoning, they are delegating to other approaches, and that's how you get the illusion that there is actually something there. But it's not. Faking intelligence is not intelligence. It's just text generation.

> In that sense, yeah you could say they are a bit "magical"

Nobody but the most unhinged hype pushers are calling it "magical". The LLM can never ever be AGI. Guessing the next word is not intelligence.

> there can be no form of world model that they are developing

Kind of impossible to form a world model if your foundation is probabilistic token guessing which is what LLMs are. LLMs are a dead end in achieving "intelligence", something novel as an approach needs to be discovered (or not) to go into the intelligence direction. But hey, at least we can generate text fast now!

> LLMs are a dead end in achieving "intelligence"

There is no evidence to indicate this is the case. To the contrary, all evidence we have points to these models, over time, being able to perform a wider range of tasks at a higher rate of success. Whether it's GPQA, ARC-AGI or tool usage.

> they are delegating to other approaches > Faking intelligence is not intelligence. It's just text generation.

It seems like you know something about what intelligence actually is that you're not sharing. If it walks, talks and quacks like a duck, I have to assume it's a duck[1]. Though, maybe it quacks a bit weird.

[1] https://en.wikipedia.org/wiki/Solipsism

> There is no evidence to indicate this is the case

Burden of proof is on those trying to convince us to buy into the idea of LLMs as being "intelligence".

There is no evidence of the Flying Spaghetti monster or Zeus or God not existing either, but we don't take seriously the people who claim they do exist (and there isn't proof because these concepts are made up).

Why should we take seriously the tolks claiming LLMs are intelligence without proof (there can't be proof, of course, because LLMs are not intelligence)?

I'm not a fan of mysticism. I'm also with you that these are simply statistical machines. But I don't understand what happened when understood transformers at a high-level.

If you're saying the magic disappeared after looking at a single transformer, did the magic of human intelligence disappear after you understood human neurons at a high level?

Is there something we are all missing? Using Claude feels like magic sometimes, but can't everyone see the limitation now that we are 4 years and 100s of billions down the road?

Are they still really hoping that they are gonna tweak a model and feed it an even bigger dataset and it will be AGI?

Its insane really, anyone who has worked with LLMs for a bit and has an idea of how they work shouldn't think its going to lead to "AGI".

Hopefully some big players, like FB bankrupt themselves.

Just scaling them up might not leat to "AGI", but they can still lead to AGI as a bridge.

Is it widely accepted that LLMs won't lead to AGI? I've asked Gemini, so it came up with four primary arguments for this claim, commenting on them briefly:

1) LLMs as simple "next token predictors" so they just mimicry thinking: But can it be argued that current models operate on layers of multiple depth and are able to actually understand by building concepts and making connections on abstract levels? Also, don't we all mimicry?

2) Grounding problem: Yes, models build their world models on text data, but we have models operating on non-textual data already, so this appears to be a technical obstacle rather than fundamental.

3) Lack of World Model. But can anyone really claim they have a coherent model of reality? There are flat-earthers, yet I still wouldn't deny them having AGI. People hallucinate and make mistakes all the time. I'd argue hallucinations is in fact the sign of an emerging intelligence.

4) Fixed learning data sets. Looks like this is now being actively solved with self-improving models?

I just couldn't find a strong argument supporting this claim. What am I missing?

Why on earth would you copy and paste an LLM's output into a comment? What does that accomplish or provide that just a simply stated argument doesn't accomplish more succinctly? If you don't know something, simply don't comment on it -- or ask a question.

None of the above is AI.

> I've asked Gemini, so it came up with four primary arguments for this claim, commenting on them briefly:

This line means, and literally says, that everything that follows is a summary or direct quotation from an LLM's output.

There's a more charitable but unintuitive interpretation, in which "commenting on them briefly" is intended to mean "I will comment on them briefly:". But this isn't a natural interpretation. It's one I could be expected to reach only after seeing your statement that 'none of the above is AI.' But even this more charitable interpretation actually contradicts your claim that it's not AI.

So now I'm even less sure I know what you meant to communicate. Either I'm missing something really obvious or the writing doesn't communicate what you intended.

Fur future reference, pasting llm slop feels exactly as patronizing as back when people pasted links to google searches in response to questions they considered beneath their dignity to answer. Except in this case, no-one asked to begin with.

The only way we'll have AGI is if people get dumber. Using modern tech like LLMs makes people dumber. Ergo, we might see AGI sooner than expected.

> ... and has an idea of how they work shouldn't think its going to lead to "AGI"

Not sure what level of understanding are you referring to but having learned and researched about the pretty much all LLM internals I think this has led me exactly to the opposite line of thinking. To me it's unbelievable what we have today.

Tbh I find this view odd, and I wonder what people view as agi now. It used to be that we had extremely narrow pieces of AI and I remember being on a research project about architectures and just very basic “what’s going on?” was advanced. Understanding that someone asked a question, that would be solved by getting a book and being able to then go and navigate to the place the book was likely to be was fancy. Most systems could solve literally one type of problem. They weren’t just bad at other things they were fundamentally incapable of anything but an extremely narrow use case.

I can throw wide ranging problems at things like gpt5 and get what seem like dramatically better answers than if I asked a random person. The amount of common sense is so far beyond what we had it’s hard to express. It used to be always pointed out that the things we had were below basic insect level. Now I have something that can research a charity, find grants and make coherent arguments for them, read matrix specs and debug error messages, and understand sarcasm.

To me, it’s clear that agi is here. But then what I always pictured from it may be very different to you. What’s your image of it?

I think the discrepancy between different views on the matter mainly stems from the fact that state-of-the-art LLMs are better (sometimes extremely better) at some tasks, and worse (sometimes extremely worse) at other tasks, compared to average humans. For example, they're better at retrieving information from huge amounts of unstructured data. But they're also terrible at learning: any "experience" which falls out of the context window is lost forever, and the model can't learn from its mistakes. To actually make it learn something requires very many examples and a lot of compute, whereas a human can permanently learn from a single example.

> human can permanently learn from a single example

This, to me at least, seems like an important ingredient to satisfying a practical definition / implementation of AGI.

Another might be curiosity, and I think perhaps also agency.

Nobody is saying that LLM's don't work like magic. I know how neural networks work and they still feel like voodoo to me.

What we are saying is that LLM's can't become AGI. I don't know what AGI will look like, but it won't look like an LLM.

There is a difference between being able to melt iron and being able to melt tungsten.

I think it's clear that nobody agrees what AGI is. OpenAI describes it in terms of revenue. Other people/orgs in terms of, essentially, magic.

If I had to pick a name, I'd probably describe ChatGPT & co as advanced proof of concepts for general purpose agents, rather than AGI.

> I think it's clear that nobody agrees what AGI is

People selling AI products are incentivized to push misleading definitions of AGI.

Human-level intelligence. Being able to know what it doesn't know. Having a practical grasp on the idea of truth. Doing math correctly, every time.

I give it a high-res photo of a kitchen and ask it to calculate the volume of a pot in the image.

You discover truth by doing stuff in real world and observing the results. Current LLM have enough intelligence, but all the inputs they have are the “he said she said” by us monkeys, including all omissions and biases.

None of what you describe would I label within the realm of 'average'

It's not about what the average human can do - it's about what humans as a category are capable of. There will always be outliers (in both directions), but you can, in general, teach a human a variety of tasks, such as performing arithmetic deterministically, that you cannot teach to, for example, a parrot.

But many humans can't do a lot of those things and we still consider them "generally intelligent"

It's more that "random" people are dumb as bricks (but we've in the name of equality and historic measurement errors decided to forgo that), add to it that AI's have a phenomenal (internet sized) memory makes them far more capable than many people.

However, even "dumb" people can often make judgements structures in a way that AI's cannot, it's just that many have such a bad knowledge-base that they cannot build the structures coherently whereas AI's succeed thanks to their knowledge.

I wouldn't be surprised if the top AI firms today spend an inordinate amount of time to build "manual" appendages into the LLM systems to cater to tasks such as debugging to uphold the facade that the system is really smart, while in reality it's mostly papering up a leaky model to avoid losing the enormous investments they need to stay alive with a hope that someone on their staff comes up a real solution to self-learning.

https://magazine.sebastianraschka.com/p/understanding-reason...

my picture of AGI is 1) autonomous improvement 2) ability to say 'i don't know/can't be done'

I wonder if 2) is a result of published bias for positive results in the training set. An “I don’t know” response is probably ranked unsatisfactory by human feedback and most published scientific literature are biased towards positive results and factual explanations.

In my experience, the willingness to say "I don't know" instead of confabulate is also down-rated as a human attribute, so it's not surprising that even an AGI trained on the "best" of humanity would avoid it.

I don’t have a very high expectation of AGI at all. Just an algorithm or system you can put onto a robot dog, and get a dog level general intelligence. You should be able to live with that robot dog for 10 years and it should be just as capable as a dog throughout that timespan.

Hell, I’d even say we have AGI if you could emulate something like a hamster.

LLMs are way more impressive in certain ways than such a hypothetical AGI. But that has been true of computers for a long time. Computers have been much better at Chess than humans for decades. Dogs can’t do that. But that doesn’t mean that a chess engine is an AGI.

I would also say we have a special form of AGI if the AI can pass an extended Turing test. We’ve had chat bots that can fool a human for a minute for a long time. Doesn’t mean we had AGI. So time and knowledge was always a factor in a realistic Turing test. If an AGI can fool someone who knows how to properly probe an LLM, for a month or so, while solving a bunch of different real world tasks that require stable long term memory and planning, then I’d day we’re in AGI territory for language specifically. I think we have to distinguish between language AGI and multi-modal AGI. So this test wouldn’t prove what we could call “full” AGI.

These are some of the missing components for full AGI: - Being able to act as a stable agent with a stable personality over long timespans - Capable of dealing with uncertainties. Having a understanding of what it doesn’t know - One-shot learning, with long term retention, for a large number of things - Fully integrated multi-modality across sound, vision, and other inputs/outputs we may throw at it.

The last one is where we may be able to get at the root of the algorithm we’re missing. A blind person can learn to “see” by making clicks and using their ears to see. Animals can do similar “tricks”. I think this is where we truly see the full extent of the generality and adaptability of the biological brain. Imagine trying to make a robot that can exhibit this kind of adaptability. It doesn’t fit into the model we have for AI right now.

Recently I realized that US are very close to a centrally planned economy. Meta wasted 50B on metaverse, which like how much Texas spends on healthcare. Now the "AI" investments seems dubious.

You could fund 1000+ projects with this kinds of money. This is not an effective capital allocation.

This is not and has not been the consensus opinion. If you're not an AI researcher you shouldn't write as if you've set your confidence parameter to 0.95.

Of course it might be the case, but it's not a thing that should be expressed with such confidence.

I think AI research is like anything else really. The smartest people are heads down working on their problems. The people going on podcasts are less connected to day to day work.

It’s also pretty useless to talk about whether something is AGI without defining intelligence in the first place.

I think something like we saw in the show "Devs" is much more likely, although what the developers did with it in the show was bonkers unrealistic. But some kind of big enough quantum device basically.

> I don't know what he was talking about

There's a bunch of ways AI is improving itself, depending on how you want to interpret that. But it's been true since the start.

1. AI is used to train AI. RLHF uses this, curriculum learning is full of it, video model training pipelines are overflowing with it. AI gets used in pipelines to clean and upgrade training data a lot.

2. There are experimental AI agents that can patch their own code and explore a tree of possibilities to boost their own performance. However, at the moment they tap out after getting about as good as open source agents, but before they're as good as proprietary agents. There isn't exponential growth. There might be if you throw enough compute at it, but this tactic is very compute hungry. At current prices it's cheaper to pay an AI expert to implement your agent than use this.

> There are experimental AI agents that can patch their own code and explore a tree of possibilities to boost their own performance. However, at the moment they tap out after getting about as good as open source agents, but before they're as good as proprietary agents.

Interesting. Do you have links?

Not OP, but the Darwin Godel machine comes to mind: https://arxiv.org/abs/2505.22954

That's the one!

Thanks!

I'm skeptical that RLHF really works. Doesn't it just patch the obvious holes so it looks better on paper? If it can't reason then it will continue to get 2nd and 3rd order difficulty problems wrong.

I think LLM as a toolsmith like demonstrated in the Voyager paper (1) is another interesting approach to creating a system that can learn to do a task better over time. (1) https://arxiv.org/abs/2305.16291

So have an AI with a 40% error rate judge an AI with an 40% error rate…

AGI is a complete no go until a model can adjust its own weights on the fly, which requires some kind of negative feedback loop, which requires a means to determine a failure.

Humans have pain receptors to provide negative feedback and we can imagine events that would be painful such as driving into a parked car would be painful without having to experience it.

If current models could adjust its own weights to fix the famous “how many r’s in strawberry” then I would say we are on the right path.

However, the current solution is to detect the question and forward it to a function to determine the right answer. Or attempt to add more training data the next time the model is generated ($$$). Aka cheat the test.

> it's pretty clear at this point that LLMs are never going to lead to general intelligence.

It is far from clear. There may well be emergent hierarchies of more abstract thought at much higher numbers of weights. We just don't know how a transformer will behave if one is built with 100T connections - something that would finally approach the connectome level of a human brain. Perhaps nothing interesting but we just do not know this and the current limitation in building such a beast is likely not software but hardware. At these scales the use of silicon transistors to approximate analog curve switching models just doesn't make sense. True neuromorphic chips may be needed to approach the numbers of weights necessary for general intelligence to emerge. I don't think there is anything in production at the moment that could rival the efficiency of biological neurons. Most likely we do not need that level of efficiency. But it's almost certain that stringing together a bunch of H100s isn't a path to the scale we should be aiming for.

I don't get it, I really don't.

Even assuming a company gets to AGI first this doesn't mean another one will follow.

Suppose that FooAI gets to it first: - competitors may get there too in a different or more efficient way - Some FooAI staff can leave and found their own company - Some FooAI staff can join a competitor - FooAI "secret sauce" can be figured out, or simply stolen, by a competitor

At the end of the day, it really doesn't matter, the equation AI === commodity just does not change.

There is no way to make money by going into this never ending frontier model war, price of training keeps getting higher and higher, but your competitors few months later can achieve your own results for a fraction of your $.

Some would say that the race to AGI is like the race to nuclear weapons and that the first to get there will hold all the cards (and be potentially able to stop others getting there.) It's a bit too sci-fi for me.

If AGI is reached it would be trivial for the competing superpowers to completely quarantine themselves network wise by cutting undersea cables long enough to develop competing AGI

I don't know if AGI will emerge from LLM, but I'm always reminded of the Chinese room thought experiment. With billions thrown at the idea it will certainly be the ultimate answer as to whether true understanding can emerge from a large enough dictionary.

Please stop refering to the Chinese Room - it's just magical/deist thinking in disguise. It postulates that humans have way of 'understanding' things that is impossible to replicate mechanically.

The fact that philosophy hasn't recognized and rejected this argument based on this speaks volumes of the quality of arguments accepted there.

(That doesn't mean LLMs are or will be AGI, its just this argument is tautological and meaningless)

It is still relevant because it hasn’t been disproven yet. So far all computer programs are Chinese Rooms, LLM’s included.

If you’re talking about it being proven or disproven you’re misunderstanding the point of the thought experiment.

That some people use the Chinese Room to ascribe some magical properties to human consciousness says more about the person drawing that conclusion than the thought experiment itself.

I think it's entirely valid to question whether a computer can form an understanding through deterministically processing instructions, whether that be through programming language or language training data.

If the answer is no, that shouldn't lead to a deist conclusion. It can just as easily lead to the conclusion that a non-deterministic Turing machine is required.

I'd appreciate if you tried to explain why instead of resorting to ad hominem.

> I think it's entirely valid to question whether a computer can form an understanding through deterministically processing instructions, whether that be through programming language or language training data.

Since the real world (including probabilistic and quantum phenomena) can be modeled with deterministic computation (a pseudorandom sequence is deterministic, yet simulates randomness), if we have a powerful enough computer we can simulate the brain to a sufficient degree to have it behave identically as the real thing.

The original 'Chinese Room' experiment describes a book of static rules of Chinese - which is probably not the way to go, and AI does not work like that. It's probabilistic in its training and evaluation.

What you are arguing is that constructing an artificial consciousness lies beyond our current computational ability(probably), and understanding of physics (possibly), but that does not rule out that we might solve these issues at some point, and there's no fundamental roadblock to artificial consciousness.

I've re-read the argument (https://en.wikipedia.org/wiki/Chinese_room) and I cannot help but conclude that Searle argues that 'understanding' is only something that humans can do, which means that real humans are special in some way a simulation of human-shaped atoms are not.

Which is an argument for the existence of the supernatural and deist thinking.

> I'd appreciate if you tried to explain why instead of resorting to ad hominem.

It is not meant as an ad hominem. If someone thinks our current computers can't emulate human thinking and draws the conclusion that therefore humans have special powers given to them by a deity, then that probably means that person is quite religious.

I'm not saying you personally believe that and therefore your arguments are invalid.

> Since the real world (including probabilistic and quantum phenomena) can be modeled with deterministic computation (a pseudorandom sequence is deterministic, yet simulates randomness), if we have a powerful enough computer we can simulate the brain to a sufficient degree to have it behave identically as the real thing.

The idea that a sufficiently complex pseudo-random number generator can emulate real-world non-determinism enough to fully simulate the human brain is quite an assumption. It could be true, but it's not something I would accept as a matter of fact.

> I've re-read the argument (https://en.wikipedia.org/wiki/Chinese_room) and I cannot help but conclude that Searle argues that 'understanding' is only something that humans can do, which means that real humans are special in some way a simulation of human-shaped atoms are not.

In that same Wikipedia article Searle denies he's arguing for that. And even if he did secretly believe that, it doesn't really matter, because we can draw our own conclusions.

Disregarding his arguments because you feel he holds a hidden agenda, isn't that itself an ad hominem?

(Also, I apologize for using two accounts, I'm not attempting to sock puppet)

What are his arguments then?

>Searle argues that, without "understanding" (or "intentionality"), we cannot describe what the machine is doing as "thinking" and, since it does not think, it does not have a "mind" in the normal sense of the word.

This is the only sentence that seems to be pointing to what constitutes the specialness of humans, and the terms of 'understanding' and 'intentionality' are in air quotes so who knows? This sounds like the archetypical no true scotsman fallacy.

In mathematical analysis, if we conclude that the difference between 2 numbers is smaller than any arbitrary number we can pick, those 2 numbers must be the same. In engineering, we can reduce the claim to 'any difference large about to care about'

Likewise if the difference between a real human brain and an arbitrarily sophisticated Chinese Room brain is arbitrarily small, they are the same.

If our limited understanding of physics and engineering makes the practical difference not zero, this essentially becomes a bit of a somewhat magical 'superscience' argument claiming we can't simulate the real world to a good enough resolution that the meaningful differences between our 'consciousness simulator' and the thing itself disappear - which is an extraordinary claim.

> What are his arguments then?

They're in the "Complete Argument" section of the article.

> This sounds like the archetypical no true scotsman fallacy.

I get what you're trying to say, but he is not arguing only a true Scotsman is capable of thought. He is arguing that our current machines lack the required "causal powers" for thought. Powers that he doesn't prescribe to only a true Scotsman, though maybe we should try adding bagpipes to our AI just to be sure...

Thanks, but that makes his arguments even less valid.

He argues that computer programs only manipulate symbols and thus have no semantic understanding.

But that's not true - many programs, like compilers that existed back when the argument was made, had semantic understanding of the code (in a limited way, but they did have some understanding about what the program did).

LLMs in contrast have a very rich semantic understanding of the text they parse - their tensor representations encode a lot about each token, or you can just ask them about anything - they might not be human level at reading subtext, but they're not horrible either.

Now you're getting to the heart of the thought experiment. Because does it really understand the code or subtext, or is it just really good at fooling us that it does?

When it makes a mistake, did it just have a too limited understanding or did it simply not get lucky with its prediction of the next word? Is there even a difference between the two?

I would like to agree with you that there's no special "causal power" that Turing machines can't emulate. But I remain skeptical, not out of chauvinism, but out of caution. Because I think it's dangerous to assume an AI understands a problem simply because it said the right words.

> I cannot help but conclude that Searle argues that ‘understanding’ is only something that humans can do, which means…

Regardless of whether Searle is right or wrong, you’ve jumped to conclusions and are misunderstanding his argument and making further assumptions based on your misunderstanding. Your argument is also ad-hominem by accusing people of believing things they don’t believe. Maybe it would be prudent to read some of the good critiques of Searle before trying to litigate it rapidly and sloppily on HN.

The randomness stuff is very straw man, definitely not a good argument, best to drop it. Today’s LLMs are deterministic, not random. Pseudorandom sequences come in different varieties, but they model some properties of randomness, not all of them. The functioning of today’s neural networks, both training and inference, is exactly a book of static rules, despite their use of pseudorandom sequences.

In case you missed it in the WP article, most of the field of cognitive science thinks Searle is wrong. However, they’re largely not critiquing him for using metaphysics, because that’s not his argument. He’s arguing that biology has mechanisms that binary electronic circuitry doesn’t; not human brains, simply physical chemical and biological processes. That much is certainly true. Whether there’s a difference in theory is unproven. But today currently there absolutely is a difference in practice, nobody has ever simulated the real world or a human brain using deterministic computation.

If scientific consensus is that he's wrong why is he being constantly brought up and defended - am I not right to call them out then?

Nobody brings up that light travels through the aether, that diseases are caused by bad humors etc. - is it not right to call out people for stating theory that's believed to be false?

>The randomness stuff is very straw man,

And a direct response to what armada651 wrote:

>I think it's entirely valid to question whether a computer can form an understanding through deterministically processing instructions, whether that be through programming language or language training data.

> He’s arguing that biology has mechanisms that binary electronic circuitry doesn’t; not human brains, simply physical chemical and biological processes.

Once again the argument here changed from 'computers which only manipulate symbols cannot create consciousness' to 'we don't have the algorithm for consiousness yet'.

He might have successfully argued against the expert systems of his time - and true, mechanistic attempts at language translation have largely failed - but that doesn't extend to modern LLMs (and pre LLM AI) or even statistical methods.

You’re making more assumptions. There’s no “scientific consensus” that he’s wrong, there are just opinions. Unlike the straw man examples you bring up, nobody has proven the claims you’re making. If they had, then the argument would go away like the others you mentioned.

Where did the argument change? Searle’s argument that you quoted is not arguing that we don’t have the algorithm yet. He’s arguing that the algorithm doesn’t run on electrical computers.

I’m not defending his argument, just pointing out that yours isn’t compelling because you don't seem to fully understand his, at least your restatement of it isn’t a good faith interpretation. Make his argument the strongest possible argument, and then show why it doesn’t work.

IMO modern LLMs don’t prove anything here. They don’t understand anything. LLMs aren’t evidence that computers can successfully think, they only prove that humans are prone to either anthropomorphic hyperbole, or to gullibility. That doesn’t mean computers can’t think, but I don’t think we’ve seen it yet, and I’m certainly not alone there.

>most of the field of cognitive science thinks Searle is wrong.

>There’s no “scientific consensus” that he’s wrong, there are just opinions.

And? Are you imagining that these aren’t both true at the same time? If so, I’m happy to explain. Since nothing has been proven, there’s nothing “scientific”. And since there’s some disagreement, “consensus” has not been achieved yet. This is why your presumptive use of “scientific consensus” was not correct, and why the term “scientific consensus” is not the same thing as “most people think”. A split of 60/40 or 75/25 or even 90/10 counts as “most” but does not count as “consensus”. So I guess maybe be careful about assuming what something means, it seems like this thread was limited by several incorrect assumptions.

The human way of understanding things can be replicated mechanically, because it is mechanical in nature. The contents of your skull are an existence proof of AGI.

The contents of my skull are only a proof for AGI if your mechanical machine replicates all its processes. It's not a question about whether a machine can reproduce that, it's a question about whether we have given our current machines all the tools it needs to do that.

The theory of special relativity does not say 'you can't exceed the speed of light(unless you have a really big rocket)'. It presents a theoretical limit. Likewise the Chinese room doesn't state that consciousness is an intractable engineering problem, but an impossibility.

But the way Searle formulates his argument, by not defining what consciousness is, he essentially gives himself enough wiggle room to be always right - he's essentially making the 'No True Scotsman' fallacy.

The A stands for artificial.

"Please stop referring to this thought experiment because it has possible interpretations I don't personally agree with"

Please give me an interpretation that is both correct an meaningful (as in possible to disprove)

> The fact that philosophy hasn't recognized and rejected this argument based on this speaks volumes of the quality of arguments accepted there.

That's one possibility. The other is that your pomposity and dismissiveness towards the entire field of philosophy speaks volumes on how little you know about either philosophical arguments in general or this philosophical argument in particular.

Another ad hominem, I'd like you to refute my claim that Searle's argument is essentially 100% magical thinking.

And yes, if for example, medicine would be no worse at curing cancer than it is today, yet doctors asserted that crystal healing is a serious study, that would reflect badly on the field at large, despite most of it being sound.

Searle refutes your claim that there’s magical thinking.

“Searle does not disagree with the notion that machines can have consciousness and understanding, because, as he writes, "we are precisely such machines". Searle holds that the brain is, in fact, a machine, but that the brain gives rise to consciousness and understanding using specific machinery.”

But the core of the original argument is that programs only manipulate symbols and consciousness can never arise just through symbol manipulation - which here then becomes 'we have not discovered the algorithms' for consciousness yet.

It's just a contradiction.

When you say something that contradicts his statements, it doesn’t mean he’s wrong, it most likely means you haven’t understood or interpreted his argument correctly. The Wikipedia page you linked to doesn’t use the word “algorithm”, so the source of the contradiction you imagine might be you. Searle says he thinks humans are biological machines, so your argument should withstand that hypothesis rather than dismiss it.

Another ad hominem, just like you calling anyone who talks about the chinese room thought experiment a deist?

Why on earth do you take it as an ad hominem attack? Do you really think your comment isn't dismissive or pompous?

Reply on news.ycombinator.com