Hacker News Reader

Comment by prophesi | Hacker News Reader

I'm guessing whether you believe it possesses intelligence or not depends on your answer to Searle's Chinese room thought experiment[0]. I'd also recommend checking out the Peter Watts' book, Blindsight.

[0] https://en.wikipedia.org/wiki/Chinese_room

View on HN

The Chinese room is a good Rorschach test for this kind of thing (but not a good thought experiment, IMO, because it's obviously correct or obviously wrong depending on where you're already coming from), but also it's not really about intelligence per se, but more abstractly awareness and more adjacent to consciousness than intelligence, and these are not the same thing (though it does seem like a lot of people have conflated them somehow, from the conversations around AI).

It all comes down to semantics. And yeah, with the thought experiment, Searle presents three axioms of what could constitute intelligence. Then proposes the chinese room thought experiment as an approval of his third rule, "Syntax by itself is neither constitutive of nor sufficient for semantics." Which only makes sense when you take the other two rules together. Oddly enough, defining intelligence with our capability to derive semantics.

There are a lot of creative counter-arguments to look into on the thought experiment though.

The thing is it doesn't really need creative counter-arguments, it's basically just assuming its conclusion. If you disagree with the conclusion, the argument is nonsense because Searle is just saying 'well the man doesn't understand, so nothing does', and if you agree with the conclusion you don't really need to jump through any hoops to get to it, the third rule is just obviously following from the first and second. And again it's not really talking about intelligence at all, Searle allows that the machine is intelligent from the start, he just rejects that it has any understanding of meaning.

Is that not one of the counter-arguments? You're disagreeing of what Searle determines to be "intelligence." We don't understand how consciousness works, what it's like to be a bat, or how this emergent property came to be. It's why it's necessary to define the terms, then prove them wrong or right. If you disagree with the definitions, that's fine. There is no agreement on what constitutes intelligence.

Again, you keep mentioning intelligence but you talk more like you're talking about awareness/'understanding'/semantics. I agree that these are tricky things to define precisely but there is a reason they are different words. It's also odd considering you mentioned 'blindsight' which is a whole book about the idea that you could have intelligence but not awareness (something I am not sure is a coherent concept, personally).

And yeah, part of it does seem to stem from disagreement about definitions. To me Searle seems to assume much more in his definitions as obvious than he explicitly states, which is why the Chinese room seems like such a non-argument from my point of view.

> Again, you keep mentioning intelligence but you talk more like you're talking about awareness/'understanding'/semantics.

Yes, if you disagree with the semantics defined by Searle (the three axioms), then you can disregard the argument.

This comment thread was started with discussions of AI doing a bad job at a task (communication).

Doesn't the Chinese Room posit an AI good at the task of communication?

The Chinese Room mainly just posits a room that passes the Turing Test, which LLMs do pretty well outside of outright adversarial situations.

Do they? https://longbets.org/1/ has yet to be settled. Either way, I doubt an LLM could fool anyone here who who knows how LLMs work into thinking it is human, at least not for an extended period of time (think about context length/compression, prompt injections, …).

You'll notice those goalposts are substantially stretched from the original test.

How so?

AIs are better communicators that most of people I have worked with in my life.

They are infinitely patient, don't mind going into more detail if I ask, not too bad at summary, have no ego and don't boast. They are also not too afraid of hurting my feelings, they will tell me my code sux if it does.

I'd don't care if they fit a definition intelligent, they are good colleagues. They have strengths and weaknesses sure, but so do people.

> have no ego and don't boast

Yes they do, and they famously do it quite a lot.

> they will tell me my code sux if it does

If they knew when code sux, someone should write an agentic loop around that.

Reply on news.ycombinator.com