[flagged]
Btw, it's particularly important not to do this when your argument is the correct one, since if you happen to be right, you end up discrediting the truth by posting like this, and that ends up hurting everybody. https://hn.algolia.com/?dateRange=all&page=0&prefix=true&sor...
If you'd please review https://news.ycombinator.com/newsguidelines.html and stick to the rules when posting here, we'd appreciate it.
I agree with your comment, FWIW - I have no idea what OP is trying to demonstrate - but to maybe suggest some context: Gödel incompleteness is a commonly suggested "proof" as to why computers can't be intelligent in the way that humans can, because (very very roughly) humans can "step out" of formal systems to escape the Gödel trap. Regardless of your feelings about AI it's a silly argument; I think possibly the popularizer of this line of thinking was Roger Penrose in "The Emperor's New Mind".
I haven't re-read Searle since college but as far as I recall he never brings it up.
As for intentionality, programs have intentionality.
As for its alleged conclusion, I love David Chalmers’ parody “Cakes are crumbly, recipes are syntactic, syntax is not sufficient for crumbliness, so implementation of a recipe is not sufficient for making a cake.”
This is indeed true, but we should bear in mind that Chalmers' parody is just that: a parody, not a rigorous argument. It seems clear that, if Chalmers wanted to make it more rigorous, he would have concluded with something like "therefore, even if you have all the prerequisites for baking a cake (ingredients, tools, familiarity with basic cooking operations...), no recipe is sufficient to instruct you in successfully completing the task." This would be a better argument, but a flabbier, less to-the-point parody, and it is reasonable for Chalmers to leave it to his readers to get his point.
I'm still coming to grips with the idea that LLMs seem to translate pretty well without understanding anything.
The most honest answer to these questions I can give is to say "I don't know", though I'm toying with the idea that they understand (in some sense) the pragmatics of language use, but not that language refers to an external world which changes according to rules and causes that are independent of what we can and do say about it. This would be a very strange state to be in, and I cannot imagine what it would be like to be in such a state. We have never met anybody or anything like it.
The thing is, minds interact with the outside world through information transmitted on the peripheral neural system, so the latter analogy is the relevant one here.
You also claim that I am begging the question. How do you justify that? It is not, of course, begging the question for opponents of Searle to suppose his conclusion is wrong: everyone disputing any argument does that.
The Enigma response is very straightforward. While, in general, simulations are not equivalent to what is being simulated, it is often the case that for information manipulation they are, owing to the substrate independence of information. It is Searle who needs a better argument here, and he never came up with one.
The first thing to note is that it is not necessary to dispute the notion that semantics come from a conscious observer in order to demonstrate that Searle's argument fails, as that argument is precisely about whether that conscious observer could be an entity deriving its consciousness from a digital computation. As things stand, the only thing saving you from formally begging the question is that you have still not presented a specific argument against my objection[1] to Searle's response to the simulation reply; you still seem to be trying to insinuate that it fails without being specific.
Maybe you also mistakenly think that what I am saying in this thread is supposed to be an argument for computationalism? It is just an argument that Searle failed to make his case against computationalism on account of (among other things) an unsuccessful response to the simulation reply. (I suspect that computationalism is essentially correct, but I do not claim to know that it is.)
[1] It's not just my objection; Dennett, the Churchlands, even Putnam and David "hard problem" Chalmers have raised similar objections.
Edit: it looks like your account has been breaking the site guidelines quite a bit. That's not good, we ban accounts that do this, and we already asked you once before: https://news.ycombinator.com/item?id=34420693. If you wouldn't mind reviewing https://news.ycombinator.com/newsguidelines.html and taking the intended spirit of the site more to heart, we'd be grateful.
And it does kind of help explain why LLMs sound like bright students who completely memorized their way through school.
It's that last part that I can't follow and (so far) totally disbelieve.
You have put your finger on the fundamental problem of the argument: Searle never gave a good justification for the tacit and question-begging premise that if the human operator did not understand the language, then nothing would (there is a second tacit premise at work here: that performing the room's task required something to understand the language. LLMs arguably suggest that the task could be performed without anything resembling a human's understanding of language.)
Searle's attempt to justify this premise (the 'human or nothing' one) against the so-called 'systems reply' is to have the operator memorize the book, so that the human is the whole system. Elsewhere [1] I have explained why I don't buy this.
Yeah. I used to assume that. But it's much less obvious now. Or just false or something.
It's actually kind of spooky how well Searle did capture/foreshadow something about LLMs decades ago. No part of the system seems to understand much of anything.
My theory is that Searle came up with the CR while complaining to his wife (for the hundredth time) about bright undergrads who didn't actually understand anything. She finally said "Hey, you should write that down!" Really she just meant "holy moly, stop telling it to me!" But he misunderstood her, and the rest is history.