The fact that you can reason about intelligence is a counter argument to this
The fact that we can provide a chain of reasoning, and we can think that it is about intelligence, doesn't mean that we were actually reasoning about intelligence. This is immediately obvious when we encounter people whose conclusions are being thrown off by well-known cognitive biases, like cognitive dissonance. They have no trouble producing volumes of text about how they came to their conclusions and why they are right. But are consistently unable to notice the actual biases that are at play.
If I ask you to think of a movie, go ahead, think of one.....whatever movie just came into your mind was not picked by you, it was served up to you from an abyss.
We're doing the equivalent of LLM's and making up a plausible explanation for how we came to a conclusion, not reflecting reality.
As one neurologist put it, listening to people's explanations of how they think is entertaining, but not very informative. Virtually none of what people describe correlates in any way to what we actually know about how the brain is organized.
We don't know what intelligence is, or isn't.
Case in point… I didn't write that paragraph by myself.
Someone needs to create a clone site of HN's format and posts, but the rules only permit synthetic intelligence comments. All models pre-prompted to read prolifically, but comment and up/down vote carefully and sparingly, to optimize the quality of discussion.
And no looking at nat-HN comments.
It would be very interesting to compare discussions between the sites. A human-lurker per day graph over time would also be of interest.
Side thought: Has anyone created a Reverse-Captcha yet?
I think the site would clone the upvotes of articles and the ordering of the front page, and gives directions when to comment on other’s posts.
I'm glad you didn't write that paragraph by yourself; I would be concerned on your behalf if you had.
(For those no longer able to follow complex English grammar: Yeah, I exaggerate, but there is no point trying to participate in this kind of discussion if that's the sort of basic error one has to start from, and the especially weird nature of this example of the mistake also points to LLMs synthesizing the result of consciousness rather than experiencing it.)