Hacker News Reader

Comment by saulpw | Hacker News Reader

Does it seem like it bifurcates based on the person? I've had both experiences myself, sometimes with the same model and within ~days of each other on seemingly similar tasks. It's almost impossible to deny sometimes that actual intelligence is being expressed (and could not be regurgitated intelligence from some random internet page), but then I see firsthand the eye-rolling "intelligence-shaped" output from something else and wonder where I went wrong.

It kinda feels like Michigan J Claude sometimes.

View on HN

> It's almost impossible to deny sometimes that actual intelligence is being expressed (and could not be regurgitated intelligence from some random internet page)

But how is it impossible? Or rather, how is it possible to distinguish actual intelligence from some internet page--not "some random internet page", but some very well selected, timely and topical internet page?

Carly Simon's "Killing Me Softly" describes a similar experience, decades before LLMs. It's amazingly easy to feel like someone is understanding you when they are just pattern matching on a common shared experience.

This seems so likely that I have a hard time understanding why some people think it's impossible.

I know; if I hadn't experienced it myself, I would have a hard time believing it. Even if we grant that it's "merely" as you say, that it can interpret my loose and often false or misleading rhetoric to identify my question, trawl through thousands of lines of code to determine the core problem, select some very specific and relevant internet page, and then translate its solution fresh into my own problem domain; that in my mind at least "an expression of intelligence", even if it is not truly intelligent or understanding.

FWIW this is my own mental gymnastics around the Chinese Room; at some point the question "where is the understanding?" is moot, because if the Chinese Room reliably delivers context-specific correct results in unique contexts, then what more do we require of intelligence? I have to admit that sometimes it does better than I can do, and I've been called intelligent by intelligent people my whole life.

(BTW Killing Me Softly wasn't written/sung/anything by Carly Simon, but composed by Charles Fox with lyrics by Norman Gimbel, in collaboration with Lori Lieberman [after she was inspired by a Don McLean performance in late 1971].)

Thanks for that final BTW which sent me on a side quest to listen to a half dozen different versions and gave me a completely different outlook on what that song was about and what it was saying. Made it even more poignant.

> if the Chinese Room reliably delivers context-specific correct results in unique contexts, then what more do we require of intelligence?

First, I'm not sure I grant the premise (esp. "reliably"), but even so the answer seems glaringly obvious: we require intelligence to extend the range of what is known so that it can be looked up / delivered by systems like libraries, search engines, and LLMs. The conflation of knowledge indexing and knowledge creation (and, more specifically, that some people don't seem to be able to distinguish them) may be the core of the issue.

> BTW Killing Me Softly wasn't written/sung/anything by Carly Simon

Yeah, I probably hallucinate that. :P

I agree that there is some 'higher' intelligence that is about knowledge creation; say, mathematical inventions like RSA, or new philosophy, or coming up with interesting mathematical problems. But I only asked what is 'required' of intelligence, and we should admit that the vast majority of people, even among those who have above-average intelligence, do not create new knowledge of the grand sort. Most people acquire knowledge in education and then apply that knowledge in their vocation, and their improvements are optimizations or remixings that we already see AI capable of making.

Most people don't create knowledge "of the grand sort" but that's moving the goal posts; they do create knowledge "of the mundane sort" constantly. LLMs with a suitable harness can simulate this (poorly) by parroting what humans do when they are flustered (mumbling to themselves things like "What was I going to the store for? I made a list. Where did I put the list? Ah, here it is. Now where did I leave my keys?") and it even sort of appears to work, at least in highly structured environments.

But it only appears to work as long as things stay within the expected statistical distribution. If I'm driving my usual route to work, waiting to turn left, and the three cars in front of me each start their turn, glow pink, and turn into circus animals with their occupants perched precariously on top I'm going to have a very different reaction than any self driving car or LLM.

I've been in situations like this, where the cars in front of me turned into circus animals, and let me tell you, I didn't react intelligently.

Reply on news.ycombinator.com