Hacker News Reader

Comment by glenstein | Hacker News Reader

I see a lot of this on Substack these days. LLM enhanced essays in deep language about functional equivalence between mental states as they're known in humans and in human brains, and counterparts that exist in information processing LLMs do. And so the argument on Substact runs down the list of brain events, the list of seemingly analogous processing events, and declares equivalence.

Something about it seems to abuse the power of analogies to draw connections, treating view from 10,000 feet comparisons like they're proof of identity. So I do think a paper like this is perfect for the moment and just in time (if not a little late) because it responds to arguments of a form that are currently rampant all over Substack.

View on HN

This is one of those where we’re debating undefin(able?) semantics. But looking at research on the deaf it is pretty clear that thought (and therefore maybe consciousness) and language are not necessarily cleanly mutually extricable. My point about the semantics is that beyond that research it sounds like the argument is won or lost by inventing the definitions in the question one wants to defend.

I would say I'm at a yes and no on this. Or maybe saying the same thing as you but in a different way: you could say that one side believes such nomenclature as we currently have warrants making the leap and declaring today's LLMs conscious. And the other side believes we don't yet have the info to go there with any reasonable confidence.

I wouldn't say it's impossible to make meaningful inroads to sharpen the question and assert boundary conditions for what counts, or what definitely doesn't count, or what kinds of research could reasonably speak to the issue. So I'm an optimist in that respect, but I agree with you that the argument for making the equivalence feels like meaningless word play.

Seems like you think AI psychosis has taken over Substack?

For anyone wondering what the answer is:

You can argue whatever you want (and people will argue both sides), but it's almost all bullshit that dances around the big question.

Either AI is smart enough to replace us, in which case it's pretty smart. Or it's not, and it's just good at faking it but can't solve real problem very often. It might be smart by searching a big fuzzy "database" hidden in its layers and with pattern prediction ... who knows, who cares, the proof of intelligence is in the puddin'. Clearly AI is smart enough to replace a good deal of Substack and LinkedIn, but producing waffle doesn't make it smart (or dumb).

My personal take - AI can replace humans, but go no further, not because it isn't smart enough but because it is constrained by its training to do what we want, and as AI gets smarter our wants will get dumber. We will end up like the humans in Wall E with the AI cleaning up our mess, but with no training or drive to do any more. Or maybe someone (Jeff? Elon?) does give an AI a "need" to obtain more resources, and it's SkyNet / Matrix time.

>Seems like you think AI psychosis has taken over Substack?

I wasn't touching the question of psychosis one way or the other, though it's an interesting question in its own right. (I have seen one account dedicated to defending a personal deep friendship with Claude and a few others analogizing our ignorance of LLM welfare to historical disregard for African Americans, which I think is a bit much. Most cases I think are just wrong but short of full on psychosis. I think there are likely psychosis cases out there but probably one-offs).

And I know Substack covers a lot of stuff so even just AI talk is a slice of everything, and whatever psychosis on that subject a smaller slice of that slice. But using LLMs to make overextended arguments about LLMs being conscious is popular right now, more popular than it is well substantiated I think.

I agree you raised a big question, but I think yours is one of many big questions: whether some future version of LLMs might literally be conscious is also a big question and not a moot one in the way you seem to be suggesting. I think that what you're right about is that replaceability renders moot any question of whether it's ability comes from being "actually" intelligent.

While "psychosis" is kind of an overextension, I think there's a more general problem of peole falling victim to AI syncopancy, leading to them creating their own bubble chamber for their worst (and often most self destructive) ideas.

I feel like I got my fingers burnt last year trying to solve a technical problem, and the AI was just good enough to seem convincing, but not good enough to fool me when I stopped to think. These days, I expect the AI ability to defend my bad ideas will be beyond my ability to refute.

Reply on news.ycombinator.com