For LLMs in particular, it's notable that they both 1) originate from a lineage that attempted to model biological brains mathematically and 2) are the only other known entities in the universe able to reproduce many types of human intellectual labor. That is surprising and striking. Does that mean that they definitively share other traits with humans like consciousness? No, but it does make them one of the most interesting candidates, particularly when we have a very limited understanding of how humans themselves are conscious.
At the same time I can say with 100% certainty that LLMs (or any current "AI") is in any way, shape, or form, conscious. Not even close. A decade or two ago it was the Internet that was a possible emerging consciousness.
The onus to prove LLMs are conscious is on you making the claim. And if you say we don’t even know what consciousness is, how can you then attribute to an entity?
Definitionally and philosophically, humans are conscious (Descartes et al.) so need no proof themselves
And it's unclear what your actual argument about the nature of consciousness is, aside from what seems like a bald assertion that humans are the only conscious beings.
The reality is we have no way to measure consciousness or how it emerges, or how to tell if other humans actually experience it the same way we do, so confident statements about consciousness being unable to emerge from simple weighted functions is baseless projection, wishing ourselves more special than an emergent property of the bag of weighted functions in our heads.
Do we? We only know that at most one thinking person believes they have it, and everyone else claims they have it.
Now what that means for other people, animals, AIs, aliens, nation states or electrons is a matter for dispute. And what ontological conclusions you do or don't draw for it are matters of belief based on what philosophical arguments sound convincing to you.
But believing you have experiences is a very weird thing to say. I'd think Wittgenstein would have something to say about such a hinge proposition. You can't really doubt it without invoking radical skepticism that undermines your ability to know anything.
Describing the human brain in terms of how LLMs work is an interesting rhetorical tactic.
You don't know how the human brain works, nobody really does, but you have chosen an analogy that supports your position. Which says more about you than anything about whether LLMs should have rights.
We are fundamentally just a -lot- of those cells trained to do a lot more things with a lot more sensors, which is sobering. We might actually not be as special as we always believed ourselves to be.
I think it is more comforting to believe our sense of self built on a big bag of those cells is something that cannot evolve on human made synthetic substrate from simple statistical functions and weights stacked very densely.
But we simply lack the evidence to confidently say so one way or the other.