We give names to all kinds of outcomes of our brains competing circuits. But our brains competing circuits have evolved to solve a fundamentally different set of problems than an LLM was designed for: the problems of human survival.
> A blind person has no notion of colour yet we don’t claim they are not sentient or generally intelligent.
Axiomatic anthropocentrism is warranted when comparing humans and AI.
Even if every known form of human sensory input, from language to vision, sound, pheromones, pain, etc were digitally encoded and fed into its own large <signal> model and they were all connected and attached to a physical form like C3PO, the resulting artificial being - even if it were marvelously intelligent - should still not be used to justify the diminishment of anyone's humanity.
If that sounds like a moral argument, that's because it is. Any materialist understands that we biological life forms are ultimately just glorified chemical information systems resisting in vain against entropy's information destroying effects. But in this context, that's sort of trite and beside the point.
What matters is what principles guide what we do with the technology.