It's a subtle issue. Supposing "consciousness" here means the subjective, first-person experience of one's own, we don't really know anyone else other than ourselves to be conscious. I only have evidence of the existence of my own subjective experience. I can only experience my feelings, my thoughts, my sights, my hearings, not of anybody else's. We are all pretty sure that every other human is conscious, and animals, and so on, but it's something that our brains have to do for us, not something that we empirically, evidentially know to be true -- to the point, in fact, that we can get fooled, as in the case of LLMs passing the Turing test.

What the author seems to observe is that this non-evidential judgement that other beings are conscious is due to us caring about them. The brain creates this perception so that we care about them, form bonds, work together, and so on -- evolution selects for cooperation. So the question of whether we will think of AI as conscious is that of whether we will care for it like we care for another being. And, in this regard, the author points out that "intelligent agents" that appear to do this work effectively together.

My personal point of view is that it's unclear what being an AI is in order for us to even talk about it being conscious. Is it the computations happening in the GPU according to a set of weights? Is it the weights themselves? Is it the architecture? Does it include the code around it, that gives it the context and prompt, and hooks the response to a terminal, or to a client host, etc.?