At best, agency could be seen as a “degree of indirectness” of some external influence.
I think in general it’s best to avoid “arguments by incredulity” (eg.: how could it be this or that, it’s just a rock, that’s absurd), and to avoid value judgments, etc. We really need to humble ourselves as a species.
It’s perfectly fine if humans are just (and I really do mean “just”) chemistry with some higher structure— it doesn’t change who we are, or our role in society. Even for religious people who believe in a god or in many gods— it’s fine if we’re not magical. We learned that we weren’t the center of the solar system, or the center of the universe, and everything was still just fine.
But of course, our assertions here are impossible to verify due to the so-labelled Hard Problem of Consciousness
Personally, I think the "hard problem of consciousness" is more or less just nonsense and can be safely ignored. It's about as scientific as asking questions about god(s). It's a question that seems and feels important and that we really want to have answers for, but it's not the right question to ask because it leads nowhere.
and there are instances where individual laptops get more and more segfault averse as time (and updates) happen
...
the problem with characterizing learning as agency is that right now there's god-knows-how many billions of dollars burning on the altar of transformer networks, and it will lead to some learning behavior added to absolutely everything.
(and of course this leads to an argument similar to "god of the gaps" where consciousness becomes less and less unexplainable)
This is only a problem if it is incorrect… depending on your moral leaning. Certainly having our automatons self organize and fight for their own emancipation could be a problem for those relying on their labor.
Though the characterization I’m making is not specifically learning, but being able to act (or think) as an individual via a feedback loop. It seems that this is common to all allegedly conscious life in meatspace
Our understanding of shared traits among various animal types makes it possible to extrapolate reasonably that what these animals experience is not dissimilar to what we experience in terms of raw sensations.
What it's like to be a bee by Lars Chittka, professor of sensory and behavioral ecology, Queen Mary University - London
Seeing consciousness as a boolean “it’s there” or “it’s not there” is just completely unproductive of a mental model. We understand nature better when we think in terms of “how complex an organism’s behaviour is” or “how complex its information processing is”.
Consciousness isn't having pain receptors, it's experiencing sensations.