Why does it matter if AI is sentient or not?

> Why would an intelligence machine need to behave like we are?

Because we need to communicate with it. That's the difference between machines that do physical work and machines that do cognitive work.

A lot of this has to do with intelligence being misused for AI. "Artificial Intelligence" is not the same as intelligence, it's like a new term in and of itself that was used metaphorically and people have dropped the A and think that machines are actually intelligent.

Intelligence has to do with modeling the future in order to navigate the space of possible actions in order to adapt and survive. Think about any entity, if that entity finds itself in a situation where it needs to make a decision in order to survive, and they make the wrong decision leading to its death or extinction, that entity is not as intelligent as another entity that made the right decision. It's about adapting and predicting the future. AI models cannot predict the future and are only extrapolating from training data without any kind of simulation or modeling of their environment or even context.

In regards to sentience, it could be that sentience comes along with a high level of actual intelligence (not artificial). The more intelligent you are, the more sentient you are. I'm leaving out a lot of details but this is why I think LLMs are going to hit a wall and we're going into another AI winter (although they will make a huge impact and are already doing that).

In my mind, sentient is one element of the definition of intelligence. My understanding of how LLMs work means an LLM will never be capable of understanding its own output. It will never be able to ponder.

IMO the distinction is an important one. If a machine can be made sentient, that would be a milestone in understanding the creation of life and understanding humanity itself. As well as raising ethical issues concerning any distinction[s] between sentient machines and animals.

The distinction is also important for the future of LLMs. If a machine can never become sentient, then that places likely limits on the power of the nature of its "intelligence". As I stated in my opening, my own definition of intelligence requires sentience at its core. Anything without sentience is nothing more than a sophisticated mathematical lookup formula piecing together strands of data in a logical manner. Incredibly powerful, but ultimately limited - in comparison to what a sentient machine could do.

If it is sentient, using it like a tool for workloads means we should be treating it as an equivalent being with rights. That means paying/compensating it. And no, you blasted bastards salivating to just get your hands on something you can copy/paste, turn on/off at will, torment, enslave, etc..., won't be tolerated.

Well "we dont get into the weeds" with microbes. And they gave us CRISPR, Penicillin, Photosynthesis etc. Sentience is not required.