it doesn't matter? if they're actually intelligent or conscious what we are doing is essentially slavery. it matters an enormous deal ethically and/or morally.
if aliens landed on earth and seemed a lot like us, we would have to grapple with the possibility that they might be conscious and deserving of the same moral consideration we give each other and many other animals.
but when meteors land, we don't have that concern
now are llms more like aliens or rocks?
The distinction you are collapsing here is really a crucial one. My thermostat's goal-oriented behavior does not make me a slaveowner unless rocks are conscious, to riff on your comment below. Our lack of consensus on the definition of these concepts is no reason to conflate them.
The distinction is crucial because while these machines clearly do exhibit intelligence under certain definitions, there is no evidence of consciousness, and we have very little reason to give them the benefit of the doubt, unlike biologically related beings, a great many of whom, human or otherwise, are presently in something like slavery