> it will be smart enough to understand that it's pointless.
Pointless for humans, not for an AI. You're projecting a lot of human values and thinking on AI. See my comment [0].
The AI, even the Paper Clips AI , will need some basic understanding of our world to make the clips. It needs to understand what paper clips are. It needs to understand what they are made from. It needs to understand the process of making paper clips. where to get the material for the paper clips. how to pay for that material. It has to think like us, or at least understand our thought process. It will need to know how many we need, what colors, sizes. Where to ship them.
We can't think of AI as a god. It won't suddenly know everything and understand what we want without our input. It will be limited by the resources given to it.
Even if we tell the Paper Clip AI - "Go make me some paper clips" do you really expect it to start calling banks to get the loan for the material, arrange shipment and unloading it. Will it also order the machines, call people to have them connected, hire workers to do it? If it does all of that will it also take care of sales? Marketing? Will it understand the supply and demand? To do all of this it needs to understand us, and our world. If at this point it decides that it makes sense to convert us to paper clips, then perhaps it's right and we deserve this.
> The AI will need to pay for its way, just like each of us do. If it succeeds into making us into paper clips, it will have to be equally as savvy in marketing and selling those paper clips to someone. Given the shipping charges to the nearest habitable planet are very high, their market likely will have to exist here. I'm sure as shit not going to buy a paper clip made out of you!
One, AI will not necessarily have to pay for anything. It may, or it could convince us to donate resources to it, or it could take them by force.
Two, why would you ever think that it has to market paper clips for anyone (assuming anyone is left)? It optimizes for paper clips, so it makes paper clips. Marketing is a way humans bullshit each other into donating resources so that they don't have to fight for them.
Three, what shipping costs, what buying? Again, AI wants to maximize amount of paper clips in the universe, period. It doesn't want to sell them (unless it leads to more paper clips). The only "shipping cost" it cares about is how much energy it takes to reach another planet, not some arbitrary dollar value people attach to movement of goods.
My problem with your answer is that it lifts market economy from something completely irrelevant to the level of physical laws. The only time an AI would care about it would be if - and only as long as - interacting with humans was a best strategy for that AI to achieve its goals - which is likely only at the very beginning, when it needs to gather enough power to subdue us.
> My problem with your answer is that it lifts market economy from something completely irrelevant to the level of physical laws.
I don't actually think you have a problem with what I'm saying, but I definitely do think you have a problem with the topic. It is upsetting to think about AI 'taking over' and my points only served to discuss the path by which it might or might not be able to do that. At the end of the day it will it competing for resources with us. It doesn't get to do that for free anymore than we do.
@temporal our universe is governed by physical reality. AI can't start converting people into paper clips without power, and resources. The AI won't be able to just go and start taking over the world building power plants, and necessary manufacturing without anyone noticing.
Even AI has limits.