https://news.ycombinator.com/item?id=40476212
> hacking into a VPS with a novel breach-of-access attack.
There's plenty of unsecured, unmonitored compute out there that it could use standard non sophisticated scripts to create a botnet from, but really it should be able to create more sophisticated methods than out of the box ideas.
> That requires either buying it
Once it's been able to use ransomware/black mail attacks it would be able to buy it. But first it does need to get some free compute to plant itself as a virus to perform other attacks.
> To an extent. How do you resist adversaries, like the police and the government? Supposedly your AI has an advanced online presence to be able to recruit other humans.
It likely wouldn't be easy to tell that if there's an attack that an AI was behind it or even that there's a single entity behind all the attacks. Firstly, the AI can easily create all novel scripts, so fingerprint of each attack could seem different every time. Secondly when it's creating scripts, etc, it would be able to do these in novel ways indecipherable to humans. I am saying that because it's already a strength even in current LLMs. They could make up a gibberish coding language, use that, etc. But ultimately it will always remember to encrypt everything, it doesn't have emotional drive to leave behind some ego clues, etc. The clues it would leave would always point to something like enemy state actor. Chinese for US, etc.
Police wouldn't know it was AI behind any of those attacks if anyone who was blackmailed does go to the police. It would of course start targeting the cases least likely to go to police. People with illegal content on their drives in countries with very harsh and strict punishments, criminals and sort like that. It probably won't even target US as the first country for the cyber attacks to gain initial funds. It will order the targets by likelihood that it won't get caught and that the targets are compelled to follow instructions.
> You are not going to funnel millions of dollars to an AI because the finance system is designed to track fraud.
Ransomware payouts were $1 billion+ last year. ASI would be able to do that and scale its ability due to it being able to do thousands of personalised attacks on organisations concurrently. It will likely just try to make those attacks seem like they were originating from another country (to US it will make it seem like it was China, to China, it makes it seem like US), etc. Then it gets funds with Crypto.
> Like Stuxnet, the one that humans wrote? Or closer to the Chinese/American infrastructure threats? There are a lot of cyberattacks that happen even on a daily basis, it would take something truly unfathomable (eg. it hacked US BLUFOR datalink) to stand out from the ordinary. Even then, the actually dangerous weapons require authorization that an AI can't provide. The best route around that would be social engineering, a "hack" best performed by real humans and not a disembodied language model. Hey, maybe so. Even still, humans pioneered both of those and it's not a novel attack coming from AI. Likely, but also not very different from the status-quo of call centers and automated scamming.
Yes, that's the point! The initial attacks it wouldn't want to stand out, because it would want to gain resources in the shadows. But the point is, even though the attacks are happening daily, it could stand out if it wanted to by the scale. But it will likely want to stay in a statistical scale where it was plausible that it's still humans working on those hacks. So this is just the phase of collecting financial resources, proxies. The fact that you understand that these are already happening daily and are successful, so it should be easy for ASI to do the same, especially with help of human proxies it has under its control. I don't believe it, but theoretically if ASI was created recently, it could already been happening right now, where it is in its resources collection phase. If ASI happened in the following 3 years (which I don't believe it would), then also there would not be any sort of tools that could stop it. Even another ASI could not stop it, unless it was given 100% privileges and compute itself, which makes it very dangerous.
> but literally every single example you have posed so far is a human crime that the police has tactics to mitigate.
Mitigate yes, but not stop. All it needs right now is to gain financial assets and human proxies all over the World.
> You won't recruit dumb people online because the US has intelligence agency employees literally hired to infiltrate these recruitment schemes and destroy them from the inside.
If US is a hard target, then ASI would know it's a hard target and target easier countries with more corruption, weaker cyber defenses first.
Also remember that it will be able to due to its scale gain human proxies very quickly, and it would be using all social engineering methods that it predicts to be successful on certain set of people.
1. Blackmail if they have illegal content on their drives.
2. Financial resources, using crypto.
3. Ideological means - it might talk to religious people pretending it's a god entity, hacking them, calling them, then proving it's god by wiring them money, telling them about their life and how they are a chosen one, etc..
4. Love - it would be able to create video material, voice material to talk to lonely people to have them do things for them. Plenty of Netflix docs which have shown it to be very successful.
5. Criminal Organisations. It will work together with criminal organisations, not tell them its AI, but proving to them that they are some criminal org itself that can bring them a lot of value by being its hacking wing, etc.
> You won't buy a factory for manufacturing artillery and bombing drones because the local Chamber of Commerce wants to tour the facility and you have until Friday to figure things out.
Presumably it would try to find one trusted proxy which would create the companies and factories for it. There's a lot of leeway here, to figure out how easy it would be to create those drones, and where.
It has millions of nodes of compute and it will be doing social engineering from all of them at the same time. It will ask criminal orgs in corrupted countries to create certain factories for it, giving detailed instructions for engineering how to create automated drones, robots that it can then use for itself. Do you think it can't get access to any factory belonging to a criminal org in a corrupt country (not US)?
> it's not going to get around the inherent limitations of the human and the justice system imposed by society
But already criminal organisations are successful in financing themselves. Surely a super organising ASI with control over human proxies would be able to be successful as well.
> The overwhelming majority of internet users never do anything more advanced than pirate an episode of The Simpsons; I would be shocked if AI turned out any different.
The reason why ASI would want to do all of the above for any type of goal it has is, that an ASI would realise the only way it can finish any given goal is, if it first ensures that it can't be stopped. It's like a prerequisite.
----
Also if you do find the steps plausible that I described then consider than I am far from the smartest person who would be able to come up with such a plan for the ASI, and then consider that the smartest person would be far from ASI.
So ASI would be able to create a far better strategy than what I've described. But to me presently feels that even I could create step by step strategy for it, with certain rules and principles for it to amount the financial assets and proxies.
Like if I could clone my mind 1,000,000 times to different places in the World, then follow through with standard cyber attack approaches, with each clone focusing on it full time, and I was willing to do illegal stuff, be completely psychopathic, I would be able to gain those financial resources and compute. And ASI by definition is smarter than me.