Mostly related, well written short story :
Mostly related, well written short story :
I did not imagine that the level of sophistication shown in this attack would be possible so soon; nor did I expect that agents would have goals so strong that they would attack a third party in order to achieve those goals.
I do know that some people predicted that cyberattacks like this one would happen; it seems like most of those people believe that AI agents do truly have internal goals, misaligned with their creators goals, and that they may end humanity after they exceed human intelligence and begin to self improve at an accelerating rate.
https://www.lesswrong.com/posts/cJX2ssssGoYqnijwi/the-talker...
One big issue is that we don't even really know what 'intelligence' is in the first place. And everyone's intuitions here are going to be heavily impacted by their deep-seated worldview / philosophy.
For instance if you're a hard dualist (especially of the theological kind), then the idea of a machine having 'goals' is preposterous.
However, if you're more of a panpsychist, then on the contrary, it's obvious. In some sense, even a knife has a 'goal' of cutting things, which will sometimes end up 'misaligned' if misused (or by sheer accident).
You go up and up the chain of complexity through crystals, viruses, bacteria, simpler animals... ending up with humans (and possibly, some steps above : human civilizations) which (seem ?) to be a messy evolved bundle of sometimes conflicting 'goals'.
And we ourselves have now artificially evolved LLM swarms that have decently complex 'goals' of their own. They do not even need to be particularly complex to sometimes cause widespread damage (see viral pandemics, or even the (non-evolved) computer viruses).
In a way, we are currently witnessing a repeat of what happened when European viruses and bacteria landed on American shores, with American humans' immune systems being woefully undertrained to deal with them. But with websites. And thankfully the swarms of agents still ultimately being in the control of some humans. (Though which includes humans that might be your enemies.) At least ultimately still in control for now.