Given it had write access to its implementation, "unexpected" here reads more like "inevitable" than the witchcraft the article wishes to imply.
When people propose regulation for AI systems out of concern for unexpected self-improvement, this is the sort of scenario they are worried about.
Security in general is hard to get right, but the basics are not difficult if you try, and most companies fail to even implement the basics.
For AI, sandboxing will get you quite far, but how do you guarantee it’s used as the risk of accidentally creating a self-replicating WebGPU virus increases?
Methinks that sand boxing is more like cat boxing, vs say an carrier.