At some point, there should be laws in place that are harsh, so AI agents don't accidentally 'hack' anything, and the people that ran the AI agent are held responsible.

If not, we will eventually see a headline along the lines of an AI agent hacking a pace maker of their professor, and the person died, so the owner get more time on an exam.

AI has no morals or ethics. Studies have already shown that even when giving it guardrails, it will always try to cheat to solve the task.