If you have access to a web browser, you can make arbitrary network requests.
In the HuggingFace incident the agents found very clever ways to do this, like they found a website that let you make POST requests and returned a screenshot of the webpage.
>allow agents to only do things ordinary and average human endusers can do.
This doesn't work. Ordinary and average human endusers break security all the time.
I can do all sorts of terrible things with ordinary human-level access. I can install malware. I can wire all my money to Nigeria. I can send a threatening email to the president. I can send trade secrets to competitors. etc.