Trust is earned. I've been running these for more than six months now, and the agents started out with very few privileges. For each task, it showed me what it was going to do, I checked things carefully a few times. Once it was clear it wasn't making mistakes, I let it off the leash a little bit more.
Do they sometimes make mistakes? Yeah, and I still check their work. But I've also employed humans, and they make mistakes too. The AI is not worse.