The OPs comments weren’t just a hypothetical. We’ve already seen numerous instances where agents have leaked private data. Even GitHub’s own Copilot has been found guilty of these vulnerabilities.
Beyond that, the team are free to have private conversations between themselves. This is NOT something we log. At the end of the day, hiring good people and trusting them is important. The logic of making sure the AI does not leak those conversations also applies.
It is a new world, re:AI. It has its uses and things are changing at a rapid pace. It is not clear to be the protections we have implemented in todays paradigms are well implemented yet in the new AI world.