I haven't tried what you guys are talking about myself, but I think when agents begin talking to other agents, the chances for unpredictable goal mixing and confusion resulting in really unexpected and bad behavior is a lot higher.

That being said, the danger is measured in computer damage, which can be a lot personally and to a company, but less existential, so your mileage may vary as to how "scary" it is.