The first problem is the people who don't know how to drive. I cannot count the number of times I have seen people stop in the middle of a roundabout to let cars in, or stop at the entrance and stare in confusion when there is absolutely nothing coming.
The second problems is inherent in the design. A line of traffic that is going straight can, effectively, prevent any cross traffic from entering the roundabout, due to the small radius of most US traffic circles. This is a trade-off, of course, and is still safer than traffic lights or a 4-way stop, and usually more efficient.