I think this is the biggest problem. I don't need a lot of random data in my database. I need a lot of specific scenarios set up. And a way to get those scenarios back after I test something.
I've definitely been in a lot of situations where test data is a problem. A particularly egregious one that comes to mind is the poor developer that had to develop the fraud functionality. Marking an account as fraud nuked it in the back end. Lots of angry testers/developers when their favorite test account got marked as fraud.