I have lots of friends who work for the big 4 firms. I think the biggest worry is that most of the people doing the actual audit procedures are so inexperienced and have no long-term goal of staying with the firm. When you talk to them they talk like they're at grad school - "I'm just going to stay here for three years to get my CPA and experience then I am going to the private sector".
If that is your mentality, are you really going to try to dig up a bunch of issues that will cause stress for you?