* There's no such thing as "the bad guys". Everybody is a potential "bad guy" depending on who the authorities are. J Edgar Hoover investigated Dr Martin Luther King Jr as a subversive and used the FBI to try to pressure him to suicide himself. Nixon is on tape ordering J Edgar to investigate his political opponents. Perhaps the US political system fails and whatever government replaces it finds a trove of NSA data and uses it to find+convict political dissenters. Just because the 1st Amendment exists and protects open political discussion now doesn't meant it will always exist in its current form.
* Hackers/crackers/phishers and organized crime. The NSA data troves _must_ be a juicy target. If any malicious intruder gets access to the data (I know, highly unlikely), what could they use it for? Surveillance, tracking, blackmail, extortion, political affiliations, personal beliefs, etc. What if they don't target you, but rather political leaders in the US. That would make the holders of that information _way_ more powerful than any political campaign donor.
* Have a US security clearance? You are subject to very high standards of conduct. Anything that could impair your judgement or lead to possible blackmail of you or your family is potentially grounds for taking away your security clearance (which likely means you can no longer do your job). Alcohol addiction, gambling addiction, sexual relationships, history of crime, immoral behaviors, etc. Gen David Patreus (Director of the CIA, IIRC) was in an extra-marital affair and tried to his this fact from "the company" and from politicians. In this scenario, he is a "bad guy".
* You are assuming the NSA only uses records for official purposes. We have already heard that some NSA employees have been reprimanded for snooping on their spouses and neighbors using work tools.
* It's not as if the NSA has a perfect record. Snowden wasn't even close to being the first whistleblower and it looks like there may be another post-Snowden disclosure. The NSA doesn't have control of its own people (or contractors) so I assume it doesn't have perfect security procedures either. That means it is open to threats against its data and procedures from both inside and out.
* The NSA is suspected to have tipped off the DEA/DHS and FBI for cases that don't involve terrorism or national security. NSA techniques are suspected to have been adopted by much of DHS. This means the threshold to be considered a "bad guy" is now a lot lower than the NSA used to be tasked with watching. Apply the slippery slope argument. What if the NSA quietly helps out with civil cases (such as MegaUpload) and not just criminal? What if it goes even further?
* The NSA isn't the only organization trying to gain access to sensitive internet communications. If anyone else finds out how to take advantage of some of the same tricks the NSA uses, they could potentially have access to the same data and communications. Think nation-states, organized crime, disorganized criminals, even marketing/tracking companies with questionable ethics.
* If SSL / TLS is no longer beyond cracking in near-real-time, MITM is now possible. This could set back peoples' faith in the security underpinnings of the web even more than it has been eroded in recent years. Even worse if people don't find out about it.