To your second point, oversight is orthogonal to my original point: no matter where you land on the breadth of collections spectrum, more oversight is better. I agree with you there and in hindsight I could have done better to point that out in my original post.
Now the first point deserves a response. This demand for proof of effectiveness for various intelligence gathering techniques misses the point of my initial comment. The public doesn't know if cracking this program was a significant source of information. On the other hand, neither does Al Qaeda. Spreading fear, uncertainty, and doubt among your enemies is part of role of intelligence gathering, to the point where spreading the same among the population can be acceptable.
Consider the revelation that the NSA is gathering only a relatively small sample of phone records [1]. This information came out through a leak. Why didn't the NSA rebut allegations that they were sucking up every single conversation by revealing this information? It probably wouldn't have saved them in the public eye, but it would have at least dispelled the extremely damaging public perception of the NSA as an unstoppable Big Brother-style surveillance machine that's going to murder you for the contents of your emails, as opposed to an agency that, despite their twisting the letter of the law, is still forced to operate within legal boundaries. Why didn't this report come out to try to at least correct the public's paranoia?
The answer is because doing so would have given the US's enemies guidance on how to run their operations, which would allow them to more effectively evade detection. In this context, this sort of secrecy and fear appears to be a hateful but ultimately necessary part of intelligence: you can't assuage the public without assuaging your enemies.
At the same time, the public still have to be protected from a spy agency, and this is where mandatory oversight and regulation comes into play. While Congress has vowed to implement strengthened oversight over the NSA, the public isn't satisfied because this oversight is performed in secret. I'm sort of prepared to let myself be convinced that this oversight exists, but the truth is that whatever solution is implemented is bound to be unsatisfying for exactly the reason I point out above: hateful secrecy is necessary.
[1] http://www.washingtonpost.com/world/national-security/nsa-is...