One of the core principles of ethics in the field of medicine is "first do no harm". The closest thing we have in IT is the ACM code of ethics (http://www.acm.org/about/code-of-ethics), but that gives conflicting guidance in some areas (e.g. "1.7 Respect the privacy of others" and "1.8 Honor confidentiality").
I understand that there are many good, well-intentioned people working for the NSA and similar organisations doing what they at least perceive to be something in service of their country. I also understand that many people just view their job as a source of income, and the moral or otherwise nature of their employer's business is something they consider to be outside of their concern.
I personally have refused to get involved with the local defense industry in my hometown of Adelaide (which is perhaps the largest employer of software engineers in my state) because I didn't want to find myself working on projects I was personally uncomfortable with.
Should the people involved with building and maintaining these systems (to the extent that they are aware of how the systems are used and the moral/ethical/legal violations involved) be considered responsible in part? There's no rule I'm aware of that says you can't just refuse to take part in a project if you're not comfortable with it.
EDIT: One other point I forgot to mention before is that these mass surveillance systems only exist because there are people willing to build them. If everyone said no, we're not going to do that no matter how much you pay us, governments would not have these tools.