To me, that's not the main point. The main problem with a universal backdoor "so that the good guys could get in" is that the good guys with all their good intentions have no business pawing through your private stuff. It's not about having a good reason, or having a mandate. It's about your rights to not be subjected to warrantless searches, which is exactly what a golden backdoor would enable.
In other words, even if some genius devised a backdoor that was perfectly secure (allowed access only to those who were explicitly granted that access), had it verified by an international group of geniuses 10x smarter than she is to be sure it's actually secure, and granted access to only the most trustworthy organizations, which would of course act in the most transparent way possible with a clear cut mandate and individual accountability, it's still a violation of your privacy, a violation of your rights, and in the US a violation of the constitution.