I assume he is referring to malicious bugs and simple/sloppy backdoors? For example, if I append my public ssh key to someones .ssh/authorized_keys, it's not a backdoor anyone else could use.
Even if you're sure you can do all that, do you think a government employee can? wants to? Is allowed to keep such things secret from colleagues? Will keep such secrets? Won't ever be hacked by a foreign power? It won't spill to his colleague with the gambling problem being compromised by <insert bogeyman here>? It won't be used in the future in ways that would still be considered non-kosher by the current NSA?
The NSA can't keep their stuff safe from contractors who happen to be good guys with a conscience let alone bad guys being blackmailed.
But I think he's talking more about appending a public ssh key to a something like router firmware authorized_keys file to be distributed to many devices.
No, he's referring to corporate or governmental backdoors, the wilful introduction of a way to bypass security for e.g. legal purposes. Once such a backdoor has been introduced, it's completely out of any control and while the entity for whom it was built will have access to it, third parties may (and most likely will if it's valuable enough) also gain access to it.