I'm no expert on spycraft or anything like that, but this sounds overly reductionist. If they can threaten you to lose your career, then maybe doing one tiny little other thing that won't matter seems like it might be worth it. Now they can blackmail you further.
> they can just find an ex lover and blackmail that person to say you raped them or whatever
How do they find that person? How do they blackmail them? It becomes exponentially more complicated than just blackmailing one person. Additionally, someone finding themselves in the situation that they are being actively targeted with blatant lies is much more likely put up an indignant and active defense vs somebody who actually has a guilty conscience.