That's why there are typically multiple secret services (with different missions but similar skillsets). They are encouraged to spy on each other. None is allowed to dominate everyone else.

Remember the movie Valkyrie? It provides another vivid illustration of what can be done to reign in a secret police once the army rolls in. The very same thing happened in the USSR during the power struggles after Stalin died.