Not at all.
Suppose the worst possible person is in the prosecutor's office. They get to decide who to prosecute. But they can't just file false charges against someone and throw away the key, they have to prove it in court with evidence and convince a jury of the defendant's peers. Moreover, we could prohibit prosecutorial misconduct and set up an independent agency for investigating and punishing it. And we could require vague laws to be construed in the defendant's favor and strike down excessively broad ones as unconstitutional.
It's possible for these checks and balances to fail -- there is some evidence that they have -- but in principle we could have stricter ones that work better. And if they work as intended, it doesn't matter how bad the prosecutor is, they don't have the power to cause harm, and they themselves are punished if they try to exceed the limits of their power.
But if they just do their job, i.e. investigate crimes and prosecute the offenders, they do have the power to do that.
And so it is with any other office in the government.