The problem is pro-punishment politicians. Singling out any particular office/position misses the point.
Even when judges practice discretion, pro-punishment legislators step in and introduce mandatory minimums.
And even when prosecutors practice discretion, pro-punishment judges will step in and force them to prosecute cases that the prosecutors don't want to prosecute [1].
[1] See Suffolk County for a recent example