Frank Herbert wrote a couple of novels about the Bureau of Sabotage.
The setting had a government so powerful that it could cause horrible problems with offhand decisions, and so large that it couldn't be reasoned with or set unified policy. (It rather reminds me computerized stock trading today, with problems like the "flash crash" where decisions outrun the actual intent of any human involved.) It couldn't shrink or slow down and still handle genuinely pressing decisions, so the solution was an agency meant to delay and interrupt whatever actions it thought would cause problems.
I certainly don't think modern governments are so efficient they need to be intentionally delayed, but the nature of the department is interesting: powerful and almost unrestricted in oversight, but strictly reactive and unable to make policy. It's not a perfect fix, but regulatory capture and incompetence are less worrying when the people deciding on government procedures aren't the same ones enforcing them.
In the real world, that might look like a Department of Exceptions. It would be unable to make or even change policy, but would have broad power to reverse specific government decisions. The point isn't to fix bad governance, that's too hard a problem, but to fix issues like this where the edge cases of a rule produce absurd results. For safety from overreach, it would probably need to be one-sided; able to undo government actions on private entities, but not apply new ones or step into private disputes. (I.e. it can grant you a car registration without the normal papers, but it can't peanlize a non-crime or reverse a civil lawsuit.) And for safety from capture, it can't be unaccountable, but holding it to strict policies would be self-defeating - perhaps the best fix would simply be to record common-sense rationales for each decision, and set a low bar to fire or vote out employees for bad decisions.
It's a wild idea, and I can see a host of possible issues. But your point about courts is interesting: we already have a system which can't make law, but has enormous power to review government decisions. This would be much the same idea, aimed at irrational actions instead of illegal ones.