If that's your argument, then surely the proper approach is to criticize the Court for those instances where their logic isn't consistent, instead of when the logic is consistent. For example, Thomas vigorously dissented in Gonzales v. Raich, where Scalia voted to uphold federal marijuana restrictions as a proper exercise of the Commerce power.
There is nothing wrong with judges having an "agenda" as long as that agenda rests on legal doctrine and philosophy, and not "outcomes."