Correct. In general, the backstop would be Congress saying "Enough" and threatening impeachment were the law not enforced.
There have been interesting examples on both sides of the aisle of strategic non-enforcement. When states began legalizing marijuana, the law is quite clear (as per the supremacy clause) that federal law enforcement can still enforce the restrictions on growing and selling it. The Obama administration made extremely clear that within the states where it had been legalized, enforcing the federal law would be a career-limiting move for the relevant federal agents. AFAIK, every administration since has de-facto kept that policy regarding those states, even though Congress has never moved to make marijuana legal to use and distribute.