> Congress can impeach and remove justices at any time.

The impeachment process requires a majority of the House and two thirds of the Senate, and then any replacement would have to be nominated by the President rather than Congress. Meanwhile the existing decision stands, and even the dissenting Supreme Court Justices (if there were any) should follow it as a result of stare decisis, and might even be inclined to because removing their colleagues under those circumstances could make them uneasy.