I'm sure plenty of people who voted Hitler thought they were in the right side of history until they suddenly weren't. That's the unfortunate part of politics, you have to build a system around not trusting people.
It does have to be justified based on existing law and this is important. But often, existing law only talks about broad principles or is contradictory, and judges are expected to fill in the details. In the case of the U.S. Constitution, these "details" can be really huge gaps.
This is what an appeals court justice does and their check and balance role on the other two branches of govt.
It's strange that the idea that judges shouldn't decide by (future) popularity gets downvoted. At times I don't understand this community at all. Is that a "I believe that my opinion will rule supreme in 20 years, so judges should submit to it now and just ignore the constitution"-thing?