But they were themselves minorities: wealthy property-owners. The elite minority. And I think history shows that they were themselves the minorities they wanted to protect. With every incentive in the world to do so. The "popular passions" were no doubt a frightening thing to them.
I think this is fairly apparent even in documents for public consumption, like The Federalist No. 10. (Of course, for more serious analysis, we'd want to turn away from the Federalist Papers and look instead to the records of actual decisionmaking.)
Like most nations, the US has a founding myth. Since it's so recent and there were witnesses, the Founding Fathers can't exactly have overt superpowers like levitation or divine birth, but they are still portrayed in a mythologized way. So when looking at the founding of any nation, a little extra care is needed not to be led astray.