It is relevant that someone writing about freedom conveniently did not counted whole groups as deserving that freedom in the first place.
It is much easier to be for free speech for example, when the group of people most likely to oppose you dont count.
Edit: by this logic, you cant critize Robespierre. Or Lenin/Stalin, because they were product of time and built an imperium anyway.
It is also relevant that nice sounding definition of freedom was used to defend the slavery or that the rational science at the time was actually exercise in motivated reasoning convenient to speaker (whether practically or just made him feel good and superior) - but not really result of fair unbiased cold look at world.