This is exactly the point. Freedom to direct your energies toward your personal passion _without artificial restraints_ is equality.
What constitutes an artificial restraint?
For classical liberals, that means being free from the constraint imposed by other individuals.
Today the definition of "restraints" is being expanded beyond the artificial to the point where everything is being declared a "human right" without any consideration for the fact that some of those things can only be provided by taking from someone else (resources and/or labor). Basically, more and more people feel that people should be free from natural restraints like the responsibility of having to provide for oneself (or at least trying to). But the rub is that the only way to free some people from those natural restraints is bondage of others to provide for them.
If you are capable of solving the toughest scientific or technical challenges, but you are prevented from doing so because of some attribute of your circumstance (your color, your gender, your age, your wealth level, your current job title, etc.), then you are being artificially restrained.
If you can solve the toughest challenge, but you choose not to because you would rather throw clay on a pottery wheel, then you are also free from constraints. (There are plenty of people that are practically forced into careers because they are technically capable, and because their elders believe that if one can, one should.)