Except that your decisions about how to live your life are integrally bound up with the infrastructure provided by society as a whole. Individualism isn't exactly what it's cracked up to be.
Being told "no" by society doesn't imply any status of rightness or wrongness, nor is it a reason to get all petulant and start asking "illogically-extending-the-argument" questions.
Go up one or two levels on the scale of life, and we call that cellular freedom "cancer."
Capitalism is the only thing that's really capable of lifting people out of poverty. We need to work with people's selfish desires, not against them.
You left off the part where de facto government granted monopolies helped build the capital base and in many cases directly funded infrastructure investments.