The issue is that someone has to make these decisions and libertarians don't think the government is the group to do it.
The most important institutions in human society — language, law, money, and markets — all developed spontaneously, without central direction. Civil society — the complex network of associations and connections among people — is another example of spontaneous order; the associations within civil society are formed for a purpose, but civil society itself is not an organization and does not have a purpose of its own.
Government is the big hammer and when you start using government casually instead of letting people decide at their own direction that's when you start running into massive problems. Because the government could give you what you want one day but then with that same power take away everything the next.