And even there, you could say everyone is "stealing" from the public goods all the time. (if you hunt and eat an animal, then you have stolen it from the rest of the people).
What is a public good? Who decides what is public?
If I hunt and eat an wild animal, i.e. one that was not raised on a farm, I am not stealing it from anyone. By your definition the only way I am not stealing it is if I kill it and divide equally to everyone.
To pay less taxes you could either relocate to some place with less taxes (this is quite possible in the US, since state and city taxes vary). Or you could work democratically to change the system. If you really want to live in any place approximating anarcho-capitalism, you would have to move to a 3rd world country.
If one could opt out of taxes, that would be the end of the tax system. Of course no individual wants to pay for the public services. Everybody want the other guy to pay! In the same manner, you could ask why one cannot opt out of the penal system. --- As to the "hunting as theft" issue: as long as there are plenty of animals to hunt, you are not taking from anyone. That was the state of affairs 100,000 years ago. When resources are limited they must be managed/taxed in some way. Humanity has not solved most of these issues at this point. See also: http://en.wikipedia.org/wiki/Tragedy_of_the_commons
So you could ask: when did I agree to those japanese fishing vessels depleting the fisheries next to my shores, do I have a choice in the matter? Their fishing raises the price of wild fish and serves as a de-facto sushi tax, only you don't get to eat any of the sushi...
My point is, we are not in the paleolithic period anymore. The world is too complex and resources are too few for fully individualistic economic philosophies.