There was a link provided earlier to opt-out.
http://oo.apple.com.
If you want complete anonymity, don't use a cell phone. Its unrealistic in today's world to expect it while also using connected devices.
EDIT: This is reality. If you think the loss of anonymity its wrong, I don't disagree. My comment was not to promote a personal opinion about what's right or wrong, but simply recognizing the obvious state of affairs. In today's world, some government entities do not believe privacy transfers to electronic mediums. In fact, the FTC has recognized that current law is not keeping up with new technology and mediums.
http://news.cnet.com/8301-13578_3-20008422-38.html