The NSA isn't a law enforcement agency. Rightly or wrongly, US intelligence agencies seem to have a rather different set of rules these days than the rest of us are following, including most law enforcement. Bottom line is if they're wanting to use it to convict you of a crime they'll need a warrant for it (before most judges, at least).

If you're worried about NSA (llegal) spying in general, presumably the carriers retaining the data only makes it a bit easier for them. Since the info will need to be traveling around the network while you're active, they could easily just intercept that like they intercepted the voice calls.