Also, Foursquare will constantly send you notifications saying "It looks like you're near Foobar, would you like to check in?". They make it very, very obvious that they're tracking your location.
It was a quick matter of going into preferences and looking to see what applications used location services recently to fix things up. The biggest offenders were Foursquare and the Google app. I ended up turning Google off completely, and disabled Foursquare except for when I have the app open. Problem solved.