You don't even have to connect to an open WiFi AP, or any AP. As Glenn Wilkinson demonstrated some time ago [1], the probe requests your device sends out can in many cases, identify you or at least the important locations you've been to. Basically, if WiFi is enabled on your device and it's not associated with an AP, it's constantly sending out requests asking for the [E]SSID of AP's it's previously connected to.
I also recommend this video: http://www.youtube.com/watch?v=03iEaKPRb9A