When I moved to the South (anything below DMV (Delaware-Maryland-Virginia) is the south to me, I 've seen cops pull over many people just to check their DL. It's called DWB/L (Driving While Black/Latino) Syndrome ~> http://en.wikipedia.org/wiki/Driving_While_Black. Comparing that to what happens to NYC, I wouldn't classify that as racial profiling. The truth is that some minorities drive wild/reckless when they are young. I use to ride my bicycle on the East Side of Town (Minority Side) and I've rode my bicycle on the West Side of Town (Non-Minority) and trust me, on the Non-Minority Side of Town, they are more cautious, considerate and more concerned.
The cop probably never saw you in the neighborhood, and from my experience in many Southern Cities, if you are an odd miniority in an "affluent community", you look suspicious. Cops just drive by slowly as they watch. Other times, I've seen many neighbors or people in a community call something to the attention of cops. It is happened to me, especially at time whenever I entered a bank to get money in the "affluent community". I would enter the bank without anything suspicious but when I was there, everybody became silent, people looked at me funny and by the time I got out, I saw cops/police cars outside. Might just be a coincidence, who knows?
So going back to being a cop, this is what I think. From the cop's perspective:
1. He never recognized you in the neighborhood. 2. You are the odd one out in the "affluent community". 3. You drove the car, looked at him and went straight for a while (without any destination) before finally going back home, thinking that you threw him off, so if I was him I'll be suspicious as well. Or I'll think that you were lost. 4. Then you went back into the car, thinking that he wasn't following you, so you decided to go back to what you were doing or earlier planned.
FROM THE CONCLUSION UP AHEAD, IF I WAS THE COP, I WOULD JUST FEEL AS IF YOU WERE UP TO NO GOOD AND YOU HAD SOMETHING TO HIDE. MAYBE THAT IS WHY HE ASKED IF YOU EVER BEEN ARRESTED, BECAUSE YOU ACTED LIKE A FUGITIVE OR SOMEONE ON THE HIDE OUT.