Nope. I definitely do kill hours at a time with people from work or people I never talked with. :D
My team has weekly happy-hours which are obviously after hours (I honestly thought every team did it!), my landlady often invites us tenants a monthly dinner that goes into the night, I attend random Meetups and just stay for several hours chatting, I participate in our co-working space "parties" and stay until late with people I never seen. Heck, yesterday I went to a restaurant with my (recent) language class and we stayed until 2AM.
Of course you can claim "this is not really hanging out, this is other stuff". I hate cigarettes and don't have a garage, but I guess drinking and random location is equivalent, as long as we chat for hours, so why not?
But I definitely do not consider "chatting while picking up the mail" as hanging out. Not even in the same ballpark.
In Germany we have a thing called "Men's day" where we get a cart of beer and go to the middle of the forest just to do nothing at all for a whole day. Now THIS is what I call hanging out. I wonder if the culture is where the disconnect is coming from.
EDIT: Although honestly pretty much every other country I lived in my life had a similar culture and... it was the rule as far as I could tell.
EDIT 2: I also don't think you're a sociopath! Far from it! I just think you're someone who doesn't enjoy long aimless chats with unknown people.