I'm not sure why you are defining a 'real social life' as socializing with non-hackers. A 'Real Social Life' consists of hanging out with like-minded individuals. I would imagine that most people spend their down time with co-workers and school friends - people they see on a regular basis and build relationships with. It would follow then that if you are a programmer, you will be spending most of your time with other programmers.

Unless you mean that you are only talking with them online and stuff. In that case, you should probably just go out and get drunk or something.