Someone can certainly enjoy their job--and software development is certainly nothing special--without also wanting to spend their nights and weekends doing the same thing. For that matter, I'd probably take such a narrow set of interests as something of a disqualification.
It doesn't mean that you have a narrow set of interests. It just mean I use a part of my free time to code. Others are watching tv, playing games or doing other time wasting stuff.
There's no need to say X is "wasting time" in such an objective tone. For many, spending hours on the weekend doing programming is a waste of time (and it can be, most of the time). For you, it's fun. For others, they just don't have time to do it. No need to judge or expect others to live the same life as you.