Don't give up. I use my lunch break to work on a project. I understand not everyone has the same amount of time available, but at least you can do something in the weekend.
What you want to do is up to you of course, but what it's important is that even 2 hours are 100 hours in a year, not much, but at least it's something.