I, for one, wholeheartedly endorse this plan. I think it's a mistake to expect to derive significant joy / self-actualization / gratification / whatever from your "job". A "job" is just a means to pay the bills - put food on the table, buy gas, pay rent, pay electric bill, etc., to enable you to do the things that you actually enjoy doing. I mean, sure, if you happen to enjoy your job, then that's a nice bonus. My problem is with the idea that your job should the primary focus of your life.
Likewise, if by happenstance the "thing" you do outside of work happens to be very similar to what you do at the job (eg, "programming", just in a different domain) then so be it. That's fine. It's fine to say "I love coding for AI research I'm doing at home, but hate coding the same boring CRUD app at $BIGBANK for 9 hours a day." The big thing to watch for here is IP conflicts, especially if you intend to commercialize what you do at home (or think there's even a chance you might).
If anything, count yourself lucky for coming to this (apparently) fairly soon in your career. It took me a long time to figure that out.