You really do have to shut out the rest of your life if you want to make a serious run at a project. When you are cramming a side project into you life, you really can "work" 80 hour weeks. The irritability you're experiencing will probably just get worse.
I'm just starting to learn to cook. I began riding my bike last year. I'm 34. The grass is always greener of course. But it seems like you are enjoying a lot of the things I wish I had made time for earlier.
Maybe shut out the art project. Turn your code into open source and let that be your art for the next few months.