Have you thought about trying to get a job with a race team?
Source: live near many teams just outside of Charlotte
Acting on it, though... :-)
Life seems balanced now, I get to scratch my itches, not unhappy at my current job, and I don't feel like I'm saying "no" and making excuses. A tiny step here, a tiny step there, worst case it turns out to be a pleasant stroll.