I find the whole "if you love it then you don't need this" sentiment at best naive, and at worst arrogant.
Sometimes we all have to do things we don't want to do. This includes, at times, working on your own project. There's nothing wrong with you, or what you're working on, if you actively try to limit your distractions.
If your project is so awesome you never get distracted, good for you. Kindly, take your hubris elsewhere.