What I found is that passion is one of those things that always seems excessive in retrospect and insufficient going forward. When I reflect on my periods of professional exuberance, I am always struck by how much I missed out being with my family and friends. Likewise, in the times of lull I am always guilt tripping myself about not being passionate enough, or not giving it my all.
Ultimately, the right balance depends on your own perspective on life, which in term depends on your age, personality, family situation, etc. But the older I get the more convinced I become that you can't go wrong if you err on the side of life.