This is definitely true. I've been labeled on occasion as "not passionate" for not being willing to work myself into the ground at 60+ hours a week with a four hour round trip commute.
That's not to say that I won't work long hours at crunch time or I have a problem with going the extra mile. But if your "crunch time" is 52 weeks a year, it's no longer a matter of passion, it's just exploitative.