The other problem is that every job goes through phases when it becomes a slog and you just have to put your head down and do it. If you are only motivated by passion, it can be a real struggle to stay motivated during these periods. This can be especially crippling for people who struggle with problems like ADHD or procrastination.
To each their own, but overemphasis of the need for passion might just be pushing societies towards even more workaholism. Being passionate about your job you would probably end up spending more hours, much more effort, making it your utmost priority. We could see these as desirable properties for the businesses, but at what cost for individuals?