I tried having jobs that I was interested in when I was young, but making it my job made me hate it. So I've spent the last 20 years working in IT because I dislike computers, and always have, except a few years when I was 13-15. There is no loss if working makes me hate something I already dislike--but if it was something I love, and work made me hate it, then it would be tragic.
The point (for me) is its the moment you make it a job, it becomes drudgery. so the "job" part is what I would need to avoid.