The freedom you have when you do something in your free time is underrated.
My wife has a childhood friend and a cousin who were both very talented in Indian classical music in their youth and regarded as potential stars. The friend came from a well to do family, the cousin from a lower income family. The friend didn't really need to pursue a college degree in a professional field and take up job to make ends meet and instead could devote the entirety of higher education and focus to music because even if she doesn't succeed monetarily, the family still owned several properties and the rental income and capital appreciation could give enough of a passive income for survival together with say teaching.
No such luck for the cousin who now does music as a hobby when he can.
People need to follow their loves... Dream of being a Rock Star? Play at your local bar. Write a novel.
For some things it's just not possible (or at least way harder than following your dreams in the first place)... the kind of things that require being part of something bigger and having a certain reputation and seniority among your peers. E.g. flying fighter jets or do underwater welding after a career in investment banking, or trying to enter a decade later an industry that was at its infancy in the first place.
There are lots of dreams (probably the majority of them) for which I fully agree with you though.
The risk of suppressing that dream and compromising everything is realizing you have an unfulfilled life.
So following your dream responsibly entails learning a skill that is lucrative but which opens the possibility to fulfill that dream.
The luckiest people are on their path to fulfill their dreams early on in their lives and all the work and effort is alligned with their vocation, all work and effort becoming much easier to attain.