When I finished university, I had a few weeks between graduation and my start date at a well known Midwestern embedded electronics company. I had a $7k signing bonus and I found a $500 round trip ticket to Rome, so I went to Rome. While I was there, I learned about the world of backpacking and hostels. I ended up spending 6 weeks in Europe before returning home. During that time I decided that travel was something I wanted to pursue in my life.
The salary at my entry level SWE job was $58k, which was pretty modest. I didn’t buy a new car. I didn’t buy a new house. I cooked most meals at home and I brought lunch to work. I tracked my expenses and budget using Mint, and set a goal to save $30k so I could leave and travel in SE Asia where I calculated the daily burn rate should be around $30/day. After three years I hit my savings goal and bought a one way ticket to Hawaii, then from Hawaii to Thailand. I ended up spending over a year outside of the country and returned home with a $10k cushion to get back on my feet.
The biggest leg up I had was graduating with $2000 in student loan debt, but that was made possible mostly through merit based scholarships. No trust fund.
I inspired a friend to do the same thing, except with a destination of Australia on a working holiday visa. Also no trust fund, just living below his means and saving over time.
My advice to you is to find a way to do the things you want to do instead of limiting yourself with beliefs that only the ultra-rich can take time off from work to pursue personal passions.