I've been (and am) there. My undergrad was at least half ecology with plenty of field work (in the states and briefly in Africa), and one of my graduate degrees was also in environmental science, but somehow here I am in a windowless concrete cube for the last 2.5 years writing code and building models. When I was finishing undergrad I was looking at park ranger type jobs, but it turned out that with a couple of wars on those jobs were almost impossible to get for someone who wasn't preference eligible. I could have gone to grad school more explicitly for fieldwork-related disciplines, but I wanted to focus on getting a solid set of technical and policy skills, which ended up with me in an office in the south, driving everywhere and getting fat.
After about a year of that I really buckled down and focusing a lot more free time on exploring the outdoors and the ecological and geological systems I enjoyed in undergrad. I end up spending a lot of time outside: hiking, camping, backpacking, even just relaxing in a hammock, and exploring the natural processes I know as best I can in a somewhat-informed sense (Why is that gullying? What's that on this tree? Was this burn prescribed or accidental? How old is this rock? What is this rock? How hard is this rock?).
Basically, you never know what path you're gonna go down, but it's not the end for you to pursue physical, hands-on knowledge on your own, you just won't get paid to do it. The motivation can be often hard to come by, but when I'm out there and get to take a nap on top of a mountain, clamber up onto a boulder mid-river, or get that perfect picture of a bee pollinating some wild flowers I feel quite at peace, even if the rest of my week is in that windowless concrete cube.