I know people who did research on the side for a decade while having an ordinary job. They then returned to academia.

I couldn't do that, but I know there are people more passionate than me.