I am with a small company focused on compilers and managed runtimes (JVM), and I can tell you that _good_ compiler consultants are in high demand. For instance, I had a discussion with a CEO of such specialized consultancy looking for GCC specialists last year, and he said that LLVM engineers are even harder to come by.
I also see processor startups popping up all the time that need compiler/tools engineers badly.
Overall, I'd repeat what others have said: "save a lot and invest your savings wisely."