> There is a world out there about compilers, run-times, drivers, emulators, VMs and OSes that I can't seem to grasp, it is just too complicated for me.
If you've already determined that you can't do something because it's just too complicated, there's a strong chance you'll be proven right. In my experience, all of the things we software people deal with down to networking, hardware drivers, task scheduling, you name it, were created and explained by humans. Nature constrains us with time and space performance and quantum mechanics. If another human was smart enough to create this stuff ex nihilo, certainly with some work you can be smart enough to understand it.
> My Computer Science degree and the newer degrees do not focus on these, they have become 4 year long coding boot camps that focus on getting people job ready with AWS, React, Ruby or whatever trendy. This is true for the majority of the not so prestigious universities out there.
No one is going to give you a checklist for how to be creative or pursue your own goals. You have to do it. If you're capable of asking this on HN, I think you're capable of setting a few goals. Just write them down and put the deadline a long ways away.
> I see a pattern and I feel the older generation is way more capable and knowledgeable when it comes to Computer Science in general. I am pretty sure a lot of people are or were in the same boat as me. I don't really have the time(Have wife & kids) to take a compiler course or set out to build my own OS, what would be the easiest way to learn these lower level things?
I'm in my thirties. When I was in school, there were a lot of kids who were just punching the requirements and doing what they were told to get by. Any time they got stuck, they'd have to ask the professor or TA in office hours. Others of us learned that we could solve a lot of problems ourselves if we cut them down to their essences and wrote small programs to see what was actually going on. That heuristic was very helpful.
Once I really did get stuck after doing this, and went to office hours. My professor (an old school Unix graybeard) was so thankful that I had put in the leg work of examining the problem with "toy programs" as he called them that he gave me an exceptionally detailed explanation for what was going on, and how I could show this to myself. He made the comment that a lack of curiosity was a problem among my peers, but that it can be grown and cultivated like other skills.
One other thing the older generations did better than we do is look at the history of our field. If you don't know who Alan Kay, Doug Engelbart, Nikolaus Wirth, Alan Perlis, Ivan Sutherland, Leslie Lamport, Margaret Hamilton or Ted Nelson are, go find some information about them (most have talks or demos on YouTube). Then find out the rest of the missing pantheon of computing. Our industry goes through fads in sort of a fashion culture, and in my experience it leads people to feel burned out our stuck exactly as you described. There are some enduring truths which our field does a poor job of passing onto posterity. Turns out more people had interesting ideas beyond Dijkstra and Turing.
> 1. Reading the Dragon book(I find this would be useful only once you have built a bad compiler) 2. Paying people older than 40-45 to teach me some C/C++ and some tricks of the trade (expensive but I gained a huge amount of knowledge in a short time)
My advice: pick a small, interesting problem and tackle it for some set amount of time each day, maybe after the kids are in bed or your wife is enjoying a bubble bath or whatever. It doesn't have to be long - maybe a half hour to an hour. Just do it consistently.
I've found that GitHub, YouTube and HN itself have loads of interesting project ideas and tutorials. I find those much better than just reading a book. In those cases, the book becomes supplementary material, and is usually most useful at exploring theoretical issues, edge cases, history and other more detailed topics.
If you are going to read, always do the exercises. Also, keep in mind that a lot of the books recommended like the Dragon Book or SICP or whatever else are typically a very thorough treatment, but may not be great for a great read. (There are exceptions to this: I find Tanenbaum's OS books to be highly engaging, interesting and helpful. He is both a practitioner and a theoretician, so I find his perspective valuable.)
Not only is my understanding better with experience (rather than just reading), but I find that I feel accomplished, which is a virtuous cycle. And I also find that I learn enough to be able to discover other interesting things for myself.