There will always be hyper-nerds and just super smart people who will study all of the low level stuff. Granted, they will be vastly outnumbered by high level programmers.
I am continuously hiring new people, mostly senior devs and tech leads. Maybe one in a hundred has any understanding of what virtual memory is. Or why one process can't access memory of another process and, if you really want it, how to set it up.
Two decades ago that was common knowledge.
It went away just as the knowledge of how networking works. Currently, if they try to open a connection and it does not work they are pretty much at a complete loss to what happened, where and how to fix it. Aside maybe from a simple problem like DNS resolution error (and not even that -- most devs don't understand how DNS propagation works, for example).
So if those problems happen, they are mostly deferring to me as a guy who can solve every problem. I see no clear path to achieving a goal of having someone else to learn to do the same.
If at least was able to purchease a home I wouldnt have this pressure.
I guess in the US is different because people have higher salaries.
You have to have the time/resources to dive that deep. Or maybe it's your job, line some of the folks in a local company that work in hard problems.
But even they are not that well payed.
It's certainly much easier to find the time before you have to support yourself
Finding time to learn if this is not your day job is another problem, but a solvable one if you're willing to learn.
It’s not too different from pro tennis players, who typically start playing before the age of 8.
For those at the top of the game, computing is a calling as much as a job. They will do substantial learning on their own initiative.
Your latter point is far more important to the matter. Those who treat it as a passion more so than a job, are more likely to be the trendsetters. Growing up and being free from responsibilities makes it easier for that thing to become your passion.
And let's also not forget, a few decades ago, computers weren't exactly a cheap thing for parents with little understanding to let their kids tinker with at will. Being born in a family with enough wealth to get a computer, enough wealth / understanding to let a kid tinker with it, was an immense boon. A long with anything that type of family tends to have going for it alongside wealth. It's not that far-fetched an idea that it's the other things, rather than the early age interest of the kid itself, that got them into such a position later in life.
I think it's important to keep things in perspective here - when I took my first programming job the first week was setup. On my team, our onboarding has you branch, review and merge on your first day. Those abstractions allow for people to focus on the areas they're working on, and iterate there. A game developer doesn't need to understand the complexities around js type conversions, any more than a js developer needs to understands rendering thread latency in a browser.
On every team I've ever worked on, that headspace from the team would have been much better served by understanding the product and project, IMO.
It's not about any particular detail, obviously.
He himself does not ever have had to previously see /proc/pid/mem before in his life in order to solve a problem involving it.
However many times a day he gets a question of any kind, that's how often.
also, devs seem to not understand that a computer network is basically a massive distributed system (on the routing level anyways), and that things inside this system can and do fail nearly constantly.
things like latency, endpoint connectivity and available bandwith is not a fixed given!.
So many devs in my experience seem to write code which assumes the network is always available with the same characteristics as the point in time in which they wrote the software.
Not to even mention devs are even entire companies building HA mechanisms which rely on ethernet connectivity to work, making it very hard if not impossible to stretch them properly across network segments/locations without stretching ethernet. (which brings its own set of problems).
Assume everything is broken, all the time.