As you get older, you are more valuable. No "silicon valley" syndrome about age: you'd never have to dye your hair, wear a hoodie to fit in, nobody will blink if you have to take a day off to take care of the kids.
You can be a real adult - nobody comes into those programs and expects a ball pit or a foosball table, and nobody seriously thinks someone right out of college but who is up on the "latest framework" could take their job - because the jobs are deeply, deeply technical and require lots of experience. If you blow the doors off of everyone else in programming, great! That's one dimension. But it usually isn't the only one.
"The pay is worse" - for software, undoubtedly. For hardware? It's usually a wash after adjusting for cost of living and the benefits are better on the defense side.
I like defense because the systems are usually pretty badass. We have people leave for FAAMG occasionally, so it's not like we don't know who the people who are "that good" are! :)