I don't have time to listen to all the great music in the world. Even if "God" gave me a list of all the great artists trying to record today, it would take me more than a day to listen to all their recordings. So the labels choose for me, and to a large extent it doesn't matter which ones they choose, what matters is there is a filter to gets the right amount of music to me. In the world being great isn't enough as there are more than enough greats, so if you can great, good looking (this helps me overlook any of your faults - I'd like to think otherwise but I know better), and good in the bed of the label - you make it. If you fail any of the above three you have probably lost (note to the good looking, the labels may use you for sex knowing that you won't make it).
Note too that labels bring in other help for the great. I can record music in my basement: I don't have the best microphones, but they are good enough to pick up the sump pump in the corner. I can mix tracks in my basement as well, lets just say I'm not quitting my day job. I could go on about other parts of the process of turning a great artist into a great recording for mass consumption that labels have figured out.
The problem is that the filters you describe are so narrow that certain genres of music don't get adequately explored, and some genres don't exist at all. In broad terms that's bound to happen, but the missing and under-served genres are so obvious that the filters are clearly limiting the maximum amount of musical enjoyment and meaning that's available to the average listener.
For a dead simple example-- Bach and Beethoven were known as the greatest improvisers in their eras. So where is the Grammy winning album of someone improvising a set of their own variations in the style of Beethoven? Who is the Michael Jordan of improvising a set of 30 new variations atop the bassline of the Goldberg variations?
Don't get me wrong-- there are people who can improvise perfectly well in those styles. But there's no industrial incentive to push those artists to release something truly transcendent-- e.g., something that reveals for a general audience something new about music in the time of Bach's, who makes it fresh, and who redefines what improvisation can be. There's no period improviser so single-minded and driven that an audience of tens of millions would spend lock-down watching a 10-part miniseries on the making of such an album.
Another one, off the top of my head-- what's the version of math rock where there's no common subdivision shared by each line? Computers make this effortless to write, but AFAICT that genre doesn't exist. For just a simple example, Steve Reich's Phasing exists as a prototype-- it shows that there's a compelling perceptual basis for presenting the same melody at tempos that are very close to one another.
There are many other examples, and the filter you refer to means you've never, ever heard them.
So growing old and getting set in a desire for nostalgic music isn't so much a natural law of musical desires as it is the result of creating "The Matrix" for music in the age of the internet by locking your ears into the same old payola scheme from the before times. (I guess the "heat" captured from your attention is to fuel the machine world of the filters... :)
Spotify does a terrible job of recommending new music to me - mostly my "Discover Weekly" is a mixtape of my existing playlists, and "Release Radar" is entirely stuff I hate.
I'm not sure how a new band would reach me, but it's not via the methods they're trying now. I haven't been genuinely excited by a "new" band since, oh, Mumford? Bad Shepherds? something like that.
[0] I don't listen to that much music. I would if I got more new stuff, though, I'm sure
By far the best discovery I've seen was on What.CD, RIP. The similar artists web they had on ever artist page was extremely tight. I tried a few months ago to find the old What.cd data to look into it but wasn't able to find it in a couple minutes of googling.
Look into Last.fm. They do a pretty decent job of recommending similar artists but its more of a what you played tracker than true music discovery.
The movie Yesterday is such a farce because it suggest The Beatles songs are super successful no matter who performs them, as long as they are the first to perform them. When effectively it was a mix of having skilled musicians, produce and perform good songs on the one hand, but also who they were, the marketing and their performances on the other hand.
reminds me of Gunnm / Battle Alita Angel, where the author coined the term panzerkunst as a martial art name.. first time I read about the real life term.
slight anecdote: some people really do have a glow that we can feel but not really explain, something electric and immediate, that will either catch an audience or even stop the air in the room. I don't go to concert often but the few time it happened I was quite shocked. You can put all the best vocal or musical technicians next to them and they would not be able to create that.
Without that packaging music is a very tough sell. Being competent and creative on its own isn't enough.
I can't think of anyone who has made it on musicality alone without at least some of the above OR enthusiastic and regular promotion by gatekeepers (top radio DJs) while they were still an influence.
Some people today do acapella renditions for SNES F-Zero soundtrack.. that says something :)