For some reason around the early 00s peoples love for visualisations fell out of favour. Meanwhile clubs would still hire dedicated VJs and/or lighting guys so didn’t rely on math-based visualisations. Which meant fewer people contributing to developing cool new math.
Worse still, the few industries left that did care about beat detection were games like Guitar Hero, Beat Sabre and those dance machines in arcades; all of which were easier to have patterns programmed by a human due to the limited selection of songs on offer.
It didn't. Music companies effectively killed it because music visualization is a combination of derivative work and performance. So, copyright, licensing etc.
Can't find the relevant link though.
[0] https://github.com/xNul/drop
You might be interested in this btw
https://delu.medium.com/a-perceptually-meaningful-audio-visu...
https://ciphrd.com/2019/09/01/audio-analysis-for-advanced-mu...
They've gone and tweaked the basic formula from just FFT to something that actually does "peak detection", which is better explained in that article than I can here as it involves a lot of math. There's some ideas out there to be sure.
YEARS ago (so it looks kind of fuzzy now) I remember this guy hacked together some synths he programmed with visuals generated for them, it's really psychedelic stuff:
I actually talked to the person who made this and they said this took like 9 hours to render. So not exactly real-time stuff back then, might be now.
The coolest stuff these days is in 3D and projection mapped.
Someone recently posted a link to Terence Tao on mathematical notation. Much the same problem. Notation is hard. Indeed I reckon notation is one of the most difficult problems we have.
That said, visualising Music is kinda like musicalising Art. Possible but not necessarily useful. Synesthesia is one way to deal with it, but is hugely reliant on very human perception and kinda iffy (not knocking those who can experience synesthetically, more suggesting https://en.wikipedia.org/wiki/What_Is_It_Like_to_Be_a_Bat%3F).
A general theory of correspondence of meaning and notation is still unavailable, notwithstanding a vast amount of research in the field. If you can make any useful contribution at all you are worthy of historical recognition.
I've been thinking about this stuff for decades, and can only recommend Charles Saunders Pierce as a good beginning for the modern mind.
https://www.semantic-danielou.com/semantic-danielou-53/hex-h...
https://llllllll.co/t/hybrid-hexagonal-keyboard-for-midi-typ...
http://www.yeco.io/introhexblog.html
https://hackaday.com/2019/07/13/isomorphic-keyboards-with-cv...
https://reverb.com/au/item/18946829-c-thru-music-axis-49-iso...
https://en.wikipedia.org/wiki/Isomorphic_keyboard
https://web.archive.org/web/20120219085411/http://www.thesha...
https://www.dynamictonality.com/hex.htm
I saw some music software once also a few years ago that mentioned this type of input device. Don’t remember if the software in question was a VST or a DAW or what it was so can’t find it at the moment. The software that mentioned it may have been open source but I don’t remember.
I know, bar graphs got a little tiresome so people tried writing more interesting stuff but at some point the "visualizer" became so disconnected from the music that even if you paused the audio the fireworks and "flow fazing" just kept right on doing its thing.
What you need to have that level of visual interaction is the original stems (or something which emulates them relatively closely).
Services like https://www.lalal.ai/ are a step in the right direction, and will probably lead to what you--and many of us--crave. But as it is now, they require way too much computing to provide meaningful enough data fast enough to build a real-time visualiser.
But even then there is the whole issue of copyright which would bottleneck development even further...
As a starting point, you could separate the different instruments, which I'm pretty sure is a solved problem by now.