I think it's more than retail investors. I think it's the hype and noise that his followers make, plus whatever noise he himself makes.
I think he's a bit like Trump in some ways. Trump was often said to represent an image of what a certain demographic of people thought was rich and successful. He made a lot of noise, threw around a lot of money, and his followers in turn made a lot of noise.
EM(I write it as such because I'm sick of seeing the name everywhere I turn) seems to have the image of "tech guy who struck gold" and "can't stop making money". While some of his success is definitely attributable to himself, I feel like people give him too much credit and celebrity status. I think the biggest drivers of his success are a gigantic war chest of money, combined with a massive following, and enough ideas to spend the money on.
There are hundreds or thousands of people in the US alone who are smarter, and have better ideas, and don't make asses of themselves in front of millions. What they lack is his money and fame, so we'll never see what potentially world-changing things those people are capable of. Unless they get hired at one of Musk's companies, in which case people will attribute someone else's genius to EM. It's like attributing innovation to a VC instead of a startup.
A fun strawman, don't take it seriously: Musk is just the Silicon Valley version of Trump. And instead of starting his own Twitter, he flaunts the idea of just buying the regular Twitter.
One other nit I pick is that I'm personally a bit salty over his $DOGE tweets. I had decided that I should accumulate some for some next "big wave", but that wave happened to be some rich guy who probably didn't own any $DOGE, and it had happened before I could get hold of anything. I "lost" potential thousands of dollars because some guy sent a tweet.