Twitter also used to be owned by average people. Like most publicly traded companies. So no, it's not from "many rich shareholders" to Elon. It's more nuanced than you're making it out to be.
Then all public companies like Boeing, Lockheed Martin, & Northrop Grumman are owned with the money of "average people"...largely managed by fund advisors.