I think this is inevitable.
One, Elon is trying to prop up Twitter by using SpaceX money to buy ads. He's dipping his hand into other companies to keep his new toy afloat for a while longer. I don't know if the word "contamination" is appropriate here, but it's pretty clear that Elon doesn't see any kind of sharp boundaries between his various enterprises.
And two, I think a lot of customers are going to start rethinking Elon's offerings. Tesla used to be a status symbol. Driving one meant you were forward thinking, that you were on the bleeding edge, and probably conscientious about the environment. Now, Tesla is "that company Elon owns." There is no way this doesn't hurt Tesla's bottom line, and there's no way investors don't realize this.
All Elon had to do was shut up and bask in the accomplishments of others, but he couldn't manage to take his own ego out of it. Will it ruin him? Probably not. We he still be the richest man in the world in a year or two? Also probably not.