I also suspect his topics just appeal to young men (fitness, comedians, Elon Musk) and that accounts for a portion of his base.
Embracing ambiguity is fine when there is genuine ambiguity. Portraying minority or fringe ideas as 'ambiguity' is dangerous.
The problem with what you're saying is that you're setting yourself up as an arbiter of reason, or sanity, or safety in the realm of ideas. This is a fine aspiration, but you can clearly see that not all, or even many, will accept your analysis.
So you go into problem solving mode - and end up with government, or trust and safety boards, or religion, or any of the myriad of communities humans develop to corral the marketplace of ideas.
All of those systems suffer from humanity's profound fallibility and until that stops being the case, actually talking out all issues will be the only reasonable method of handling freedom of speech.
Some limits are obvious, but because human fallibility is so profound, we should be equally as profound in allowing ambiguous ideas the space to be understood and charity in interpretation. To do otherwise is embracing ignorance and folly.