Again, I get it, and you're not wrong. In theory.
But in practice, is Medium currently helping people build audiences in a significant way? My experience suggests no. It was easier for me to do that with a self-hosted site.
I'm not sure exactly why this is -- maybe it's that Medium's discovery algorithms take into account correlations between existing readers, so if you don't already have a bunch of readers you'll never get picked up. Maybe Medium is just super crowded, so the odds of you getting recommended over someone who's already popular are low.
All I know is that my readers are return viewers (which self hosting helps a lot with) and people coming from Hackernews, Reddit, Facebook, or curated link aggregators. I don't personally see any data that suggests Medium is driving any traffic to my content. And at this point, universally across the board, my self hosted blog performs better than my Medium blog.
I'm sure that varies from publisher to publisher, but I think it's simplistic to say that Medium by default drives traffic. I suspect that for small sites it's a lot more complicated.