I feel like if people truly cared about what you had to say they would seek you out regardless of the platform. Follower numbers are glorified internet points meant to keep you engaged/chained to the platform.
Without that large network, the only people seeking him out will be fans. There are at least two negatives to that. The audience is smaller, and there's less exposure to non-fans
One former president has kinda sorta managed it, and he was in the news every single day for years. Who else could achieve even his limited level of success? Maybe Barack Obama?