You can fly VFR without services and you really only have to worry about yourself. There's no equivalent as an ATC. You're often working multiple frequencies on your own. Even if you don't typically do it on your own, if your coworkers gets sick now you are. I've had friends manage two tower frequencies as well as ground due to staffing issues.

Not to mention the fact that when you become a controller it's much like the military where you get assigned a place and you go there. You get preferences of course, but there's only so many suburban airport controllers needed - somebody needs to be in JFK tower and somebody needs to be in the middle of nowhere Montana and they're not flying in for their shift the way pilots can.

I'm inclined to trust the FAA's data that it's just not feasible to dump training resources into someone who statistically is either not going to make it or is going to quit if they don't get their preferred assignment and find out they need to move their family to Utah.

I assure you the mental picture you need to keep in your head as a controller is far more taxing than single pilot operation when you are primarily concerned with your own separation. A controller can be managing a 1 to 2 dozen aircraft, dealing with pop up IFR, flight following, traffic advisories, handoffs, etc. There's a reason you get a mandatory break after 2 hours.