Only 4 are still alive, all in their 90s so that’d be a long time - even if some do have cancer at this stage it’s not likely to affect life expectancy I guess.
So no, pilots or astronauts are not "some of the most physically fittest people in America". They were exceptional human beings but lets be realistic.
Maybe he was just naturally fit. Some people are. But he was undoubtedly fit.
It's the same with F1. "We have the best drivers in the world!" You have the best drivers from the self-selection mechanism you impose on the sport. There are zero reasons to think these categories have good overlap.
No, they're not Olympic athletes but they're considerably more fit than the average American.