There's two general reasons for a person to get a vaccine. One is that it makes them healthier, and the other is that it makes society healthier.

If the person gets to choose, then there's a natural counterbalance against things that are harmful to the person but good for society. Only an idiot would turn down a vaccine that has a 90% chance of saving their life.

If society gets to choose, then forcibly sterilizing poor single mothers/disabled people /those of poor breeding/etc might be on the table again. (It's a health intervention that is bad for the individual but also leads to a healthier next generation)