People wouldn't care except that your "personal" decision affects them.
(1) Everyone pays for the public health care system e.g. hospitals. So if you're increased risk of serious disease results in increased visits or treatments then that's my problem as well.
(2) Everyone pays for insurance. So if you're making premiums more expensive then that affects me too.