(1) It can be a fair chunk of your audience -- 1/10 men are colorblind in some locales.
(2) There are many flavors of colorblindness. Tools like https://www.color-blindness.com/coblis-color-blindness-simul... are helpful to make sure your palette works for most of them.
As a practical matter, adjusting some parameter like texture in addition to hue is helpful. If you adjust lightness or saturation it'll serve the dual purpose of making your charts understandable when printed in black-and-white.