It takes more effort to wear an N95 than nothing. But it’s certainly not impossible, and for most people (including my overweight, out-of-shape self), it’s not insurmountable.
It’s like wearing pants. You’d have a lot more mobility if you didn’t, but we do it anyway.