I saw an image on Facebook yesterday that said "Men with beards are sexy", with the same man in three stages, beard (closer to stubble), moustache and shaved.
It's a thing. I've been told when I don't shave and have a bit of "designer stubble" that I'm more attractive and look older. I get an itchy face though, so it goes.