My Grandfather was born before the days of safety razors and tells me that Navy personnel were able to petition to avoid shaving on account of delicate skin (something about the salt water and fresh air). I think anybody who could grow a beard would have done.
https://en.wikipedia.org/wiki/Danger_triangle_of_the_face
I never thought that mustaches had a health / self preservation / natural selection origin, but it could very well have been the case. Shaving every day without modern tools and products could easily cause infections, which in that area could prove fatal.
Also special forces are expensive to replace and are out in areas that could have NBC related threats and have some of the biggest beards I've ever seen.