Good point - on the Vive you can easily remove the face padding and rinse it off. They also sell replacements, I have a few so I rotate them in/out depending on which is driest etc.
Can't be all that different than sweat itself. If we're not supposed to be getting these wet, then we shouldn't be doing VR workouts at all.