I think my favorite stupid argument though is that it increases the size of the rider's head, which increases the likelihood of ground collision in an accident. As though it's so easy to stop yourself from falling as long as there aren't two inches of material strapped on your noggin.
The basic fact is that if you do have a head injury on a bike, a helmet is likely to reduce the severity of it. Everything else is an unquantifiable argument about human behavior.