My experience with high school rugby was that head impacts were exceedingly rare. Lots of bruises, though. The showers after a game were loud, with kids screaming their pain away.

I stopped in college because I didn't have time (or will) for the gym, and it suddenly wasn't fun to play against people several years older than you. I imagine American football would be much worse in terms because of the external protection provided by the padding and helmets. It hurts more to hit skin to skin, so you don't do it so much, and when you do, it's carefully. But in older age, and professional settings, I imagine things are different.