The Neanderthals are thought to be many times stronger than modern humans, judging by the thickness of their bones and the size of the muscle attachment sites, as well as their likely lifestyles in the brutal Pleistocene era.
If one male were to strike another with full force, it would more than likely cause concussion if not death. In those days, you couldn't afford to lose a healthy male from the tribe, even one you're pissed off at for wanting to mate with your woman.
Probably ditto for other ancestral variants such as homo erectus. They weren't exactly sophisticated, but you have to logically assume that a population that tolerated high levels of violence would be more likely to die out. A population that tended to resort to symbolic confrontations like shouting and gesturing and jumping would be more successful.