Chris Boardman, an Olympic cyclist for GB, recently appeared on BBC News to talk about cycling safety while not wearing a helmet during his demonstrations.
He gives his reasons in this short video https://www.facebook.com/video.php?v=974497519231053&set=vb....
I certainly don't disagree with him, or with you, that there are plenty of other safety aspects to consider while cycling, but I'd always wear a helmet anyway.