I am somewhat similar. The irony is that when you're riding next to traffic, a helmet offers little protection against being hit by a car. The biggest gains are solo crashes against the asphalt, with no cars involved. So if I'm riding a city bike for fun or chores, I may not wear a helmet (especially a shared bike). But if I'm riding the hills for sport, helmet on every time.