Car drivers often get head and brain injuries if they are involved in a collision with another car or with a tree/building. Do you wear a helmet when you drive? The helmet doesn't add almost any inconvenience, is reasonably cheap and light, as you say.
Unless you're on a racetrack or driving a test vehicle, probably not.