The data so far suggests that the booster grants substantial protection against severe disease with omicron. If anecdotally there doesn't appear much difference in your circle, that's much weaker evidence. I would still recommend it. Also my parents got the booster with no side effects of note, so not everybody reacts that way.