That assumes you want the benefit.

If living a 'natural' life is higher on your value scale than preventing a possible infection, then you wouldn't want the vaccine even if no one else was getting it either.

Let's say your neighbors want to build a $1M lake. They want you to chip in $50k. You're going to get the 'benefit' of the lake one way or the other. But you don't even like lakes. In fact, you'd prefer they didn't build a lake, as lakes attract mosquitoes, and you are (perhaps irrationally) concerned about infections from mosquitoes. Would it be selfish for you to not chip in? Would it be rightful for them to take the $50k from you by force since you're getting the 'benefit' of the lake?