I think you are right. It is un-ethical to put others at risk when you could easily avoid that by getting vaccinated.
I personally think it should be a crime or misdemeanor to not get vaccinated because of that.
For a precedent I cite the case of Typhoid Mary. She was forcibly quarantined because she refused to stop putting other people at risk.
https://en.wikipedia.org/wiki/Mary_Mallon
Today's anti-vaccers of course say: "You are taking away my liberty to infect others!"