You may have been right at a time where almost nobody had a previous infection. However, estimates by reputable sources https://covidestim.org/us show that a large percentage of people have been infected by covid at least once. The data are by state, I see ~75% for MA, 80% for CA and NY, and you can pick your favorite state.
At this point it is more correct to say that the vaccine does not matter, and immunity by prior infection is the dominant effect. Either that, or the Yale school of public health, the Harvard school of public health, and the Stanford Medicine institutes who run the covidestim.org site are full of shit.