1. This is mandating that vaccine companies make more money, the incentives are suspect. We might reasonably expect people who already had COVID should have better immunity than the vaccinated since the vaccination is basically just a simulation of having COVID. Why not then be suspicious of reasoning motivated by unscrupulous profiteers? People will be getting vaccinated who don't want or need it, which raises an eyebrow. This policy is probably a fig leaf for government handouts to big pharma.
2. The evidence changes. We don't know what the 2 year effects of COVID are yet since the disease hasn't existed for 2 years. Even now week-to-week what everyone knows can change. I do not trust that the scientific basis for this decision will still be sound in 12 months - the data here is raw.
3. What about the argument that people have a right to control their own bodies? There has been a concerted push by reasonable people for that, eg, in the area of abortions. If the NYC government can demand vaccination for the greater good then it is basically the same logic as demanding carriage of babies to term for the greater good. We know a lot of people disagree with that logic.
4. You also haven't distinguished between (1) thinking that the vaccine is a bad idea and (2) thinking that governments feeling empowered to mandate the most experimental vaccines in a generation is unwise. It already appears they don't work as advertised given that having taken the vaccine doesn't acceptably improve their risk profile vs. COVID (in the NY government's opinion). Which I think it does.
5. The far more effective measure would be tight border controls to prevent even worse variants - the biggest risk here as far as I can tell has always been something like a delta variant appearing and breaking through the first generation of vaccines. But defending against that would inconvenience the people making the decisions, so it is unlikely they'll think of that. The decision makers aren't serious about stopping the virus, so their decisions made to allegedly stop the virus are suspect.