In my book, negative externalities are worse than taxes. If we agree that a car driving in Lower Manhattan inflicts a non-trivial cost on everyone else, then not taxing it leads to socially inefficient outcomes.
Effectively, locals are paying a price either way - either by having their bus moving slower, inhaling fumes, etc, or by buying goods and services that reflect congestion pricing. The difference is that congestion pricing aligns incentives - for instance, delivery drivers may choose to travel to Manhattan during the off-peak hours whereas in the status quo they do not care at all about inconveniencing others.