In my book, negative externalities are worse than taxes. If we agree that a car driving in Lower Manhattan inflicts a non-trivial cost on everyone else, then not taxing it leads to socially inefficient outcomes.
Effectively, locals are paying a price either way - either by having their bus moving slower, inhaling fumes, etc, or by buying goods and services that reflect congestion pricing. The difference is that congestion pricing aligns incentives - for instance, delivery drivers may choose to travel to Manhattan during the off-peak hours whereas in the status quo they do not care at all about inconveniencing others.
Thought experiment: If we could somehow ban private residential use of all cars and only allow "work" vehicles, there would be no congestion and no tax.
It would also increase the incentive for people to play games like claiming their personal car as a "work vehicle", throwing a little advertising decal on the side, things like that.