Well, I think my position is a little more nuanced than that. I think it's heavy-handed [given our current circumstances]
and restricts individual choice [excessively]. Obviously we have no absolute freedom in society, and individual choice is constrained to varying extents. The question is what extent is appropriate.
Being discriminatorily restrictive is fine, in my opinion. If you can't afford to pay the costs for your action, you shouldn't do it. The smoking example is a tough one, because if it gives someone else cancer due to second-hand smoke I don't think you can put a price on that.
But let's say in the car case we are okay with a certain level of pollution (we must be), and we have a floating congestion and pollution tax that was assessed hourly (like surge pricing), based on both how many cars were on the road, and the global air quality level (just an impractical example). When roads are particularly busy, and a lot of CO2 has been released, let's say the cost of driving into the city that day spikes to $1000. In that case, if a rich guy wants to pay $1000 to drive into the city, that's just fine. That $1000 goes to building public transit infrastructure and environment cleanup or whatever else. Poor people will be restricted, but they're already restricted. They can't fly private jets, or helicopters, or drive Ferrari's that have 12 cylinder engines and 500 horsepower. Restricting the options down the level that everyone can afford is impractical in a market economy.
> for people who don't have alternatives available
Well, I've also said repeatedly that in order to restrict car usage you first have to build a robust (affordable) public transit infrastructure. My main opposition to an outright ban is most cities don't have sufficient transit to support it. A good way to fund such a project, in my opinion, is with the aforementioned taxes from rich people who want to drive private cars. However, I also think if the costs are appropriately priced, that everyone benefits from the individual who wants to drive a private car, as society gets the income to offset those costs, and he or she gets to drive a car. I may be in the minority here, but I don't see a future where there are no private cars as practical, or desirable.
> because your freedom ends where others' begin, and it's becoming pretty clear that car usage today is in fact encroaching on other people's freedoms
I don't know if that's that clear. Maybe we have a different understanding of freedom.