Taxes are a crucial pillar of even a purely capitalist society, it’s just that they are not easy to implement politically.
To your point, onerously high taxes could largely remove plastics from the waste stream, mooting the need to determine the cost.
I'd like to go even further. Incentives and prizes and grants and whatever-it-takes to find alternatives. eg Bulk pills in plant-based baggies, instead of monthly oversized labeled bottles and caps.
Nudge the culture and expectations. Every little bit would help. Because better is better.
In theory, everyone should be on board. The fiscal conservatives should want it, since negative externalities are a violation of the non-aggression principle, and taxes are the least intrusive way to encourage change, it's basically delegating the specifics of the change to the market. It is consistent with their principles to want it. It seems like the easiest way to potentially get the most people on board.
I also genuinely believe it's the best solution most of the time. The market is ingenious as long as prices accurately reflect costs and benefits.