We have a problem where regulators are bad at their jobs most of the time, and then people develop the heuristic that all regulation is bad.
You need some rules to price major externalities. Not minor externalities, because regulatory overhead and enforcement are themselves externalities and it doesn't do any good to burn $1 in overhead to prevent $0.90 in some other cost. But you want a ban on leaded gasoline.
And you need antitrust rules, because "the market is more efficient" is fundamentally predicated on competitive markets. Real competition is the sine qua non of that actually working. Most government inefficiency is because the government is a monopoly, and private monopolies are just as bad.
The problem is politics breaks everyone's brains. One party says "regulation good" and the other side says "regulation bad" before either of them considers what the regulation actually does. And then you have one party promoting illiterate nonsense like price controls or justifying busybodies who want to micromanage things they don't even understand or trusting the government with mass surveillance data just because they're not a private company, and on the other side you have people with no objections to tying arrangements or companies with double-digit market share buying even more of their competitors.
"Everything should be made as simple as possible but no simpler." That last bit turns out to be kind of important.