We could eliminate food inspection and driver's licenses. That would increase liberty, but I don't think most people would find the freedom worthy of the cost.
Food, water, stuff like that have potential to kill off everyone. Drunk drivers on the other hand are not likely to ever pose an existential threat. So I tend to agree with those who insist that, of course there must be regulation, there must be informed discussion, no we shall not consult the i-ching, but definitely loss of liberties require an exceptionally high public good threshold to be considered.
tldr; to keep that line in an optimal space between the extremes, clearly two opposing forces are necessary.
While what you're saying does seem to make sense, sometimes the situation isn't always that simple.