Internet libertarian here. That's not the argument at all.
We could prevent tons of harm by tightly regulating all aspects of life. Banning fast food or cigarettes would undeniably be a public health win, and I don't think many libertarians would disagree with that.
The argument is that it's simply not the government's job to prevent people from hurting themselves, even if it could be done at scale and effectively. It's a question of the role of government, not so much about whether loud music or fireworks cause harm (they do) or if it's enforceable (it is).
If you ban everything that's dangerous or unhealthy, I'm not sure you'd be happy with the culture that results. Humans are experimental by nature, even if sometimes that results in harm. These types of restrictions that can't be bypassed hurt our experimental nature.
In the beginning of the pandemic, and even now, a lot of attitudes were shaped by the lack of government regulation. If the president isn't wearing a mask, and there's no legal requirement for me to wear a mask, then it must be safe to not wear masks... right? That's the kind of thinking that a highly regulated society produces because we've insulated people from danger for so long. At this point, it seems most people feel that if you can do something, it must be safe, otherwise it wouldn't be available for you to do. That's even more dangerous than loud music or fireworks.