Clearly it's a spectrum, not a binary. I doubt you'd say we shouldn't impose more precautions during an Ebola outbreak than we would during a normal winter cold & flu season.
Precautions are generally proportional to the risks involved. Personal responsibility ceases to be the only determining factor in what precautions to take when an irresponsible person imposes the consequences of their poorly judged actions on other people. Your freedom to swing your fist ends where my nose begins. Punching me in the nose isn't justifiable simply because I will, at some point, die regardless of your actions.
That said, it is a spectrum. Which means, if you want to avoid absurd extremes then you must draw a line somewhere on that spectrum. Assuming you don't espouse political anarchism or anarcho-capitalism, approaching me & swinging your fist at my nose while we're both walking down the street is would be on the wrong side of the line for pretty much everyone. If I approach you and tell you I'll stab you if you don't hand over your wallet then swinging your fist at my nose is on the other side of that line.
COVID is clearly more complicated because in addition to the above spectrum, there is the spectrum of a probability distribution that dictates the likelihood of spreading the virus if infected, and the likelihood of serious illness or death if infected by someone spreading the virus. Drawing the line between acceptable imposition of precautions is therefore much more complicated, and absent absurd extremes is necessarily going to be drawn somewhere that some subset believes is authoritarian, and others believe is too permissive & intrudes on their own freedom from the negative consequences of other people's choices.
In all cases though, if your concept of "No one “wins” against things like death, disease, etc." were to be the guiding force behind societal decisions, then literally any action is permissible.