There are two factors now which make things different:
1. As other commenters have pointed out, the ease of spread of any disease is orders of magnitude higher now than it was during cave-man days. With crowded cities, public transit, and air travel, pandemic-level diseases can spread across the world in a matter of days or weeks.
2. Much more importantly, our use of medicines (antibiotics, antivirals, antifungals, etc.) have put extreme selective pressure on pathogens to evolve to become resistant to them. We are, in a sense, selectively breeding pathogens that are both good at infecting humans and resistant to our treatments. That is a focused selective pressure that has never existed before, and may pose the biggest threat to us in the next 50-100 years. For a concrete example, look at this experiment which shows how quickly bacteria can evolve to overcome levels of antibiotics that are orders of magnitude more than we would ever give a human [0].
Yes, there will generally always be some subset of the human population that is resistant to any particular disease, but our measure of success is not "5% of humans will survive the next pandemic, so we're all good, let's it move along nothing to see here." If we start to regularly have drug-resistant pathogens that make hospital stays very dangerous for patients (imagine having a 1/3 or even 1/10 chance of dying at a hospital due to an infection entirely irrelevant to your condition) or just more common COVID-level pandemics, that would be a serious disruption to humanity, significantly increase general mortality rates across the board, and drastically change life as we know it.
[0] https://www.youtube.com/watch?v=w4sLAQvEH-M