Unlike COVID-19 which is spreading a lot in the US (due to lack of coordinated government action) and which other countries have more or less stamped out and are controlling (Japan, Korea, New Zealand, Australia, Taiwan etc.), a vaccine will actually be given to something like 80-90% of the population.
So whereas the US has currently had a total of about ~10 million cases to date, this represents about 3.3% of the total population. There have been about 200,000 deaths so far, so the US case-fatality rate is about 2% - note, the mortality rate is most likely quite a bit lower overall since it's spreading so much. This is simply deaths vs. observed cases.
EDITED(got the math off by a factor of 10 here previously - at 0.01% fatality it's not 360,000 - that said, see comments below, actually establishing that in clinical trials is not easy): Let's suppose we rush through a vaccine that turns out to cause fatal complications in 0.01% of the population, and we give it 90% of the population. Straight up - we've just killed 36,000 people. But that's assuming that the complications are binary - but COVID-19's complications aren't binary, and neither would negative side-effects be. So we might kill a bunch of people, and injure tens of thousands more seriously. Or we might have inadequately diverse trials and miss a whole population of potential negative side-effects (a classic problem in a lot of trials: university aged students with the distribution of whichever university developed the product are over-represented in samples).
You cannot rush clinical trials. To even stand a hope of getting to 0.01% risks you need a population of at least several thousand in your trial. You need to monitor them aggressively because hopefully you have good data that whatever can go wrong will be slow enough you can intervene, but...there are always risks. Clinical trials can and do go badly wrong and people involved in them are taking a selfless risk for the betterment of the rest of us - and that's a very different proposition to declaring "this'll probably be fine" and not collecting the data.