Calling COVID-19 "just the flu" is certainly wrong, but it's still fair to draw comparisons between the two. The latest projected death estimate for the US is 60-80k, down from an original estimate of 100-240k. We're about a week past the peak death rate and the total death count is approaching 60k, so the revised estimate was likely too optimistic probably closer to 100k is more realistic. By comparison 80k people died of the flu last winter in the US (last winter was higher than the average flu season, average is apparently closer to 40-60k). 600k die annually of cancer, on average, as another point of comparison.
Even if the current estimate is undercounting by half, COVID-19 will still have killed 2x as many people as last flu season. It's inherently speculative to compare against the death toll of what would have happened without a total economic shutdown. Figures ranging from 1-2 million have been tossed around here on HN. NPR ran a story with a claim of 1.3 million. However the experiences of countries that did not adopt distancing restrictions or adopted much less drastic ones, including denser and more populated ones like Japan, are not seeing anything close to these death rates. We'll be able to glean better information once testing becomes more widespread and comparisons can be made more reliably, but the 10-20x increase in deaths without social distancing that was initially claimed doesn't seem to be happening.
It's not "just the flu", its "just 3-4x deadlier than the flu, and spreads twice as fast". How scared are people of the flu? How scared are people of COVID-19? The latter is way more than 3 to 4 times larger than the former. If you asked people in 2019 "If we sacrifice 25% of the economy and make 25 million people unemployed, then we can stop a virus that will probably kill ~4x as many people as the last flu season[1]. Should we do it?" How many people would say yes? Probably not very many.
Phrase the same question differently: "If we sacrifice 25% of the economy and make 25 million people unemployed, then we can halve cancer deaths for this year. Should we do it?" Again, people would balk at this.
This is why the coronavirus panic is dumb in a lot of people's views. People are experiencing fear that is orders of magnitude off from the actual threat. Much in the same vein that people fear shark attacks when they are incredibly rare.
1. This is assuming a death toll of a quarter million, all of which are prevented in this scenario by economic shutdown. In reality, we'd only be saving the delta between the deaths with the shutdown and without.