The mortality rate is not the only consideration. People don't want to die, sure, but they also don't want to spend a couple of weeks in the hospital. And that risk, even for those under 65, is much, much higher: at least 1 in 100 (Source:https://www1.nyc.gov/site/doh/covid/covid-19-data.page).

Not only do individuals not want to spend time in the hospital, we as a society don't want them there either. Assuming we need 60% of the population to recover in order to achieve herd immunity, and assuming we somehow isolate all of our 65+ year olds from the virus, that would mean almost 2M hospitalizations in the US.