The solution is to kick out the tenants/landlords who are concerned. Then they won't have to live near a medical worker, and everyone is happy.

Those medical workers have as much right to their place of shelter as anyone else.

This is a despicable form of NIMBYism. The people complaining expect medical professionals to save them, but don't want to treat them as equals.