What are you talking about? It gets under -20C routinely each winter for a couple nights every year through the whole of the northeast; upstate New York, Vermont, Maine, New Hampshire, Ontario, Quebec.
Toronto is the 3rd/4th most populous city in North America (depending on how you count it). Almost every winter there's a -25C overnight. Not for days on end like where I grew up in Alberta, and not every year, but it definitely happens. There's 7 million people in the Greater Toronto / Hamilton corridor alone. 2 million people in the Montreal area. Almost a million in Quebec City, 1.4 millionish people in the Ottawa/Gatineau area. Not to mention Detroit, Windsor, Buffalo, Albany, Syracuse, Burlington, etc. etc. etc. Oh, and I'm seeing a mean minimum of -19C in Chicago for January, too, so throw in a few more million people there because that's close enough.
"Not a lot indeed." Only maybe a couple dozen million people.
EDIT: Oh yeah, I hear the US has a "midwest", too. I hear there's people there, too.