At peak times in our house, say, heat on, lights on, TV on, cooking, doing laundry, we pull perhaps 3kW. Our house is pretty efficient; gas appliances, LED lights, no air conditioning, etc.
The car charges after 9pm, but that's just because our rate drops then. If midnight was even cheaper, it would charge then. We don't charge every day, but if we did, it would take perhaps 1 hour. And this is for a car we drive 18,000 miles on in its first year.
3-4x the demand of a 'normal' household is not nothing, but it's not insane, either. Considering EVs don't need to charge every day, or every hour of every night, it's not that tricky to see the aggregate load as fairly similar to peak afternoon/evening hours in your average suburb as people return home from work.
To be sure, if everyone started charging their EVs at 6pm, we might have a problem. Likewise, if everyone had an EV and they all clicked on at midnight, we might have a problem. But the load on the grid is not that huge from an instantaneous perspective, and can be managed by variable pricing and/or some kind of grid scheduling tech.