Intuitively, I'd expect a substantive difference between a "we've just got to get through this and everything is fine" power outage - complete with cookouts! Can't let what's in the freezer go to waste - and the calendar-melting, interpersonally isolated anxiety hell and economic collapse, then un-collapse, then maybe collapse again? of COVID. Power outages also have much precedent in the living memory of the US, crucially including that things really will go back to normal again - while pandemics lack any such cultural familiarity. Just that uncertainty all by itself I'd expect to depress birth rate somewhat, nine months hence. (Not to mention, how do you even social distance in bed with somebody? If you're married or live-in partners, then sure, but I have to figure booty calls have plummeted lately, and that's also not a pattern you'd expect to see in a blackout.)
All in all - yeah, people have time on their hands in both cases, but they're not otherwise very comparable, and I feel like, in the much higher-anxiety environment of the latter, it's considerably less likely that people will opt to pass some of that time by boning down - and on top of that, safety measures foreclose a considerable range of opportunities for doing so in the first place. That said, we'll know pretty well one way or the other by probably January or February, so for anyone not trying to front-run the diaper and formula markets, the uncertainty of the moment is probably fairly bearable.
Fwiw, some Brookings economists found not wholly dissimilar reason to predict a very measurable bust, along with an interesting aside on rates of conception during and around the various waves of the 1918 flu. I don't know whether I entirely buy their analysis, but it's at least not absurd on its face; make of it what you like. https://www.brookings.edu/research/half-a-million-fewer-chil...