That really depends on which part of the city you’re in, my neighborhood is a sea of strollers!
Do you also live in Noe Valley? I'm surrounded by children and strollers on the J most days.
I guess, but I'm just saying walking downtown you just don't see families, kids anywhere...whereas in most other cities you see kids, families, retirees, students, workers, etc in the "public centers" of a city.
My experience is only anecdotal, but I think most city centers I've spent much time in (Austin, Houston, Tokyo) haven't had many families around during the work week either. If you go to fisherman's wharf, you'll find it crawling with families (undeniably mostly tourists), and most weekends you'll find Union Square swarmed as well.
The real difference I've noticed with kids in this city is, compared to other places I've visited recently, the parents elsewhere acknowledge and/or comfort their kids if they are crying or signaling for attention. Here in SF it seems commonplace for parents to ignore their kids entirely and pretend they don't exist when they are screaming/crying on MUNI, running around at restaurants, or asking "hey mom, hey dad, hey mom, dad, hello?". Parents here either tune out or just don't care.