I find having kids in my late 30s hits really hard because of this. Biologically, I can see it being best having kids in your early 20s when you're made of rubber and can bounce back after lack of sleep.
In college you can bounce back after an all nighter not just because you're younger but also because it's an opt-in single event and you can opt out of the next one when you recognize that you're sleep deprived. You can also do what my brother does and regularly sleep in till 10 to make up for the late nights.
Once you have kids there's no opt out mechanism, and sleeping in isn't an option any more. There's not really an opportunity to bounce back from the sleep loss, even for a younger adult.
It really gave me a fear of situations that might involve sleep deprivation - one reason among many I never thought I could handle medical school for instance.
Since I hit about 36, I've found that a poor night of sleep, such as from a red-eye flight, will wreck me for about 2 days.