Most kids grow up fine. Some will be late. My first child didn't start talking until she was about 3. she even stopped a babbling phase around 1. But after 3, she just never stops talking. She couldn't hold a pencil until around 5 or 6, way behind other kids, and now at 9, she draws better than anyone in the family.
This is quite common. You have kids who talk and walk late, but they might be strong in other forms. I see it sort of like assigning skill points, they just grow in different direction, but you're not the one who gets to pick where.
Probably the worst thing you can do is send them to a bad nanny. We've had several, whose roles I describe as "keeping them alive" but these will often be little different than putting them in cages like a chicken. Those development points are wasted only because they're forced to not develop. I think school can have a similar effect. A lot of teachers and classmates can be envious and vicious, so kids try not to stick out.
Our first child was behind the other kids, but second and third were far ahead. I'd say there's a strong role model effect too. The kids imitate older siblings. Older sis draws, and the youngest will pick up a pencil at 1 year old. So don't worry too much if your first child is statistically behind.
Also from experience, my kids can stress themselves plenty. They will get pissed that they can't walk or put on their pants or talk. As a parent, just give them hugs and reassure them. Adding to the stress has the same negative effects as scrum.