In my experience as father of two children (now teenagers), the most challenging part didn't have to do with raising them. It was the slow and unavoidable process of disappearing from the lives of our friends. My partner and I were the first to have kids in our group of friends, even though we had them in our early thirties. Our friends waited until their early forties to have children.
It didn't matter that we made an extra effort to be available for plans (not plans with kids, adult plans). Our friends would organise something but wouldn't tell us ("We didn't ask you because you must be so busy/tired").
So it wasn't my/our experience of parenting that I regret but the experience of frienship and the assumptions that many people seem to have about how being a parent changes your relationship with the rest of the world.