I've spent a lot of time thinking about that particular question, and the simple answer is that yes, my partner and I may find ourselves late in life wishing we had done otherwise. We also very easily could find ourselves wishing we hadn't. I look around and see many people who also have very poor relationships with their elderly parents - I don't discount the fact that given the culture of America at this particular moment that I could have children who bring me great joy for the 18 years they are with me, who only then become all but distant strangers once they enter into their adult life. This seems increasingly common nowadays. Or they could love me sincerely!
Either way, if I had children, I wouldn't think it right or just to burden them with an expectation that they owe me endless years of emotional, physical, or social support.
Additionally, I think our brains are very good at rationalizing and justifying the choices we already have made, and so I'd mostly suspect that I'll continue to convince myself that I made the right choice regardless of any loneliness that may crop up later in life.
It also might seem callous, but there are simple psychological needs (such as feeling that you're caring or providing for another being) that are easily fulfilled without having children. Pets help a lot. Volunteering helps as well. And, of course, just like your older relative, spending time around other children and young people when you can.
My hope is that between all of these things, I'll lead a happy and fulfilling life. Or maybe I won't – but I ultimately don't have any control over that. And my personal opinion is that having children doesn't assure you a fulfilling life either!