All the best and don't forget that it can get better!
They might just have said that not to sound alarming, and to close the subject.
You know yourself infinitely better than anyone else does, so if you have even the slightest doubt, you should never rely on others to decide what's best for you.
Be especially wary of people close to you, as proximity increases the belief of knowledge much more easily than actual knowledge.
Reminds me of a Philip Roth quote: "The fact remains that getting people right is not what living is all about anyway. It's getting them wrong that is living, getting them wrong and wrong and wrong and then, on careful reconsideration, getting them wrong again."
Kids aren’t kids forever. They get older and become more autonomous. You want them to grow up and be successful in all things in life. Surprisingly, beyond some life skills, if kids have people in their life that care about them then they have a greater chance of loving themselves, which helps them face life’s challenges.
Now you are just making him feel worse, since his problem is that he can't give his kids love and compassion.
Best of luck, friend. :)