This is me. I am 43, entering my fourth year of PhD. If you have money saved and want to challenge yourself in new ways a PhD is a worthwhile goal. A couple of thoughts:

1) Many of my 20 something peers are struggling financially, 20-30k a year is difficult to live on in CA. I made good money in my 20s and enjoyed life, having less now does not bother me. 2) I don't have children. My peers with children face a much different set of challenges than I do. I suppose it comes down to the support structure in your life, but I am not sure I could do this with school age kids. 3) I cannot stress enough the importance of a top tier school and a strong advisor.