I'm a fairly recent PhD, but on the whole I had a very positive time. It took some time to figure out what I wanted my research to be, and that was the most stressful part (first two years/in conjunction with a Masters). After that, it became pretty fun, with a very flexible schedule, very talented, driven and supportive people around me, and the ability to work on what I want with a lot of academic freedom.
One thing I emphasize to all the new graduate students/people thinking of grad school is that, of course, pursue research that is interesting, but also to make sure to pick a good adviser, one who's PhD students can vouch for in private. That was key for me and many of the people I know to having a good versus bad PhD.