It's not that I don't believe you. I can rationally accept that what you've said makes sense. However, being on the outside, it takes some mental effort to make the fame you've described not seem extremely nice.
A long time ago, I remember the difference being illustrated at a Dragoncon in the early 2000s. Guests on the fame level of, say, Babylon 5 cast members, could walk around the convention just fine. They could do standard things like, say, go to the restaurant and have a meal without being heavily interrupted. Walking around they'd maybe get some hellos and photo poses and whatnot but it didn't seem terribly intrusive.
And then there was Alice Cooper, who walked around the main convention floor one day and instantly got mobbed by a huge crowd.
My hunch is that Taylor Swift probably is at the fame level where she needs security of some sort merely to walk around in public. That's a negative in my mind.
There’s no magical set of circumstances that result in happiness or joy. In fact I think the external things that we most often identify as bringing happiness or joy, apparently when most people are faced with these things, it results in quite the opposite.
In my opinion Taylor Swift has one of the hardest and most unrelenting jobs on the planet. Going out night after night trying to appeal to the teeming masses sounds like absolute hell. Everyone you meet more likely to be a vampire than not. No thanks!
Maybe I'm just old :-)