Speaking from my experience so far as an "adult learner": yes, it's brutal. At least, tournament chess is. Online play is easy for me not to take so seriously. But when I get up on Saturday morning to go to a tournament and spend all day there, and I am writing down every move, and I'm face-to-face with my opponent, the whole thing is a lot emotionally weightier. I've found that I dislike losing and that I experience empathy for my opponents when they lose to me. My favorite games that I've played have been draws.
I spoke a bit with an older and much more experienced player I know about this. He said the only thing that worked for him was to completely stop caring about winning and losing, and to let go of seriousness about the game. His idea was that chess is supposed to be fun; fun is the whole point. Improving is nice, but why bother if the process is not enjoyable in itself? Been thinking a lot about this since then.