It's a been a point of frustration for me and the people I'm closest with for several years now. It doesn't seem to affect work nearly as much.
I've had a feeling that it's because I don't let myself just think. Your comment makes me even more sure of this. I've developed habits since I was a teenager to keep my brain constantly busy, always either consuming or producing something. But never integrating or just thinking. It seems I need to replace some habits.