Fully agree that programming for me brings along "a kind of brain rot". Hard to describe, but working deep in code just shifts the mind to a state that's ... less open, more focused, more obsessive ... and seeps into daily life.
Vernor Vinge's concept of "focus" in "A deepness in the sky" really clicked with me and put into words what I'd always had at the back of my mind. A highly recommended read.