I'm a self-taught programmer with around 15 years of experience. (I studied physics)
The one thing I really miss from not studying computer science is a network of programmer friends. It's a lot harder to meet new people after university. I'm currently trying to hire my first employee, and the advise I usually get is "hire your friends from university" -- but that doesn't work if non of them are programmers.
You can meet new people at meetups etc, but it takes a lot of effort.