I could program before studying at uni, but thanks to our two-term course Algortmics I+II at FAU Erlangen (part I based on SICP), it seems a fog that I wasn't even aware of cleared.
Volume 1 really works well if you also work through Concrete Mathematics (my professor and I started on Volume 1, diverted to CM, then back to Volume 1). CM provides enough additional math to get through chapter 1, where I was having a bit of trouble on some of the math heavy portions.
I actually started working through TAOCP one summer during a very boring sys admin job. I couldn't actually change the system to improve it, just complete whatever tasks were given to me and help customers. Maybe 1-3 hours a day of actual work effort. And in the pre-smart phone days, a book was easy to work through without distractions until the next ring of a phone call or ding of an email.