If you want a book to help get you started with TAOCP I've heard Concrete Mathematics is good. It's the math precursor to TAOCP (so if that's what's holding you back it could be helpful.) I haven't finished concrete mathematics yet though either :P so I can't tell you if it actually helps reading TAOCP. I would recommend knowing calculus, linear algebra, and discrete math before concrete mathematics. MIT OpenCourseware should have all those courses with Mathematics for Computer Scientists working as the Discrete Mathematics supplement.

I've never taken linear algebra so that is what is holding me back right now from continuing concrete mathematics. I bought Gilbert Strangs Linear Algebra book and it seems good so far, but I am busy with schoolwork now so I'll have to get back to it after I graduate in a spring.