I can't comment on applications of category theory to computer programming or computer science, but I believe that the book Conceptual Mathematics by Lawvere and Schanuel is a nice introduction to the basic ideas of category theory.

http://www.cambridge.org/catalogue/catalogue.asp?isbn=978052...

I believe this book has been discussed "recently" on HN, but I couldn't find a thread.

Mathematicians write for multiple audiences, but there are two major mathematical audiences for their works: specialists in the same field and researchers in other fields. Phrases like "relatively little mathematical background" normally signal that a work is intended to be accessible to non-specialists, but it's often safe to assume that it is aimed at research mathematicians. I think if a work actually requires relatively little mathematical background, a mathematician is more likely to say something like "no, really, you don't need to know mathematics to understand this!" even when it's not quite true.