Yep, I'm exactly like you. I also have a CS degree, but after many years of doing web dev. I feel stupid. I have recently started buying books that teaches algorithmics in a more easy approach.

I am currently reading "Python Algorithms: Mastering Basic Algorithms in the Python Language". I can't recommend it enough. Besides clearly explaining the basics, it helps you learn how to think about transforming naive solutions in to efficient algoritms.

Besides that, I find it fun to practice on code tests provided by codility, interviewstreet etc.