I ended up joining a course but dropped out after less than a month; the first sign that it wasn't going to work out was at the initial interview where someone said "You've done some web work I see? I bet you'll be able to teach me a thing or two!" that person was the head lecturer.
Algorithms, advanced maths, and cryptography are also the areas I feel I lack.
All in all, I don't think I've suffered in the slightest either. Finding the first job was probably a bit tricker, but in hindsight I've had an additional 3/4 years earning good wages than those people who did do the degrees. I'm not strongly opinionated as to whether they're worthwhile or not, but I don't believe I've been negatively impacted by it.
I have thought a few times that the traditional style of education simply isn't applicable to some people; I was never very good at school, but flourished once I learnt how to teach myself what I wanted to learn.