That's a good point, but the same occurs with programming. We generally can't catch every corner case. Of course, this is much more devastating in a mathematical proof, i.e. the proof is invalid. However, it is still very possible to develop good mathematical skills without a degree.

Also, this is somewhat unorthodox, but I've read that Godel proposed a new method of doing mathematics based on the experimental method, due to the implications of his incompleteness theorem. If that were the case, then math becomes more like computer science, or perhaps computer science is this new way of doing math.