The main ingredient we need here is to realize that numbers can represent sentences in formal systems. This should be obvious to computer scientists of today, but it was ground-breaking at the time.
Another thing we'll need is diagonalization. Some folks get hung up on this one; ultimately, you'll have to convince yourself that it's a valid technique.
Hofstadter's explanations, both the long gentle one in GEB and the quick fuzzy one in "I Am A Strange Loop", are much more detailed and correct on this, so I'd advise going back and making sure you understand each step.
Also, something that doesn't get enough airtime is Tarski's Undefinability, which says that a formal system afflicted by incompleteness is not capable of talking about truth in certain basic and important ways which would enable a formal system to fully represent itself.
At least, that’s my understanding of it.
For any sufficiently powerful system (Turing completeness is definitely powerful enough) there are statements which are true but cannot be proven by that system. Secondarily, no system can demonstrate its own consistency.
However, if you limit your software enough, you can prove that every possible input is handled correctly. That involves removing Turing completeness. E.g. once you accept a language as a configurator, you can no longer prove that. CSS3 is Turing complete. JSON is not.
One requirement is limiting the size of inputs.
Cynics might take this to mean that maths is meaningless. On the contrary, I'd like to suggest that maths is the canonical emergent pattern; any formal system with sufficient complexity to represent itself will automatically exhibit some algebraic structure, and maths emerges naturally from there.
The biggest implication for computer scientists is that, combined with results of Church and Tarski, we should expect that meta-languages are more powerful than object-languages, whether this is by preprocessing and macros like in C/C++ and CL, or by avoiding using meta-features except when necessary like in Python and Ruby.