It seems the main point of difference is that you found the meandering digressions annoying whereas I really enjoyed them. For me, understanding the topics (computation, recursion, use/mention, self-reference) wasn't the main reason for reading the book. The blurring of form and function in the writing and general playfulness were just as important and anything but boring.
PG, Feynman, and _why are useful points of reference. I like all three but recognise how different the latter one is to the former two. A similar distinction would be between Coetzee (precision) vs. Wallace (verbosity). I strenuously suggest you avoid Wallace even though I think both authors are amazing. :) Having enjoyed Wallace's non-fiction book on infinity, "Everything and More", I recommended it to a work mate (also a programmer) and he hated it. Upon reflection, I think he has similar tastes to you and would probably not like GEB either.
Out of interest, what style of fiction (if any) do you enjoy?