Personally, Godel Escher Bach was pretty formative on my path to becoming a programmer; and The New Jim Crow is a monument to overkill, there are so many statistics that it's impossible to read the entire thing and not come to believe her thesis.
By that point I was already playing with numbers and looking for patterns in the Fibonacci triangle and various series, so being able to play with just symbols was natural and fun, especially if the system is constructed in just the right way so as to have meaning that transcends itself.
Later the book meanders through lots of other wonderful topics, but the theme seems to be simple systems which add up to something more than the sum of their parts. There's something about realizing how much depth lies in a handful of simple rules that really clicks and makes you value those rules in some way. After he's primed you with all kinda of these things, he introduces lisp.
It took another step or two after GEB for the beauty of lisp to really hit me, but GEB laid the groundwork by fetishizing recursion and simple foundations, among other things.
One explains why our foreign policy fails and the other explains who we believe ourselves to be and how that shapes who we are. McPhee's "Encounter with the Archdruid"[2] and Reisner's "Cadillac Desert"[3] have similar elements...
[1] http://www.amazon.com/Cities-Hill-Brilliant-Exploration-Comm...
[2] http://www.amazon.com/s?ie=UTF8&field-keywords=encounter%20w...
[3] http://www.amazon.com/s?ie=UTF8&field-keywords=encounter%20w...