He develops a lot of ideas in "The Structure of Evolutionary Theory"... basically:
* Organisms are constrained by their evolutionary history... it's "easier" for them to develop in some directions than others. For example, the vertebrate body plans is extremely ancient and there are a lot of genes that do exactly the same job in mice and men, or even in more distantly related ancestors. Embryology and the field of Evolutionary development ("evodevo") has made a lot of this more concrete recently. Similarly, there is evidence that organisms draw on their evolutionary history as a "library" of possible responses to new stimuli.
* Punctuated equilibrium: the fossil record shows millions of years of little to no change, followed by sudden mass extinctions that set the stage for another equilibrium period.
* Sexual selection: if females are looking for some feature, males evolve to have that feature, even if it has no actual use or is harmful to the species. And vice versa. The peacock's tail is the classic example.
* More controversially, Gould argues in favor of hierarchical selection, the idea that natural selection can act on groups or species as well as individuals. This is something that Dawkins vigorously denies, preferring to view the individual gene as the sole unit of evolution.