I read Origin of Species before The Selfish Gene and I enjoyed it a lot. I actually knew enough about evolution to follow all arguments easily. This is not surprising given that Darwin tried to introduce the idea to an audience that was largely alien to the concept.

I think what I liked most about the book is the (clearly labeled) speculations. He was already anticipating a lot.