I suspect most people who read unabridged Pepys for any length of time end up feeling very ambivalent about him as a person. His diary contains the some of the deepest introspection of a human life--both the good and the bad--in the history of literature. Rousseau was a cheap marketer by comparison.
That depth of revelation is one of the things that make Pepys hard to put down, even as you find his personality increasingly disagreeable. It is a truly remarkable work.
Edit: added unabridged for clarity