The average man in the 17th century did not, when bored on Sunday, attempt to molest multiple random women in church. Even for his time Pepys was, well, a bit much, and he knew it, and felt guilty about it occasionally in the diary.

It's an absolutely fascinating read, and it's well worth reading, but you should not fall into the trap of thinking "well, that's just how it was back then"; Pepys was abnormally badly behaved on a number of axes even for the time.