Yes, there are of course medical cases and disorders. But I'd guess a lot of variance just comes from differing individual "training intensity".
There must be quite some variance in the average number of times that individual people are triggered/challenged to recall some memories (and process them) in their everyday lives. Some people don't communicate with others - generally, and also more specifically about what happens/happened in their life - as much as others do.
And some people are more inclined to subconsciously cultivate their own memories, than others. Those for example, who enjoy storytelling. Their mind probably creates stories (which apparently are an essential mental feature) around their memories more easily, which facilitates recalling these memories.