I don't take notes. (It's worth pondering whether taking notes is really an aid to internal remembering...) In any case, what I was trying to point out was that it could be the case that we actually remember almost everything, but have real difficulties in recalling our memories without cues. (It's also worth pondering just what it is that "autistic savants" might actually be doing ...)