could it be that memory must require language as a predicate? Without language, how does one "record" the memory?
Just last week, my two-year-old spied the freezer pops in storage. She pointed out back and said, "Eat on deck!" Clearly, she remembered eating freezer pops on the deck, but the last time she did that was last summer (northern hemisphere) when she didn't know how to talk at all, let alone say "deck".
The same way one rides a bicycle without telling themselves to balance and operate the brakes.