There is more than one kind of memory. You're talking about episodic memory. I think you have many memories from childhood that you aren't considering. I'm sure you still have many semantic memories (facts, concepts, etc) from before age 10. For example: if you're from the US, you still know the name of the first president of the United States.
You probably have even more information from childhood stored in implicit memory. If you're like most people, you learned how to ride a bike, tie your shoes, understand language, and play catch. The memories of learning these things are separate from the memories needed to actually do them.
Erasing all of a person's memories would truly destroy them. Once they formed new memories, they would be as different from their original self as an identical twin separated at birth.