I think you're actually proving Kuhn's point when he argues that the progress of science involves wiping out almost all memory of obsolete paradigms. When we read a science textbook, all of the information is placed within the structure of the current paradigm, as if none other came before it.
To give an example, how much do you know about theories of geology before 1960? Any educated person today knows the basic idea of plate tectonics, but only a historian of science might know what came before.
According to Kuhn, the impression you get that "puzzle pieces are rarely removed" is something that happens after science has been rewritten to fit a new paradigm.