> Their goal/claim is that the language model will not retain copyrightable sequences.

In case people are not aware, we know for a fact this is demonstrably false. It may not even be possible to achieve this with existing models. People have replicated entire sections of GPL code (with comments) using CoPilot. This is true of other models, like image generation models which can return clearly recognisable images which were present in their training sets.