Your reply is mostly a red herring because as you yourself said, the OP was talking about regular training data, not the ICL you focused on.

Just because you can find one way that GPT might be slow doesn't invalidate the point that its training does use massive amounts of data.