I read (probably on HN) where someone suggested that pre-LLM data will become the 'low background steel' (pre-Nuclear-age steel with lower levels of internal radiation) of machine learning.
It seems like provenance/traceability of data sources will become important, both for the builders and users of LLMs.