I read this paper (https://arxiv.org/pdf/2410.13722) on pre-training poisoning of LLMs. The authors demonstrated that poisoning 0.1% of the pretraining dataset can measurably manipulate beliefs of the model.

What I read about is a more sophisticated supply chain attack. But these days, search previews use retrieval-augmented generation to parse websites to generate answers on the fly. In that case you don't need to poison the supply chain; good SEO is enough.

- Companies Are Using Reddit to Manipulate ChatGPT and Google AI Search - https://www.404media.co/companies-are-using-reddit-to-manipu...

- AI Search Has a Misinformation Problem — and DuckDuckGo Just Proved It - https://www.informertech.com/post/duckduckgo-reddit-misinfor...