Hacker News Reader

Your model/harness will indiscriminately do web searches to get answers. I believe that’s where the real risk is.

I don't know for sure, but will it require something like with high "SEO" rankings to fake the reputation of the sources? So if the LLM search for a specific spreaded knowledge splitted across many bad sites, it can poison the model maybe.

You mean, how difficult will it be for an attacker to get their text seen by the LLM that does arbitrary searches?

It could be as simple as a malicious prospectus for AcmeCo, and then try to get AcmeCo on the radar so that the LLM-tools find your document and incorporate it. The malicious bits don't even need to be AcmeCo-related, they could be to pump (or dump) practically anything.

Reply on news.ycombinator.com