Hacker News Reader

Make some of your content, which is invisible to normal users, so exceptionally toxic to the bots that they will begin to avoid your site by choice.

View on HN

How can doing this cannot affect your SEO?

Serve different content to non search engine bots

I often read that doing that was a big no-no as Google would flag your website as cheating. But that may be a SEO legend.

If SEO's a concern, then there's nothing you can do about the LLM bit. Either put the website behind a login, or deal with robots sucking everything up.

Pre-LLMs, there were already many websites showing scraped versions of other people's sites, and ranking higher in search results.

Reply on news.ycombinator.com