Poisoning Well

I'm running Iocaine[1] which is essentially the same thing on my tiny $3/mo VPS and it's handling crawlers bombarding the honeypot with ~12 requests per second just fine. It's using about 30 MB of RAM.

[1]: https://iocaine.madhouse-project.org/

Do we know if LLM scrapers are running JavaScript on the pages? If they are, maybe it's worth offloading the Markov model to the client side.

Crazy how close to coherent it reads, yet it clearly is gibberish.

This will remain so as long as the models don't need to ingest any new information. If most novel texts will appear with slightly more insidious nonsense mirrors, LLMs would either have to stay without this knowledge, or start respecting "nofollow".

It's competition. Poison increases in toxicity over time.

I could generate subtly wrong information on the internet LLMs would continue to swallow up.

I don’t know about that. Have you seen their output? They’re poisoning their own well with specious nonsense text.

Isn't the Iocaine developer from Australia?

Humans have been refining their gibberish for centuries.

I admire your idealism that "the real thing" is coherently different from good gibberish. The poisoned version of the same article is great: https://heydonworks.com/nonsense/poisoning-well/ (I love me some surrealism so gibberish is something I sometimes choose to input into my own model in my head).

The scary part of AI is that it shows how crappy most of the training material is.

This is the first I've heard of Wikipedia starting with a copy of Britannia. Where did you see that?

It is about imposing costs on poorly behaved scraping in an attempt to change the scrapers behavior, under the assumption that the scrapers' creators are anti-social but economically rational. One blog doesn't make a huge difference but if enough new blogs contain tarpits that cost the scraper as much as the benefit of 100 other non-tarpit blogs, maybe the calculus for doing any new scraping changes and the scrapers start behaving.

> they only got the head start by taking copy of Encyclopedia Britannica

Wikipedia used a version of Encyclopedia Britannica that was in the public domain.

Go thou and do likewise.

I do not accept the assertion that the content I host is anything other than my own to serve to whom I wish. Get your hands off my belongings, freeloader.

Or, shorter: I hold you in as much contempt as you hold me.

> If guys like this have their way, AI will remain stupid and limited

AI doesn’t have a right to my stuff just because it’ll suck without it.

Okay, what percent of content is poison like this? Way less than 1%? If AI is so smart maybe it can figure out what content in a training set is jibberish and what isn't.

Anyway, a big problem people have isn't "AI bad", it's "AI crawlers bad", because they eat up a huge chunk of your bandwidth by behaving badly for content you intend to serve to humans.

> I continue to have contempt for the "I'm not contributing to the enrichment of our newest and most powerful technology" gang.

Ok, when do I get paid for my contribution?

If you put stuff outside, in public, then it is correct for people to observe and learn from it. You should be happy that your web presence is enriching our culture. If not, you're selfish and I think you're bad.

If you don't want it to contribute to the public use, make a login page. I see no reason for you to get the advantage of a public access without contributing to the community.

I don't want having to pay extra money for vibe-coded LLMs companies bots to scrape my website constantly, ignoring cache headers and the likes.

Every single person who has wrote a book is happy if others read their book. They might be less enthusiastic about printing million copies and shipping them to random people with their own money.

I can think of several anti-LLM sentiments right now:

- developers upset with the threat of losing their jobs or making their jobs more dreadful

- craftspeople upset with the rise in slop

- teachers upset with the consequences of students using LLMs irresponsibly

- and most recently, webmasters upset that LLM services are crawling their servers irresponsibly

Maybe the LLMs don't seem so hostile if you don't fall into those categories? I understand some pro-LLM sentiments, like content creators trying to gain visibility, or developers finding productivity gains. But I think that for many people here, the cons outweigh the pros, and little acts of resistance like this "poisoning well" resonate with them. https://chronicles.mad-scientist.club/cgi-bin/guestbook.pl is another example.

> Does it really matter that meat bags find out about your message to the world through your own website or through an LLM?

Yes, it matters a lot.

You know of authors by name because you read their works under their name. This has allowed them to profit (not necessarily in direct monetary value) and publish more works. Chucking everything into a LLM takes the profit from individual authors and puts them into pockets of gigacorporations.

Not to mention the facts the current generation of LLMs will straight up hallucinate things, sometimes turning the message you're trying to send on its head.

Then there's the question of copyright. I can't pirate a movie, but Facebook can pirate whole libraries, create a LLM and sell it and it's OK? I'd have a lot less of an issue if this was done ethically.

> Don't you want people to read your content?

There's the problem right there. If all you produce is "content" your position makes sense.

I think it’s obvious.

In simpler terms, it comes down to the « you made this ?, I made this » meme.

Now if your ‘content’ is garbage that takes longer to publish than to write, I can get your point of view.

But for the authors who write articles that people actually want to read, because it’s interesting and well written, it’s like robbery.

Unlike humans, you can’t say that LLM create new things from what they read. LLM just sum up and repeat, evaluating with algorithms what word should be next.

Meanwhile humans… Oscar Wilde — 'I have spent most of the day putting in a comma and the rest of the day taking it out.'

I think at least partially, it's not an anti-LLM stance, it's an anti-"kill my website" stance. Many LLM crawlers behave very poorly, hurting websites in the process. One result of that is that website owners are forced to use things like Anubis, which has the side-effect of hurting everybody else, too.

I prefer this approach because it specifically targets problematic behavior without impacting clients who don't engage in it.

> Don't you want people to read your content?

People, yes. Well-behaved crawlers that follow established methods to prevent overload and obey established restrictions like robots.txt, yes. Bots that ignore all that and hammer my site dozens of times a second, no.

I don't see the odds of someone finding my site through an LLM being high enough to put up with all the bad behavior. In my own use of LLMs, they only occasionally include links, and even more rarely do I click on one. The chance that an LLM is going to use my content and include a link to it, and that the answer will leave something out so the person needs to click the link for more, seems very remote.

If it's ad-supported or I'm seeking donations, I only want people reading it on my website. Why would I want people to access it through an LLM?

A well drilled on my property, without even asking for permission, let alone receiving it? I'll happily take that side.

We're only at the beginning.

There are definitively scrapers that ignore your robots.txt file. Whether they're some "Enemy State" LLM outfit, an "Allied State" corporation outsourcing their dirty work a step or two, or just some data hoarder worried that the web as we know it is going away soon, everyone is saying they're a problem lately, I don't think everyone is lying.

But it's certainly also true that anyone feeding the scrapings to an LLM will filter it first. It's very naive of this author to think that his adlib-spun prose won't get detected and filtered out long before it's used for training. Even the pre-LLM internet had endless pages of this sort of thing, from aspiring SEO spammers. Yes, you're wasting a bit of the scraper's resources, but you can bet they're already calculating in that waste.

> The first isn't worth arguing against: it's the idea that LLM vendors ignore your robots.txt file even when they clearly state that they'll obey it:

That’s testable and you can find content “protected” by robots.txt regurgitated by LLM’s. In practice it doesn’t matter if that’s through companies lying or some 3rd party scraping your content and then getting scraped.

> Since LLM skeptics frequently characterize all LLM vendors as dishonest mustache-twirling cartoon villains there's little point trying to convince them that companies sometimes actually do what they say they are doing.

Even if the large LLM vendors respect it, there's enough venture capital going around that plenty of smaller vendors are attempting to train their own LLMs and they'll take every edge they can get, robots.txt be damned.

> The first isn't worth arguing against: it's the idea that LLM vendors ignore your robots.txt file even when they clearly state that they'll obey it: https://platform.openai.com/docs/bots

So, uh... where's all the extra traffic coming from?

Your link does not say they will obey it.

After I set up a self hosted git forge a little while ago, I found that within minutes it immediately got hammered by OpenAI, Anthropic, etc. They were extremely aggressive, grabbing every individual file from every individual commit, one at a time.

I hadn't backlinked the site anywhere and was just testing, so I hadn't thought to put up a robots.txt. They must have found me through my cert registration.

After I put up my robots.txt (with explicit UA blocks instead of wildcards, I heard some ignore them), I found after a day or so the scraping stopped completely. The only ones I get now are vulnerability scanners, or random spiders taking just the homepage.

I know my site is of no consequence, but for those claiming OpenAI et al ignore robots.txt I would really like to see some evidence. They are evil and disrespectful and I'm gutted they stole my code for profit, but I'm still sceptical of these claims.

Cloudflare have done lots of work here and have never mentioned crawlers ignoring robots.txt:

https://blog.cloudflare.com/control-content-use-for-ai-train...

I somewhat agree with your viewpoint on copyright, but what terrifies me is VCs like a16z or Sequoia simultaneously backing up large LLMs profiting from ignoring copyright and media firms where they'll use whatever power and lobby they have to protect copyright.

I don't think the content I produce is worth that much, I'm glad if it can serve anyone, but I find amusing the idea to poison the well

Looks like OpenAI publish the IPs of their training data crawler too: https://openai.com/gptbot.json

From how I read it, the author seems to be suggesting that this list of IPs be on an allowlist as they see Google as "less nefarious". As such, sure, allowing google IPs is as easy as allowing all IPs, but discerning who are "nefarious actors" is probably harder.

A more tounge-in-cheek point: all scripts take an LLM ~10 seconds to write, doesn't mean it's right though.

Ah, so the NSA defence then -- "it's not bulk collection because it only counts as collection when we look at it".

Your post-rationalization just doubles down on the stance that these crawlers are abusive and poorly developed.

You're also under the blatantly wrong misconception that people are worried about their data, when they are worried about the load of a poorly configured crawler.

The crawler will scrape the whole website on a regular interval anyway, so what is the point of this "optimization" that optimizes for highly infrequent events?

Not every intruder who enters your home is necessarily a burglar.

Tencent scrapers are hitting my little Forgejo site 4 times a second, 24/7. I pay for that bandwidth. Platitudes sound great, but this isn’t a lofty “drinking from the public well”. This is bastard operators taking a drink and pooping in it.

My thoughts will have more room for nuance when they stop abusing the hell out of my resources they’re “borrowing”.

So I get to freely copy Windows and Office and use them in products that I sell to others without Microsoft's consent now? Or is this only true when it benefits big corporations?

I'd be totally down with "content belongs to everyone."

The problem is when you steal my content, repackage it, and resell it. At that point, my content doesn't belong to everyone, or even to me, but to you.

* I'd have no problem with OpenAI, the non-profit developing open source AI models and governance models, scraping everyone's web pages and using it for the public good.

* I have every problem with OpenAI, the sketchy for-profit, stealing content from my web page so their LLMs can regenerate my content for proprietary products, cutting me out-of-the-loop.

> One of the many pressing issues is that people believe that ownership of content should be absolute, that hammer makers should be able to dictate what is made with hammers they sell.

You’re conflating and confusing two different concepts. “Content” is not a tool. Content is like a meal, it’s a finished product meant to be consumed; a tool, like a hammer, is used to create something else, the content which will then be consumed. You’re comparing a JPEG to Photoshop.

You can remix content, but to do that you use a tool and the result is related but different content.

> Content belongs to everyone.

Even if we conceded that point, that still wouldn’t excuse the way in which these companies are going about getting the content, hammering every page of every website with badly-behaved scrapers. They are bringing websites down and increasing costs for their maintainers, meaning other people have limited or no access to it. If “content belongs to everyone”, then they don’t have the right to prevent everyone else from accessing it.

I agree current copyright law is toxic and harmful to culture and society, but that doesn’t make what these companies are doing acceptable. The way to counter a bad system is not to shit on it from a different angle.