Wrong, anubis stops mass crawling of web pages, by requiring a proof of work, which makes accessing these websites more expensive.
Anubis was not built to stop individual users with llms.
Wrong, anubis stops mass crawling of web pages, by requiring a proof of work, which makes accessing these websites more expensive.
Anubis was not built to stop individual users with llms.
If it becomes widespread (as it has been doing), they will. Anubis' strategy only works while it remains a niche approach only adopted by a small number of sites.
I think this is a pragmatist vs idealist debate. The pragmatic answer is that Anubis solves a problem now, and so it will be used until either it doesn't or a better solution presents itself. The idealist approach is that it's obvious that there's ways to get around Anubis, and so some people argue from there that it shouldn't be used. But the only other alternatives being offered are to either eat up the costs (in server resources or engineering time), or to go behind cloudflare with its own tradeoffs.
Every individual practitioner has a strong incentive to overuse it, because it's extremely effective for them individually. Every additional practitioner that uses it increases the selection pressure on their collective adversary to develop resistance. Eventually, it reaches a tipping point and becomes ineffective. But the ecosystem-level impact of its historical use remains, and makes things worse for everyone.
Recently, the baseline expectations of the Web have shifted, and I need to enable JavaScript to read a non-CloudFlare-proxied simple HTML site like LKML. I do not expect this shift will revert once Anubis outlives its effectiveness.