Lots of work has been done to do just that by monero’s proof of work algorithm. They achieved ASIC and GPU resistance by making you generate entire programs (with quite a bit of RAM usage) and finding one that produces a desired output. Then you submit the seed used for the program (a series of programs actually but let’s not get into the details)
Anubis wants "slow for bots but fast for humans" which is not possible if somebody has a farm of devices that costs them basically zero.
Opportunity costs are real.
Combine with heuristics like in the article, so you can crank up the difficulty only for likely bot traffic, and you've got a bot supported site.
[1] https://krebsonsecurity.com/2018/03/who-and-what-is-coinhive...
That's not true. The goal is to raise the cost for bulk scrapers to the extent possible while minimizing any inconvenience to regular users. It's not about being slow for bots but rather reducing the asymmetry of the problem (from the perspective of the server) by imposing a (minor) cost on anyone requesting a page.
I know having the Anubis javascript at all gets in the way of some bots. But is the calculation helping enough?
It's complicated. I think anubis currently only collects low hanging fruit so to speak but in practice that's the bulk of what was causing problems so it's good enough.
PoW in general absolutely yes - I think monero clearly demonstrates this. Last I checked it remained viable to mine monero on at least somewhat low end and at least somewhat outdated consumer CPUs. It seems to me that fact implies that a well constructed PoW scheme can in practice squash the impact of any performance differences to within at most a couple orders of magnitude.
And even if you can guarantee the same CPU time on bots and legitimate requests, you still have a bunch of asymmetry in favor of the bots.
But it's also true that charging anything is going to be expected to reduce server load, that the cheaper something is the more carelessly people will tend to make use of it, and that at least in general bot operators are expected to be more cost sensitive than regular visitors on account of scale.
To that last point consider that a regular visitor will generally have an idle CPU and multiple gigabytes of free RAM. Meanwhile a competent scraper will generally be optimizing request parallelism per cheap VPS while an incompetent one will just point and shoot (and proceed to blindly visit every cgit url). The latter is by far the most troublesome and conveniently also the most likely to be impacted by even a poorly implemented PoW scheme.
I keep seeing comments along the lines of anubis only working because it bounces bots running lightweight setups without js but stop and ask yourself why are they running such lightweight setups to begin with?
It's easier and still gives access to almost everything. I don't think CPU limitations are the main issue there. As far as computer resources, I'd bet RAM is the bottleneck to running the script, not cycles. That suggests a RAM-heavy hash might do better than SHA256, but still agrees with the idea that the PoW in Anubis is not the important part.
> To that last point consider that a regular visitor will generally have an idle CPU and multiple gigabytes of free RAM. Meanwhile a competent scraper will generally be optimizing request parallelism per cheap VPS while an incompetent one will just point and shoot (and proceed to blindly visit every cgit url). The latter is by far the most troublesome and conveniently also the most likely to be impacted by even a poorly implemented PoW scheme.
I considered that. But while a visitor is idle they need the result quickly and they face the full brunt of the challenge all at once. A scraper can be working on a thousand sites at once so the per-request impact of the challenges is a lot lower, and patiently working through challenges in the background usually reduces the real cost, potentially to almost zero.
And you'd better not need multiple gigabytes of RAM to visit your site, that'll freeze my computer.
I might try to tackle it again when I can give it some more attention.