Hacker News Reader

Comment by Joel_Mckay | Hacker News Reader

Rate-limit the GeoIP list for the affected areas to drop if more than 20% of active traffic. i.e. the service outages get co-located only with the problem users areas.

Also, when doing auto-updates: always add a chaotic delay offset 1 to 180 minutes to distribute the traffic loads. Even in an office with 16 hosts or more this is recommended practice to prevent cheap routers hitting limits. Another interesting trend, is magnet/torrent being used for cryptographic-signed commercial package file distribution.

Free API keys are sometimes a necessary evil... as sometimes service abuse is not accidental.

View on HN

That would only work if they had an API; AFAICT, they're just hosting a file.

At this point, they might be better off coordinating with the other major adblocker providers and just outright move the file elsewhere. Breaking other people's garbage code is better than breaking yourself trying to fix it. Especially on a budget of $0.00.

If the defective code for the browsers are in public repos, it might also be more effective for someone to just fork the code, fix the issue (i.e. only download this file once a month, instead of every startup), and at least give the maintainers a chance to merge the fix back in.

It is very common to see API keys in urls for access to what are essentially flat files. Thus, fairly trivial to change from:

https://127.0.0.1/file.csv

https://127.0.0.1/file.csv?apikey=abc123

This could allow client specific quotas, and easy adoption with maintained projects in minutes. Thus, defective and out-of-maintenance projects would need manually updated or get a 404.

API keys are most successful when they're issued for server-side use; when used client-side the usual pattern that I see is for individual clients to request their own API key?

In this case, it would need to be distributed to myriad users who legitimately need to ask for the lists and then could be scraped by the "attacker", but at least then they'd have to be knowingly malicious vs. accidentally malicious.

You generally add a small "cost" to request an API key. For example submit your email to this form and wait a day.

Then browser makes like this will not reasonable be able to request a new key automatically for every install. So they will just request one and ship it.

Then when you get abuse like this you can disable it.

Ahhh, good point!

Moving the file elsewhere won't fix it. They are serving terabytes of traffic on Access Denied, it won't go away if that changes to "Not Found" instead, the developers seem already entirely ready to ignore their adblocker just not working.

The query limit from the lets encrypt SSL service operates in a similar manner. If you hit it more than a few times a week for the same domain/token/key, than you are banned for 5 days.

In general, it is easier to setup filters after differentiating legitimate from nuisance traffic. For example, fail2ban looks at the log of errors from invalid hits, and bans the IP or entire ISP block ranges for 5 days. This ban than get propagated to the rest of the web via spamhaus.org listing.

i.e. the users start to see the worlds internet become unreachable, as admins start to block traffic at the NOC's routers, and so on... India knows about Karma.

I am more surprised the app store for the apk isn't getting sued for theft of service.

Good luck finding the legal contact, not to mention suing, some random developer in India who apparently already abandoned the project.

I will also point out that if you fail2ban the IPs requesting the spamblock list, it could become worse if the browser just retries endlessly in the background. The traffic for a 404 page could be much smaller than the traffic of the very same devices trying again every few seconds, constantly, instead of only checking that 404 every app restart.

In general, the client socket needs to timeout on black-holed connection attempts (several minutes), and the server never sees a TCP handshake packet if the IP is on the global routers ban lists.

As a side note, some people build spider traps that reply with a pre-baked bzip file as a spoofed HTML compressed response. Thus a client program dutifully decompresses a few TB sized document, and browser exits due to memory issues. Note most modern Browsers are wise to this trick these days, but I doubt a dodgy plugin disk-usage limit check would catch a client side storage-flood. People shouldn't do this though, even if it is epically funny and harmless. =)

The client can just set a custom timeout and close the socket after 5 seconds of no or low activity. Then try again.

On your side note; EasyList probably does not want people to start suing for starting to distribute malicious content to users (and crashing your browser on purpose is arguably malicious).

There are all sorts of games people could play, as a DoS is technically an act of war under some legal systems. For example:

1. take the top 200 most popular websites in the given nuisance area

2. add ban rules to a version-B list that also includes all social media, search engines, and Wikipedia.

3. Look at the user-agent string for that specific problem client, or extreme apikey quota abuse

4. Randomly serve version-B filter list that breaks the browsing experience after a frequent update. Increase random breakage until traffic rolls off to normal levels.

The TOS for the ban list file does not specify which sites it will ban, and most users will just assume it is the App that is broken (it is already). People should not do this either, even if it is also funny and relatively harmless. Also, suing people while participating in an attempted crime probably would not go well. =)

Even then I would not do this without clearing it with a lawyer first. You could still end up at the wrong end of a lawsuit that you'll have to defend in India.

There are numerous legal/accounting specialists that protect businesses and investment decisions. We already won't serve _any_ content to IN networks as business policy... so are unlikely to ever have to visit with the cobras.

Have a gloriously wonderful day =)

Not serving means there is already no content. That is different to serving 100TB of 404 content from clients that will never stop.

Reply on news.ycombinator.com