[1] Yeah, that can be a bit of a pain to setup depending on the server settings, but some people have to pay for bandwidth and server resources, so it's probably worthwhile.
It's a standard response code, and with a bit of luck google will scale the requests accordingly. If not, this will still allow the proxy to operate properly without burning too much server resources.
(I understand that this may leave some customers unhappy since the proxy may be up to 1 hour stale. If that's the case, the server could also add a check to see if there have been pushes since last request, though this is quite more complex)
Google already says in the comments [1] that it would be "a fair bit of work" for them to read the standard robots.txt for "boring technical reasons". I would not necessarily rely on them to respect HTTP 429 either.
In fact, given the facts at hand with the current situation, I would guess that they probably wouldn't handle it correctly, and would cite more "boring technical reasons" when asked.
----------------------------------------
[1] https://github.com/golang/go/issues/44577#issuecomment-11378...
that's embarrassing. What other reasons should there ever be?
Stop taking their money, stop using their shit. Turn your attention to where it's beneficial.
It also would be likely to result in an increased number of requests?
Drew's actions so far (as far as I can tell) are not only inoffensive/reasonable but overtly good-willed and presumably result from a considerable amount of effort.
(These kinds services are also supposed to rate-limit their requests.)
If that's the case, then great. It solves the DDoS issue without impacting the Google side, and everyone can be happy.
Most likely these errors would manifest down to the users of Go and those who've a dependency which lives on sourcehut
> I can’t blackhole their IP addresses, because that would make all Go modules hosted on git.sr.ht stop working for default Go configurations (i.e. without GOPROXY=direct)