Gubernator: Cloud-Native Distributed Rate Limiting for Microservices

Does this name have any significance other than what you said here and what they say in this article?

A leaky bucket?

They call this out as a use-case that's supported by Gubernator:

    Egress limiting – Blasting external SMTP servers with millions of messages is No Bueno.

I think this could be used for both incoming and outgoing requests. From what I gathered reading the docs and API, this doesn't actually forward your request to the API, it just keeps track of rate's and limits, leaving the actual limiting up to you. Its a microservice that keeps track of limits for you, and is highly distributed.

Basically, and I can be totally wrong, what it looks like is you send a rate limit request stating I'm doing 1 hit, I can do 100 hits a minute, am I over limit? Gubernator responds with either a status 0 (no, under limit), how many requests you have left, and the time the rate limit will reset. Or it will return 1 (yes, over limit), the time the limit resets, and you then have to hold your request until the reset time.

So you could use this as a self imposed rate limit for external services. I think the confusion is from the announcement, it sounded like this was a proxy type service that would hold your request until the limit was met. From what I read, it looks like you ask for the rate limit status, then its up to you to actually limit your requests.

You can use haproxy+squid for this purpose. Basically implement the rate limiting on haproxy and send requests to squid “backend” that is a forward proxy to the internet.

I'd usually use a queue + worker for this purpose. You will have to work out how to deal with old requests, backlogs, etc.. But then, if you are considering effectively forcibly making requests time out to internal servers, you will also need a strategy for handling that, retries, etc.

Isn't that just Service Discovery? https://kubernetes.io/docs/concepts/services-networking/serv...

You can do service discovery with a k/v store like etcd, or consul, which is how Kubernetes does service discovery. In fact, Consul very much positions itself as a service discovery tool.

I'm not sure how it would be possible to do it in a "library" (without external centralized storage).

It maintains zero configuration state. It is not completely stateless.

Consumption of available requests is maintained statefully in RAM. If your Gubernator goes down, the quotas are effectively topped back up when Gubernator returns from the dead, upon next request from that client.