Throttler: a Clojure library for rate limiting | Hacker News Reader

Throttler: a Clojure library for rate limiting

brunov.org

68 pointsrobgering11 comments

Looks very well put together. I am not familiar with throttlers. What is throttling normally used for?

I'll give an example of a situation where I recently had to write a throttle function that works in the same way as this library.

We have a TCP connection down which we need to send (soft) real-time updates about a large data set. However, the thread sending out the updates has a fixed sized buffer per client that's shared between every part of the system communicating with that particular client. If the buffer gets full, it disconnects the client (incorrectly reasoning the fault was the client was requesting too much data).

In certain, not particularly uncommon circumstances, it was possible to end up needing to send out a large number of updates which would blow the buffer and disconnect the client. We considered various fixes that would give feedback about the size of the buffer, but the only feedback that would reliably not blow the buffer would be the block the producer when the buffer got full. This obviously has a huge rippling effect throughout the rest of the system, some of which have firmer real-time requirements, and was too big a task to fix at that time.

Looking at the updates, we observed there were two types of data patterns: frequent short bursts and these (comparatively) rare long sequence of updates. By rate limiting them to a maximum of n messages in any given window of a second the frequent short bursts would get sent immediately and unimpeded, while the long sequences would be broken up, giving the previous messages time to be dequeued from the buffer and sent down the TCP connection before the next batch arrived.

It's crude, but so far it's worked and given us time to work out a better system-wide solution to the problem.

Typically I use ratelimit (see below) to make sure I don't make outbound API calls too often, or soften processing bottlenecks in places where I don't control the throughput completely.

In Ruby, ratelimit (https://github.com/ejfinneran/ratelimit) works nicely and the limit is shared across processes, which is very convenient.

Keeping users from making an insane amount of requests to your site/api per some unit of time.

It looks more like it's for limiting your own outbound API calls, or resources usage, etc.

Hi, author here. That was exactly my motivation for writing it. There's all sort of middleware and plugins out there for throttling from the server side, but I found nothing that would let me control my own outbound API calls to a an external service in Clojure. I also wanted support for burstiness, and that was even harder to find.

I bet you could come up with other uses for it though. Someone told me over email they're planning on using it in traffic simulation for a game, for instance.

Not a clojure guy, but this throttler seems to be for single machine (Throttling not shared between servers).

API Rate limiters should share global state using redis or something.

So perhaps, this is more for throttling functions

For what it's worth, here's what I use for redis rate limiting: https://gist.github.com/anonymous/4dcb6892346ff17709fe It's in Python, but the bulk of it is a redis Lua script.

Can it be used from ClojureScript?

Hi, author here. I haven't tested it on ClojureScript yet, but since core.async is supported by CLJS I don't see the reason why it wouldn't work there as well.

If it doesn't, let me know on Github and I'll look into it! I don't have any experience with CLJS, but I'm interested in learning so it'd be a good excuse.

I submitted a pull request which sets up the project to use cljx to work in both Clojure and ClojureScript.

ETA: Just realized that <!! and >!! aren't available in ClojureScript, so it needs some more work to be complete, but the pull request should help you get set up to develop for both.

Reply on news.ycombinator.com