Show HN: ReliableGPT run 200 GPT-4 requests in parallel

colab.research.google.com

14 pointsij233 comments

reliableGPT handles GPT-4 429 rate limit errors

it optimally allocate jobs for incoming requests to maximize usage of Requests/min and Token/min limits

Looks like this will only be effective for short prompts/responses, eg if you have 4k tokens in your prompt, you can only fire 10 requests/minute with 40k token/minute rate limit

Why not using just a fetch with retry function ?

It is, in fact, a try-catch. https://github.com/BerriAI/reliableGPT/blob/main/reliablegpt...

Reply on news.ycombinator.com