Hacker News Reader

Full thread

throwaway888abc

Use The Cache Luke...

Langchaing exemples:

[1] Caching https://python.langchain.com/v0.1/docs/modules/model_io/llms...

[2] Fake LLM https://js.langchain.com/v0.1/docs/integrations/llms/fake/

View on HN

options if im not using langchain?

Cloudflare AI gateway [0]

[0] https://developers.cloudflare.com/ai-gateway/configuration/c...

options if im not using cloudflare?

You can do it easily with object caching / function memorization patterns in any modern language which should fit your desired solution. Best of luck!

there's an open source ai gateway - https://github.com/Portkey-AI/gateway

Reply on news.ycombinator.com