Hacker News Reader

I'm running a gateway for my local LLMs, are you opening it to the internet? Who's going to craft a malicious payload?

was using LiteLLM for the company i work for. we were doing a poc.

But now we switched to leanroute.dev, we use our own provider keys. Gateway helps with the routing, caching, spend cap and monitoring usage.