Ask HN: How to manage AI APIs for SaaS application?

4 points·sbinnee··6 comments

I want to build an AI app. My idea is to support BYOK (bring your own key) or to manage one API key for each user.

What are the solutions to issue a lot of API keys for commercial apps? I would love to hear from the community.

As someone rightly pointed out already, issuing these many keys (a few thousands) is not an issue. But, handling abuse, retries, spikes, fallbacks, logging, monitoring, hacking attempts, etc. ... these are the real issues that you need to focus on.

My suggestion: go with openrouter... it offers management keys that you can use to issue keys on the fly for your customers. You can also set quota per key.

With this setup, everything gets offloaded to them. You just pay 5% commission (when you load credits to their platform).

I've used LiteLLM for my 1k-user service https://LibreThinker.com Works okay-ish, but some vendors now default to MFA on API keys (eg OpenAI) which makes things tricky; I decided to simply let those fail for now.

I am a bit familiar with litellm. Do you proxy server?

BYOK is becoming more common for AI SaaS. Interested to know how people handle rate limiting and key security at scale.

i did this with aws kms. i had a root key that encrypted & decrypted data keys, which were unique for each customer. those data keys then encrypted & decrypted sensitive customer info. so called envelope encryption

Honestly the API key part ended up being the easy part for me.

The messy stuff was abuse prevention, retries/fallbacks, and sudden cost spikes once real users started using different workflows.

Reply on news.ycombinator.com