Hacker News Reader

Besides trying to run on your own hardware, anybody have recommendations for running some decent models on one of the many "AI clouds" providers? This is for sporadic use and so maybe one of the "serverless" providers that bill by the hour or minute or similar as opposed to monthly renting GPUs.

There are quite a few of them but their marketing is just confusing and full of buzz words. I've been tinkering with OpenRouter that acts as a middleman.

View on HN

use openrouter, and call it a day. auto switching between providers, connectivity to all clouds and even works with free models

Yeah, that's what I've been doing. But in terms of privacy policies, I have to review(and trust) 2 providers instead of 1. OpenRouter and whatever provider is used for any particular model. I agree with you that it is more convenient though.

I mean AWS bedrock fits your use case pretty much. They have a bunch of models that are serverless that you can use on a per token pricing cost.

Gemini api use also comes with a free tier.

Thanks, I'll check out Bedrock. I was under the impression they only provide "enterprise" access as OpenRouter uses them as one of the providers but I didn't actually check. Looking at their docs now, seems I was wrong.

Reply on news.ycombinator.com