Hacker News Reader

Yes, I have wanted something like this for a while. I try to avoid using gpus where possible because of the expense, and the ephemeral nature of my use.

View on HN

Interesting to hear. What kind of workloads are you running?

Various forms of content analysis. It is mostly traditional heuristics with ml models sprinkled in. We don't run inference on every request, but it would be great to access a dynamicly provisioned gpu for a single request (kind of like its another serverless container in the system).

That's fascinating to hear and I think it would work really well with what we do.

What I am picturing is that you could run the whole workflow including traditional heuristics in a CPU instance, which would connect to a GPU on-demand.

If you are interested would love for you to try this. We're running a (very unprofitable) beta with a T4 instance + a CPU-only instance for $10/month for those who are willing to help us test this with production workloads. If you'd be interested would love to chat at [redacted]

Reply on news.ycombinator.com