Comment by version_five | Hacker News Reader

Comment by version_five | Hacker News Reader

My preference is not to have to change my code to use some special framework, and just get access to a gpu machine I can run my stuff on.

I'm assuming you know what you need for a GPU. If you're unsure, consider trying to run inferences on a CPU and see how long it takes and if it could work.

And then just look at price and reliability for a gpu machine with the different cloud providers. Ovh is cheap but the only thing worse than their reliability is their customer service. Various niche players offering V100s used to pop up that were pretty cheap. AWS is more expensive, more reliable, they may still have availability problems. Paperspace looks pretty good. Etc.

> are worth avoiding so you don't get stuck with somebody else's framework.

Modal eng here. Modal is not setup as a framework. Think of more as Python-defined serverless infrastructure that has native support for the Python runtime. This is in some places called "Infrastructure from code", as opposed to "Infrastructure as code" which means just source-controlling K8s YAML and Cloudformation.

A major benefit of this approach is that the cloud becomes part of your dev loop, as opposed to doing `docker build`, `docker push`, `kubectl`, etc just to ship a change to a GPU.

In the script I posted Modal APIs are mixed in with standard Python code for brevity, but many customers just keep their code in their own modules and have a `modal_infra.py` module that defines the serverless infrastructure.

Understood, thanks for clarifying. I'll edit my post.

Glad to see this concept being used. I was thinking the other day how odd it is to have IaC then keep the code siloed from the app code. Why doesn’t the app make it’s own infra? I think you have articulated how that would work.

That makes sense, Brev.dev is a really simple way to run your code on a configured GPU without having to change your code. It'll also optimize your GPU to save money when possible.

Reply on news.ycombinator.com