Hacker News Reader

You train the model locally and push it for inference to the cloud?

What exactly are we talking about here?

Couldn’t you build a dual NVIDIA 20XX / 32 core / 64 GB for a sub $5k and then save money while training/developing faster?

View on HN

Except they (the gender non-specific singular) is probably running kubernetes and has multiple clusters of 10 or so gpu hosts. Not that I disagree, but spinning that up locally and orchestrating it will take time and money. And explaining why training is paused because you keep blowing breakers in the office will cost political capital.

You can just say “Except they are probably”.

Reply on news.ycombinator.com