> serving models is very profitable (70%+)
All your argument hangs on this.
I see no evidence of this being true.
All your argument hangs on this.
I see no evidence of this being true.
https://www.mindstudio.ai/blog/anthropic-inference-margins-7...
its even higher depending on the model, how optimized it is, and the chips!
I wouldnt die on this hill
Just because it is on some blog post, it does not make it true.
I wasted the time to read the first blog post. It considers 100% utilization over the course of years to calculate an estimation, and it did not consider depreciation for the model itself. That thing is extremely extensive to create, and after a relatively short amount of time is considered outdated.