Hacker News Reader

Comment by whywhywhywhy | Hacker News Reader

I used to be concerned with this back when GPT4 originally came out and was way more impressive than the current version and OpenAI was the only game in town.

But Nowadays GPT has been quantized and cost-optimized to hell that it's no longer as useful as it was and with Claude or Gemini or whatever it's no longer noticeably better than any of them so it doesn't really matter what happens with their pricing.

View on HN

Are you saying they reduced the quality of the model in order to save compute? Would it make sense for them to offer a premium version of the model at at a very high price? At least offer it to those willing to pay?

It would not make sense to reduce output quality only to save on compute at inference, why not offer a premium (and perhaps perhaps slower) tier?

Unless the cost is at training time, maybe it would not be cost-effective for them to keep a model like that up to date.

As you can tell I am a bit uninformed on the topic.

Yeah, as someone who had access to gpt-4 early in 2023, the endpoint used to take over a minute to respond and the quality of the responses was mindblowing. Simply too expensive to serve at scale, not to mention the silicon constraints that are even more prohibitive when the organization needs to lock up a lot of their compute for training The Next Big Model. Thats a lot of compute that cant be on standby for serving inference

Reply on news.ycombinator.com