Hacker News Reader

Comment by jostmey | Hacker News Reader

I've noticed a substantial drop in the performance and usefulness of Chat GTP 4. It makes me think that OpenAI created Chat GTP almost by accident because they do not seem capable of refining and improving their model.

View on HN

It is well known that GPT-4 is downgraded because of GPU capacity. They ran out of GPUs to even train new models on, hence downgraded the consumer version of GPT-4 to free up GPU resources.

'refining and improving' skills doesn't particularly matter, because they could have just continued to serve the frozen GPT-4 0301 version without any issues, if they weren't forced to downgrade GPT-4.

I’m more worried some “bigger players” have exclusive access to its capabilities while the plebs get access to the basic version

Same happening at the version that Microsoft is hosting in their cloud. You can get on the waiting list, but it's heavily implied that if you don't have a good use case (aka lots of prompts/money coming in), you won't get access soon.

I see it differently. They built the best they could, but it was expensive. Now it feels to me that they are "optimizing" it for performance by seeing if they can lower the quality but still give people just enough value to pay for it. Makes me think they are trying to find a sweet spot between performance and cost. I am seeing this drop in quality, but mostly when it comes to the model remembering things you've told it in the past, and a much bigger dependence on custom instructions for prompts.

I still get value, but I have to give it the documentation I want it to focus on during the prompt, instead of referencing it earlier in the conversation.

This makes me think they are lowering it's ability to store context so that the calls don't cost as much to make.

Pure speculation: most likely quantizing models to 8bit to make them more efficient (and cheaper to run), and losing some accuracy on the way.

I still use it, but I have found it to be less useful for coding than it was earlier. So, something is up.

I agree, this is exactly my experience as well. My earlier requests for code were more thorough then they are now. I can still use it if I give it the documentation in the prompt, but it has trouble remembering context across prompts.

Reply on news.ycombinator.com