Hacker News Reader

It's pretty dependant on the questions you ask. Obviously when you push them to their limits, GPT 4 goes a lot further. But what about for your average queries though? I don't have the data to make a proper claim about that.

View on HN

> It's pretty dependant on the questions you ask.

That’s the description of a house of cards rather than some revolutionary new technology.

That's why we have Lmsys for. And spoiler alert, it performs better than 3.5 no matter the complexity of the question.

And are you sure people ask the chatbot arena the same caliber of questions they ask a single model? Wouldn't you tend to ask more difficult questions to find where one fails but the other doesn't? I'm not convinced that the average query will get a significantly better result from GPT 4. I only know that is true for more difficult queries.

Do you have access to ChatGPT? You can try it yourself (you can query GPT-4o and GPT-3.5 afterwards). The difference is large enough to be noticeable at least 80% of the time in my experience.

Reply on news.ycombinator.com