Hacker News Reader

Comment by paradite | Hacker News Reader

I just tried one prompt for a simple coding task involving DB and frontend, and Claude 3 Sonnet (the free and less powerful model) gave a better response than ChatGPT Classic (GPT-4).

It used the correct method of a lesser-known SQL ORM library, where GPT-4 made a mistake and used the wrong method.

Then I tried another prompt to generate SQL and it gave a worse response than ChatGPT Classic, still looks correct but much longer.

ChatGPT Link for 1: https://chat.openai.com/share/d6c9e903-d4be-4ed1-933b-b35df3...

ChatGPT Link for 2: https://chat.openai.com/share/178a0bd2-0590-4a07-965d-cff01e...

View on HN

Are you aware you're using GPT-3 or weaker in those chats? The green icon indicates that you're using the first generation of ChatGPT models, and it is likely to be GPT-3.5 Turbo. I'm unsure but it's possible that it's an even further distilled or quantized optimization than is available via API.

Using GPT-4, I get the result I think you'd expect: https://chat.openai.com/share/da15f295-9c65-4aaf-9523-601bf4...

This is a good PSA that a lot of content out on the internet showing ChatGPT getting things wrong is the weaker model.

Green background OpenAI icon: GPT 3.5

Black or purple icon: GPT 4

GPT-4 Turbo, via API, did slightly better though perhaps just because it has more Drizzle knowledge in the training set, and skips the SQL command and instead suggests modifying only db.ts and page.tsx.

I see the purple icon with "ChatGPT Classic" on my share link, but if I open it in incognito without login, it shows as green "ChatGPT". You can try opening in incognito your own chat share link.

I use ChatGPT Classic, which is an official GPT from OpenAI without the extra system prompt from normal ChatGPT.

https://chat.openai.com/g/g-YyyyMT9XH-chatgpt-classic

It is explicitly mentioned in the GPT that it uses GPT-4. Also, it does have purple icon in the chat UI.

I have observed an improved quality of using it compared for GPT-4 (ChatGPT Plus). You can read about it more in my blog post:

https://16x.engineer/2024/02/03/chatgpt-coding-best-practice...

Oh, I see. That must be frustrating to folks at OpenAI. Their product rests on the quality of their models, and making users unable to see which results came from their best doesn't help.

FWIW, GPT-4 and GPT-4 Turbo via developer API call both seem to produce the result you expect.

FYI, the correct method is

  created_at: timestamp('created_at').defaultNow(), // Add created_at column definition

Which Claude 3 Sonnet correctly produces.

ChatGPT Classic (GPT-4) gives:

  created_at: timestamp('created_at').default(sql`NOW()`), // Add this line

Which is okay, but not ideal. And it also misses the need to import `sql` template tag.

Your share link gives:

  created_at: timestamp('created_at').default('NOW()'),

Which would throw a TypeScript error for the wrong type used in arguments for `default`.

Reply on news.ycombinator.com