After 2 Weeks of Testing, What Do Developers Think About Claude 3?

I’m not fully decided but I haven’t seen Claude edging out ChatGPT in my tests. Every query I’ve had over the last week or so I’ve sent to both of them (manually writing the question in one, then copying it to send to the other as well, no fancy tricks) and the results have been very similar.

I have used ChatGPT’s responses in some cases and Claude’s in others. I never had a “oh wow, X is way better” case with either of them and sometimes I even sent the same replies to both for a few rounds of back and forth since they made the same mistake/misunderstanding.

However, I will say that ChatGPT’s UI is better than Claude’s. The chat-like interface with the conversation list on the left hand side feels better, the larger chat text input box and wider UI of ChatGPT feels better as well. Claude is very narrow for no good reason. Also the model selector for Claude is annoyingly placed, I’ve accidentally clicked it on multiple occasions. Lastly there is no app for Claude, there is no voice chat (that I’ve seen), nor can you modify the prompt/response which I didn’t think I’d miss as much as I do.

After this month is up I think I’ll stop paying for Claude unless something changes but I’ll continue to double-ask until then.

My very unscientific test is to ask new models I'm trying to build a basic React Native component using a specific library. "Please make me a react native component, using typescript and react native reanimated that creates an animation of a circle that bounces when it hits the ground. It should loop indefinitely."

Both GPT4-turbo and Claude 3 Opus produce a generally acceptable output. Claude 3 gave me a ball that bounces off the "ground" and "roof", but in a smooth and continuous motion. GPT4-turbo gives me a ball that bounces on the ground then "snaps" back up to the top and does it again. Mistral large gives me garbage output. It imports a bunch of function from react-native-reanimated that don't exist and produces nothing usable.

Technically GPT4-turbo's response is a bit more faithful to what I asked, I suppose. However, Claude 3 Opus animation looks much more "polished" even though I never asked for the ball to bounce off the roof.

I have tested it in huge contexes and it leads. It is great. Code writing? Don’t know. Refactoring? Miles ahead from GPT4. No laziness, true large context, YMMV

I have been using Claude 3 since it came out. And I am somewhat surprised to find anybody who claims that GPT-4 is about equal. For software dev tasks, for me, there is no comparison. I always have both tools open and I feed them the same exact question. Claude has been winning nearly 100% of the time.

The only downside for me is that Claude's limit of questions you can ask per day is much more limited than ChatGPT-4's. You exhaust your limit much faster if you feed it a lot of tokens/data. Which I often do.

I was impressed by the IKEA demo until I looked more closely and realized it was complete gibberish. Every single part is number "101339" and it keeps talking about "attaching the hammer-shaped piece"; it gets worse from there. Meanwhile the article breathlessly claims perfect performance. Given the clear lack of even cursory critical analysis I would take the other less verifiable claims with a small mountain of salt.

> Developers worldwide have tested it…

Claude does seem to be explicitly blocking people living in GDPR jurisdictions to sign up. I don’t know what to make of that, but this statement from the first paragraph rings false.