Ask HN: Anyone else find ChatGPT 3.5 a much better co-programmer than ChatGPT 4? | Hacker News Reader

Ask HN: Anyone else find ChatGPT 3.5 a much better co-programmer than ChatGPT 4?

8 pointsandrewstuart7 comments

I always use ChatGPT 3.5 for assisting my coding because it seems to give much better answers, and is more willing to do what it is actually asked, than ChatGPT 4.

Anyone else having the same experience?

View on HN

I've noticed the exact opposite, in my experience GPT3.5 is more likely to make code that doesn't work correctly, mess up URLs and other strings, or forget about certain variables and not include them. It's also more likely to make up non-existent libraries and other code.

GPT4 in my experience has been much more consistent with it's results. It also doesn't seem to lose the context as quickly. And it can deal with more niche libraries a lot better.

Due to the lack of consistently I've found it difficult to use 3.5

I haven't tried it with LangChain type stuff so maybe it's different there?

As a side note - I've been noticing their performance change a lot in between their "updates" and they mostly seem to be getting worse at following instructions :/

Yeah GPT4 seems worse lately. I had it build a complex tree view and make incremental feature updates which really impressed me. But last week or so it's giving me junk - messing up variables or rearranging code for no reason even though the file I'm copying/pasting as reference hasn't changed in size.

Likewise. I find that I regularly hit roadblocks with GPT3.5 and then switching the convo to 4 (in the Playground, so I can just change the toggle on the side), 4 often gets it right. Increasingly, I just start with 4.

It depends! For example with Django, if I want something, which I already kind of know, but haven't done in a while, I will always go to 3.5 first, because its much faster, and I can get the answer within 5 seconds. But if there is any sort of complexity, I won't bother with 3.5, because its going to suggest me doing things very sub-optimal. Like rebuilding an entire Model's save method and creating a Mixin, while all I would need to do is override an existing admin function with a single line of code. I mean ... the suggestion is often valid, but switching from 3.5 to 4 on complex issues is like going from a side-kick who is a very experienced junior to a senior sidekick.

I have definitely noticed this.

In fact, today I tried the Observation/Thought/Action/Execution pattern found in LangChain agents and GPT-3.5 did it perfectly, but GPT-4 stopped after formulating the observation

I have consistently seen 3.5 hallucinate specifics that 4 gets right. We are talking entire language features, etc.

Nope. GPT4 is way better from my experience.

Reply on news.ycombinator.com