Hacker News Reader

Comment by hi_hi | Hacker News Reader

Every single one of these discussion, at some point, devolves to some version of

- <LLM Y> is by far the best. In my extensive usage it is consistently outperforms <LLM X> by at least 2x. The difference is night and day.

Then the immediate child reply:

- What!? You must be holding it wrong. The complete inverse is true for me.

I don't know what to make of this contradiction. We're all using the same 2 things right? How can opinions vary by such a large amount. It makes me not trust any opinion on any other subject (which admittedly is not a bad default state, but who has time to form their own opinions on everything).

View on HN

Lots of possibilities here.

People are learning to prompt LLMs in ways that produce better results for their LLM of choice, so switching to another one they find their approach no longer works as well.

Or.. LLMs have different personalities in terms of output; some being more or less direct/polite than others, or sounding more or less confident; and that is causing people to perceive a difference that in terms of factual answers may not be different.

Or just personal preference masquerading as intelligence - a classic among software engineers.

Most of the LLM quality discussions have a shelf life measured in weeks. The companies involved are leap frogging each other with model updates and tweaks every few weeks.

Try it yourself. I'm getting a lot of value out of just using chat gpt for coding. It's not without flaws. But I can get it to do a lot of routine stuff quite quickly. What I like about the desktop client is that a prompt is just one alt+space away. I usually just copy paste whatever I'm working on and then ask it to do stuff to it.

There's some art to the prompting and you usually have to nudge it to not be lazy and do the whole thing you asked for. It seems engineers on the other side are working really hard to minimize token usage.

I find it's increasingly the UX that's holding me back, not the model quality. Context windows are now big enough to hold a lot of stuff. But how do you get everything in there that matters? Manually copy pasting together stuff is tedious. I actually wrote a script (well, with some llm help) that flattens things in my repository into a file that I then simply attach to a conversation. Works surprisingly well.

Personally I've been getting great results from Claude. I was initially using chatGPT, but now I'm finding Cursor + Claude to be a good fit.

And no, I'm not taking the alternative view to you just to prove my point haha.

I feel like AI is ripe for a tabs vs spaces moment. If only Silicon Valley (the show) was still around. The material would practically...write itself...

I also found that passing all the codebase in a single file in the context works really well. I've tried Cursor et al, but I found that it not having the full context of the codebase (and having to do some back and forth for it to requests files) was slower and didn't really yield any better results. Granted I work on projects where the codebase fit in 100-200kb text files, but I'm still only at 20% of the context limit in Claude.

Also I found the UX of Claude to be better for this, especially their Projects feature. I can just put the codebase in the Project's context, and start a new conversation to ask different questions/solve different problems.

The only pain point I have is that it seems to be pretty optimized to only show changes in existing files, not rewriting them in full, which is a bit of a pain to copy-paste into my IDE. I'll see if I can write out a system prompt to force it to generate diff or a similar format that could more easily be applied automatically to my code.

Same reason as programming language flamewars. Coders assume their individual experience holds for every domain. Yet LLMs are good for some things, bad at others.

There are tests for LLMs coding capabilities.

However, unless one is severely worse, I don't think LLM choice matters much, but how well the tool that uses LLM is thought.

So it's not Claude vs Chat GPT vs Llama vs Qwen vs Deepseek but Copilot vs Clide vs Cursor vs Aider.

They might very well be holding it wrong. Getting decent use out of an LLM requires knowledge, skill and perseverance. It's a pretty new field and everything is improving and changing incredibly fast.

Ever heard of astroturfing?

Reply on news.ycombinator.com