Hacker News Reader

Comment by julienchastang | Hacker News Reader

As usual, when we see a thread on this topic on HN, the reactions tend to be bimodal: either "Yes, AI has transformed my workflow" (which is where I mostly fall), or "No, it's over-hyped." The latter often comes with an anecdote about how an LLM failed at a relatively simple task. I speculate that this diversity in opinion might be related to whether or not the user is employing a pro-tier LLM. Personally, I've been very impressed by ChatGPT-4 across a wide range of tasks, from debugging K8s logs to coding and ideation. I also wonder if some of the negative reactions stem from bad luck with an initial "first contact" with an LLM, where the results fell flat for any number of reasons (e.g., poor prompting), leading the user to conclude that it's not worth their time.

View on HN

I'm using the latest top paid models, gpt4o and claude 3.5.

They work when there's a lot of examples on github or google, but once you get into something that doesn't have a lot of examples like closed source code or rarely used libraries, it will start hallucinating and even mixing up different API versions to create a mess that doesn't work at all.

I don't believe LLMs will get any better than this without a new major breakthrough, but this is already better than using Google search.

I had the same problem with using the latest neo4j API and Sonnet 3.5 -- except it wasn't really a problem. I just created a project where I explain that its knowledge on neo4j is outdated and to instead use an API reference and changelog that I add to the project.

It's not magic, you need to think what you would need in a similar situation and then provide that to the LLM. It definitely does suffer from severe overconfidence, though -- if you were to think of it as a person.

Also, you need to break up your project into manageable portions and provide context to the other portions (without providing the entirety of them) for it to effectively work on the portion you want to work on.

> don't believe LLMs will get any better than this without a new major breakthrough, but this is already better than using Google search

I mean .. every single org is invested in that research right now

Money helps, but doesn’t guarantee breakthroughs.

The sort of transformation needed here seems achievable. There are various roadmaps to success whereas there were not any roadmaps to general language agents just a few years ago.

I don't see how it seems achievable.

Backprop was popularized in 1986. Transformers came out in 2017.

It could be >10-20 years before the next breakthrough comes. Given all the progress so far, these things still don't seem to be learning any logic.

True, if by "every single org" you mean "maybe a dozen orgs". All that the vast majority of companies is doing is figuring out how to run an existing LLM or API thereof and shipping an RAG integration button for their application.

I don't think so, I think it's more about openness. I've noticed older software engineers tend to be more anti-LLM and quick to dismiss.

The shortcomings are aplenty, but they don't bother me. The things it can do weren't possible 2 years ago. I'll leverage those and take the bad with the good.

Similar experience with Tesla FSD. I know other Tesla owners who tried it a few times and think it's trash because they had to disengage. I disengage preemptively all the time but the other 90% of my drive being done for me is not something that used to be possible. I tried to give up my subscription because it's expensive and couldn't hold out two days.

Your self-driving car is so unreliable you have to manually disable it 10% of the time to stay safe?

What percent of the time do you have to drive your car?

>I don't think so, I think it's more about openness. I've noticed older software engineers tend to be more anti-LLM and quick to dismiss.

Wow, a highly ageist comment, if there ever was one.

Congrats. Trying for a job and looking for less competition, maybe?

Notice that your statement is as full of assumptions as mine. That was intentional on my part, to bring out my point.

I’m in the middle. I use paid models. It’s great for scripts, reminding me of commands, and surprisingly good at debugging error messages. But if I’m working in our main code base I’m basically never able to just use the code it spits out. It can get me in the ballpark of what I want, but it often does weird convoluted shit on the first pass. I can keep prompting to refine the code down to something closer to what I want, but at that point it would have just been faster to write the thing myself.

I kind of like LLMs for learning new languages. Claude or ChatGPT are good for asking questions. But copilot really stunts learning for me. I find nothing really sticks in my brain when I have copilot running. I feel like I just turn my brain off, which seems kind of dangerous to me in the long run.

I'm concerned about the "turning the brain off" effect too. When we put backup cameras on cars it means that everyone can parallel park perfectly. Now take away the backup cameras and see what happens to our parallel parking.

I'm also concerned, because a lot of people are using these models for the early stage drudge-work and I hate to sound like Calvin's dad here, but I feel like those early stages are good for a developer to build a mental model of their larger goals and targets. Sure it's easy and a lot of boilerplate, but our brains are working while we're doing it, and for me, it feels like a necessary part of the process. I'm not convinced automating it away isn't going to result in a more difficult time understanding of your product and code base later on.

The way people put food on their table also matters.

I see a lot of negative reactions from programmers precisely because it is good at what they do. If you’re feeling threatened you’re much more likely to focus on the things it can’t do

To be fair there is a certain level of blindly-pragmatic "so AI can help me with programming - so what! It takes just as long to converse with it as for me to code it myself" - which is mostly true. But I think those people fail to grok the sheer ridiculous power of what they're saying - that these tools are close enough they're nearly at parity. Sure, the error-correcting architectures aren't sufficient yet for them to run unsupervised, but this is crazy already.

We do grok it. We also think they've reached the apex of the technology and have seen no evidence they'll close the hallucination gap.

You can usually tell that a lot of people just go off rumors they read once off Twitter or reddit or somewhere about hallucinations or doing math, against a weaker model, without every validating what they read online or updating their model of how well latest models work.

Just have to learn to let it go, despite xkcd 386.

I use gpt4o and claude 3.5 while coding every day. The rumors on twitter and reddit are accurate.

I constantly run into incorrect answers from the LLMs every day. Just recently I asked whether I needed to reverse the bit shift to mask the upper 24 bits in an IP address on a little endian platform and it incorrectly told me no probably because most of the answers on Google appeared to answer no to similarly phrased questions.

That's probably the best we can do for now with LLMs. Surely it would be unreasonable for LLMs to provide a correct answer when the majority of people on various forums also provide the wrong answer. Some day LLMs will be able to objectively provide truthful statements but we're not there yet. Regardless, that LLMs are competing with longstanding programmers is already an impressive feat, even if they're not 100% correct.

I'm not going to pass any judgement on what is reasonable or unreasonable for LLMs, just pointing out that it makes serious mistakes.

Many people here argue that LLMs are able to reason, you can check my post history 3 months ago to see an example of this.

And if you have to double check everything, how much time are you really saving? And is this thing really on track to replace programmers any time soon? Absolutely not.

I'm confused, you said earlier that you use it every day.

Where did I say I didn't?

I mean, call it dot producting if you don't want to use the word reason, but it takes a bit of multiplying matricies and converting tokens to words to be able to say that if b follows a, it doesn't mean that a follows b, but that not b follows not a (the contrapositive). sure, that's in an unknown amount of textbooks in the training corpus, but at what point does putting down a bunch of axioms and applying them ok each other lead to what you'd actually call reasoning? at some point when a human does it, the human's gone past its training corpus and entered the hallowed realm of reasoning, unless we're able to deliniate clearly where that line is and isn't, and then all agree on it, we're going to keep going in circles about if it can be called reasoning or not.

> Regardless, that LLMs are competing with longstanding programmers is already an impressive feat, even if they're not 100% correct.

But they're not. They're a little tool some programmers use to save a small amount of time. They're not replacing programmers.

And there's no evidence they will ever be as good as actual programmers.

Thankfully! In the 90' and outsourcing became a thing, programming as a career was over. and then when ChatGPT came out, programming as a career was over again. turns out, programming's not dead quite yet, though time will tell if it ever does.

Maybe you're using a strawman.

you need to give a counterexample to claim it's a strawman.

"The people who disagree with me have no good reason to" is prima facie a strawman.

Yes, I think if we ask about a more trivial thing we will have the same responses. For example, “Do you put the toilet paper roll with the paper coming over the top or hanging under the bottom?”.

Regarding LLMs I bet we will see them evolving. Don't forget about https://news.ycombinator.com/item?id=41269791 there are many problems that LLMs are no good for but they are being better that many Google search results, and that means something from the economic point of view.

I am sure that The Economist analytics is having a good moment /s.

Reply on news.ycombinator.com