Hacker News Reader

Comment by fumeux_fume | Hacker News Reader

> If you’ve played with the most recent AI tools, you know: They’re very good coders, very good legal analysts, very good first drafters of writing, very good image generators. They’re only going to get better.

Most of the bullshitters will tip their hand pretty early that they're just hype men for AI. Right off the bat, the fact that AI is disruptive and transforming society is apparently self-evident because they never cite a single premise or event to back this up. In the quote above, the phrase "if you've played" stuck out to me. Yes if you play around with them a little it's easy to believe they're really good at so many things. When you stringently evaluate them, you begin to see they make a lot of mistakes and perform inconsistently on even trivial tasks.

View on HN

You sound more prejudiced against AI than they seem to be hyping AI in that sentence.

Yes, they’re very good coders; no, they aren’t perfect. Sometimes they make trivial mistakes or hallucinate, and other times they have unique insights about difficult problems or one-shot a lot of complex tasks.

If you don’t see the value of AI even at this point, you’re either lacking imagination and/or just too stuck in your own ways.

I appreciate the value of AI in my own work, but "they’re very good coders" and "they make trivial mistakes or hallucinate" seems incongruous.

For me, AI tools shine when I know enough about a topic to quickly error check, but not enough where I can code fluently without documentation. I'm sure it will get more useful over time, but that's where it's been for me for the last year or so.

> "they’re very good coders" and "they make trivial mistakes or hallucinate" seems incongruous.

It seems that way if you judge them the same way you would human coders, but they’re different. They might be able to do things that veteran coders can’t without spending days on it, and fail at things that beginners can do in half an hour.

A car might not be able to traverse difficult terrain as well as a horse, but it doesn’t mean the car is not a good mode of transportation.

I'm not sure if the analogy fits. I would say AI is more like a fast car that turns the wrong way every couple of blocks, or a car that randomly refuses to go on certain roads. Is it still a useful mode of transportation? Sometimes, but, it's not reliable.

The sibling comment says it better - it's about predicability of mistakes, and the effort and knowledge necessary to correct them.

Maybe the next generation of models will solve these problems, who knows.

> AI is more like a fast car that turns the wrong way every couple of blocks

So if you drive well and can control this "car", you are able to fix that and benefit from the fast "car" a lot.

> or a car that randomly refuses to go on certain roads

This doesn't seem like a good comparison. AI will do anything, it may just be wrong.

The AIs that I use will definitely not do anything, and it the limits are not always obvious without a lot of prodding.

I'm fine with ditching the car analogy though.

The problem I have with AI is that you cannot reason with them after they make mistakes. They always reply with "Good point!" and then they give me another iteration of bad fixes. You really have to know when you should give up and do things manually.

Gemini 2.5 is much more confident.

Anecdotally, I think this behavior is undesirable for most commercial LLM use cases. I have several friends that have complained about Gemini’s “back talking” and prefer ChatGPT’s relative sycophancy.

I was talking about ChatGpt, but other llms also have this problem. I don't really care how polite they are, etc., I just want to get the job done, but the llm often gets stuck at some point.

Yes, human coders are predictably 'bad' - it scales roughly with the complexity of the task and the years of their experience, along with some randomness ('brain farts'). LLMs are completely unpredictably 'bad' - sometimes a task that a high schooler can solve sends an LLM into a spin, sometimes an LLM solves a task that would require a PhD in the field. The unpredictability is what makes these tools so unfit for purpose.

> LLMs are completely unpredictably 'bad' - sometimes a task that a high schooler can solve sends an LLM into a spin, sometimes an LLM solves a task that would require a PhD in the field.

That doesn’t mean they are unpredictably bad, it means assuming that they behave like humans is a bad way of predicting their behaviour. “Unlike humans” and “unpredictable” are not the same thing. If you spend time working with them, you get a better sense of what they are good and bad at and get better at predicting their behaviour.

> The unpredictability is what makes these tools so unfit for purpose.

Also, their eagerness to produce. A good employee (senior or not!), knows when they're at the end of their wits and say so. In areas where LLMs are knowledgeable, but not knowledgeable enough, they'll still produce. They're stuck at unconscious incompetence, kind of like a systematic Dunning-Kruger effect. To solve that, we can either increase competence (moving out the area where this effect is noticeable), or maybe introduce some kind of feedback, self reflection (one could say, some kind of conciousness [0]).

[0] inspired by "Goedel, Escher, Bach"

I don't think they're good coders at all, I think they're good at generating small snippets of codes.

If you tried to replace a programmer with whatever LLM it would completely flop. Just write a script to fetch Jira tickets and let it rip - it won't work well. Because it doesn't know that maybe the Jira ticket is stupid, or that if they implement this change it requires insight from Team X, or that this bug is really intended behavior, etc.

Being a programmer is much more than, like, making a neat regular expression or generating slightly customized boilerplate. As it stands, these are assistants in which programmers guide and handhold them the whole way. They are not coders.

So whats actually important are PMs. Got it.

No, what I mean is you’re going to encounter hundreds of technical and business challenges while you’re writing code (that you don’t know before you write the code).

Making those decisions and communicating is a big chunk of being a programmer.

They aren't good coders. They know nothing about the problem domain requirements. (The meat of programming.)

> bu-but you can waste your day writing prompts to get around the fact that LLMs are brainless!

Yes, and?

I don’t understand this viewpoint at all. Even if AI tools are nothing more than complex autocomplete systems - and not some kind of new consciousness - that alone is enough to dramatically shift entire industries. And it already is doing so…this isn’t theoretical anymore.

Not OP, but I think the viewpoint here is simply "if someone makes an evidently false claim ('great coders') right off the bat, it's enough to color the reading of the rest of the article. That's independent of what you're saying

Fair enough, but not sure that's much better of a viewpoint. It might be a worse one. The off-hand comment was "very good coders" (not "great") and dismissing the person's entire thought process because of a nitpick is not a good way to learn anything.

Just check any mature project's commit history. It's rarely that huge, even for things that brought in features. Here's a PR[0] that's less than the usual rambling ChatGPT produces, and add a valuable feature to the software. More often than not, you spend more time designing the feature than actually coding it. And a huge part of the design process is dealing with impact to other parts of the project.

[0]: https://github.com/OpenRC/openrc/pull/723/files

The gap between demo impressive and reliably useful in real-world scenarios is still huge in a lot of cases. That said, I think the concern isn't that AI is flawless now, but that it's improving at a pace we haven't really seen with other technologies

It's really funny to see someone say this technology, which seems to be plateauing with marginal improvements after a few years, is improving at a pace we have never seen before, in an industry that is built on the microprocessor. Astoundingly ignorant.

The marketing for it all has been something else. It's a technology that demos extremely well, and is surface level very impressive, and to those who wern't paying any attention to its evolution, appeared to come out of nowhere.

But this line of "its only going to get even better" is a mantra that's endlessly expounded on, a brain worm even. It's never backed up with any observable evidence. It's marketing that they're tricking people into repeating.

Correct. We've reached negative returns on increasing parameter counts. This is a dead end unless they figure out how to microtune models on ultra-specific niches.

there are hundreds of thousands of developers, lawyers, artists, musicians, etc who have transformed what they're doing with AI

The progress is fast, no doubt - but the hype sometimes skips over how messy the middle still is.

> very good legal analysts

This one in particular is a hearty kek.

I'm sick of this forum's lame performative cynicism masquerading as depth—it's a lazy cop-out that spares you people from the uncomfortable, demanding work of actually building something better.

Technology that any of us 5 years ago would've thought was a hundred years away and all you read is moving goalposts from ornery developers in denial.

Silicon Valley is cooked.

Reply on news.ycombinator.com