Hacker News Reader

Comment by scubbo | Hacker News Reader

> Hours of time saved

Come back in a week and update us on how long you've spent debugging all the ways that the code was broken that you didn't notice in those 15 minutes.

Usually I don't nitpick spelling, but "mimnutes" and "stylisitic" are somewhat ironic here - small correct-looking errors get glossed over by human quality-checkers, but can lead to genuine issues when parsed as code. A key difference between your two examples is that the failure-cases of an HTML download are visible and treated-as-such, not presented as successes; you don't have to babysit the machine to make sure it's doing the right thing.

EDIT: plus, everything that sibling comments pointed out; that, even if AI tools _do_ work perfectly (they don't, and never will), they'll still do harm when "working-as-intended" - to critical thinking, to trust in truth and reporting, to artistic creation, to consolidation of wealth and capital.

View on HN

> Come back in a week and update us on how long you've spent debugging all the ways that the code was broken that you didn't notice in those 15 minutes.

I was a non believer for most of 2024.

How could such a thing with no understanding write any code that works.

I've now come to accept that all the understanding it has is what I bring and if I don't pay attention, I will run into things like you just mentioned.

Just about the same if I work with a human being with no strong opinions and a complete lack of taste when it comes to the elegance of a solution.

We often just pass over those people when hiring or promoting, despite their competence.

I was being sold a "self driving car" equivalent where you didn't even need a steering wheel for this thing, but I've slowly learned that I need to treat it like automatic cruise control with a little bit of lane switching.

Need to keep the hands on the wheel and spend your spare attention on the traffic far up ahead, not the phone.

I don't write a lot of code anymore, but my review queue is coming from my own laptop.

> Usually I don't nitpick spelling, but "mimnutes" and "stylisitic" are somewhat ironic here

Those are errors an AI does not make.

I used to be able to tell how conscientious someone was by their writing style, but not anymore.

> I was being sold a "self driving car" equivalent where you didn't even need a steering wheel for this thing, but I've slowly learned that I need to treat it like automatic cruise control with a little bit of lane switching.

> Need to keep the hands on the wheel and spend your spare attention on the traffic far up ahead, not the phone.

Now _this_ is a more-balanced perspective!

(And, to be clear - I use AI in my own workflow as well, extensively. I'm not just an outside naysayer - I know when it works, _and when it doesn't_. Which is why unreasonable claims are irritating)

Yeah, that sounds very much like the arguments parents gave to those of us who were kids when the web became a thing. "Cool walls of text. Shame you can't tell if any of that is true. You didn't put in work getting that information, and it's the work that matters."

Except it's turns out it's not a problem in practice, and "the work" matters only in less than 1% of the cases, and even then, it's much easier done with the web than without.

But it was impossible to convince the older generation of this. It was all apparent from our personal experience, yet we couldn't put it into words that the critics would find credible.

It took few more years and personal experience for the rest to get up to speed with reality.

> Except it's turns out it's not a problem in practice

Come on, this problem is now a US president

There remains a significant challenge with LLM-generated code. It can give the illusion of progress, but produce code that has many bugs, even if you craft your LLM prompt to test for such edge cases. I have had many instances where the LLM confidentially states that those edge cases and unit tests are passing, while they are failing.

Three years ago, would you have hired me as a developer if I had told you I was going to copy and paste code from Stack Overflow and a variety of developer blogs, and glue it together in a spaghetti-style manner? And that I would comment out failing unit tests, as Stack Overflow can't be wrong?

LLMs will change Software Engineering, but not in the way that we are envisaging it right now, and not in the way companies like OpenAI want us to believe.

Proper coding agents can easily be set up with hooks or other means of forcing linting and tests to be run and prevent the LLMs from bypassing them already. Adding extra checks in the work flow works very well to improve quality. Use the tools properly, and while you still need to take some care, these issues are rapidly diminishing separately from improvements to the models themselves.

> Use the tools properly

> (from upthread) I was being sold a "self driving car" equivalent where you didn't even need a steering wheel for this thing, but I've slowly learned that I need to treat it like automatic cruise control with a little bit of lane switching.

This is, I think, the core of a lot of people's frustrations with the narrative around AI tooling. It gets hyped up as this magnificent wondrous miraculous _intelligence_ that works right-out-of-the-box; then when people use it and (correctly!) identify that that's not the case, they get told that it's their own fault for holding it wrong. So which is it - a miracle that "just works", or a tool that people need to learn to use correctly? You (impersonal "you", here, not you-`vidarh`) don't get to claim the former and then retreat to the latter. If this was just presented as a good useful tool to have in your toolbelt, without all the hype and marketing, I think a lot of folks (who've already been jaded by the scamminess of Web3 and NFTs and Crypto in recent memory) would be a lot less hostile.

The problem with this argument is that it is usually not the same people making the different arguments.

That would be true if I was making a argument of criticism of a certain person or class-of-people, but I'm not. I'm describing my observations about the frustrations that AI-skeptics feel when they are bombarded with contradictory messages from (what they perceive as) "the pro-AI crowd". The fact that there are internal divisions within that group (between those making absurd claims, and those pointing out how correct tool use is important) does mean that the tool-advisers are being consistent and non-hypocritical, but _it doesn't lessen the frustration by the people hearing it_.

That is - I'm not saying that "tool advisers" are behaving badly, I'm observing why their (good!) advice is met with frustration (due to circumstances outside their control).

EDIT: OK, on reading my previous comment, there is some assumption that the comments are being made by the same people (or group-of-people) - so your response makes sense as a defence against that. I think the observations of sources of frustration are still accurate, but I do agree that "tool advisers" shouldn't be accused of inconsistency or hypocrisy when they're not the ones make outlandish claims.

How about:

1) Unbounded claims of miraculous intelligence don't come from people actually using it;

2) The LLMs really are a "miraculous intelligence that works right out-of-the-box" for simple cases of a very large class of problems that previously was not trivial (or possible) to solve with computers.

3) Once you move past simple cases, they require increasing amount of expertise and hand-holding to get good results from. Most of the "holding it wrong" responses happen around the limits of what current LLMs can reliably do.

4) But still, that they can do any of that at all is not far from a miraculous wonder in itself - and they keep getting better.

With the exception of 1) being "No True Scotsman"-ish, this is all very fair - and if the technology was presented with this kind of grounded and realistic evaluation, there'd be a lot less hostility (IMO)!

I mean, I think the truth is somewhere in the middle, with a sliding-scale that moves with time.

I got limited access to the internet in the Netscape Navigator era, and while it was absolutely awesome, until around 2010, maybe 2015 I maintained that for technical learning, the best quality materials were all printed books (well, aside from various newsgroups where you had access to various experts). I think the high barrier to entry and significant effort that it required were a pretty good junk filter.

I suspect the same is true of LLMs. You're right, they're right, to various degrees, and it's changing in various ways as time goes on.

Ca 1994 was the tipping point for me, when I could find research papers in minutes that I wouldn't even know about if I had to rely on my university library.

What gets me the most about the hype and the people arguing about it is: if it is so clearly revolutionary and the inevitable future, each minute you spend arguing about it is a minute you waste. People who stumble upon game changing technologies don't brag about it online, they use that edge in silence for as long as possible.

> People who stumble upon game changing technologies don't brag about it online, they use that edge in silence for as long as possible.

Why? I'm not in this to make money, I'm this for cool shit. Game-changing technologies are created incrementally, and come from extensive collaboration.

>Come back in a week and update us on how long you've spent debugging all the ways that the code was broken that you didn't notice in those 15 minutes.

This so much - can't believe how much of these "I am not even reading the LLM code anymore it is that good" comments I am reading. Either you are all shit programmers or your "You are an expert senior software developer" prompts are hitting the LLM harder. Because I'm here LLMing as much as the next guy, hoping it will take the work away - but as soon as I start being lazy, jumping over the code and letting it take the wheel it starts falling apart and I start getting bug reports. And the worst part is - it's the code "I wrote" (according to git blame), but I'm reading it for the first time as well and reading it with attention to detail reveals its shit.

So not sure what models you guys are getting served - especially the OpenAI stuff for coding, but I'm just not getting there. What is the expert prompt sauce I am missing here ?

For me it’s a constant nudging the LLM in the right direction either one off like removing this over ambitious configuration value or something permanent via its internal rule system (e.g. cursor rules) like here’s how to always run this command.

I’m still telling it pretty much exactly what to do but it’s fuzzy enough to save a lot of time often.

> they'll still do harm when "working-as-intended" [..] to consolidation of wealth and capital.

Fairly sure you didn't mean this :-D

LLMs will probably lead to 10x the concentration of wealth.

Eh, phrasing ambiguity from the run-on sentence (analogous to how "I fought with" could mean "on the same side as" or "against").

Yes, I meant that LLMs will do harm to - will exacerbate - _the problem of" wealth consolidation.

> Come back in a week and update us on how long you've spent debugging all the ways that the code was broken that you didn't notice in those 15 minutes.

Same as if I let a junior engineer merge code to main w/o unit tests.

Complete garbage, of course.

Oh wait, my code is also trash w/o good unit tests, because I am only human.

Instead I'll write out a spec, define behaviors and edge cases, and ask the junior engineer to think about them first. Break implementation down into a plan, and I'll code review each task as it is completed.

Now all of a sudden, the code is good, independent of who/what wrote it!

Yep, precisely my point. Reviewing a one-shot in 15 minutes is irresponsible.

Reply on news.ycombinator.com