Hacker News Reader

Comment by GolfPopper | Hacker News Reader

What is it insanely useful for? (Besides convincing investors to sink more money into LLM-related companies? Because that is the one thing it does seem to truly be good at.)

LLMs generate text output that appears to be useful, but regularly is not. They're alleged to be a substantial boost to writing code, but that verdict seems to be in dispute. They can generate custom mediocre prose at scale, but that seems to be of ultimately limited utility (although it may be a godsend for propagandists).

We're coming up on the 4th anniversary of ChatGPT's release. And while I get that revolutionary technologies can take a while to mature, the Wright Brothers and Goddard weren't preaching imminent societal transformation by the end to the decade from the rooftops, either. (And that's before we get into the how they got there - getting to ignore laws and steal whatever they wanted might be insanely useful to a lot of people.)

View on HN

With good input (prompts, specs...) LLMs can generate code that is often correct, faster than a human could generate equivalent code. Even when there are bugs, it is still "useful" from purely a time savings perspective. If you don't like the results, you can iterate rapidly.

Yes, you can use it to generate crap. I find Claude especially bad at writing like a normal person.

LLMs are not being promoted as "this can help write code faster/better/cheaper" (for the sake of argument presuming that they really can), they're being promoted as "Cortana" (. And their underlying economics are likewise premised on "Cortana" (for a huge amount of money). Which is going to be a disaster if/when they don't deliver.

> What is it insanely useful for?

Gulling humans.

This is the primary strength of LLMs and the emtire secret to their current success.

> LMs generate text output that appears to be useful, but regularly is not.

No, they are empirically useful, and only getting more useful. This is not even a debate anymore.

What is useful about nearly starting a war based on incorrect output?

Yes it is. I find them empirically not useful. You may not wish to debate it, but the fact remains that there are a great many people who are not convinced of their usefulness.

LLMs are very good at writing code. The reality is that they are able to write code faster, at higher quality and with fewer bugs, with correct prompting. They are also really good at code analysis, penetration testing and discovery, and adjacent computer science disciplines.

No they are not perfect, nor do they produce the best code. But the undeniable reality is that any good engineer will produce more code, at higher quality, using an LLM.

So, that’s not really up for debate. The debatable part is if all that code is a good idea or has as much value as we think. The conversation has long moved passed “can LLMs write code?”. Yes, they can, very well, particularly if they’re steered by trained engineers.

We have good engineers that gave us curl, ffmpeg, the 4.4BSD, tmux, vim and emacs, x11,… To this day, no one can show any particular important software that comes from LLM assistance at scale. It’s all slop.

Building software takes time. It remains a difficult craft, with or without the help of LLMs. You can't simply build a piece of "important software" in just a few months, and the good models capable of reasoning on complexity came out just a few months ago. And no, it's not "all slop". If you're incapable of using tools to build something meaningful with them, it's on you - don't blame the tools. LLMs are qualified to help with developing software, and if you're focusing on just the code-generation aspect of it, you're using it wrong; that's not how it's done.

llama.cpp has significant AI contributions. The Linux kernel too is receiving AI patches. You are seriously out of touch.

Well it's been barely a year, but basically all commercial software, and most open-source software, is engineered with AI assistance. It's not all slop, you just aren't looking around. Ask any engineer at any company - are they using LLMs to assist in writing code? All of them will say yes. Source: I'm a software engineer working on commercial software at a company, I also talk to many engineers at other companies.

Agreed. Despite the many claims of how awesome LLMs are for productivity, we have yet to see that supposed productivity produce fruit. Moreover, I dispute the claims of productivity: in my own usage I find them to be at best neutral, or even a drain on productivity. In my opinion, there is to date zero evidence of the supposedly insane utility.

What would you consider as sufficient evidence of LLMs being useful in a particular domain?

To meet the threshold of "insanely useful"? The Sagan standard is, "Extraordinary claims require extraordinary evidence."

I can see that some people can make some use of them. (This is true of almost everything.) Whether or not that usefulness is worthwhile overall, whether it is a net good, or even ethical is a different question. But insanely useful?

Computers are insanely useful. So are engines. Water. Sunlight. Electricity. Grain and bread. Writing. Printing. And I don't feel bad making those sorts of comparisons, because that's the level of impact LLMs' advocates are promising. But it's not what we have.

What I would consider sufficient evidence for insanely useful? Reliably replace a human in prolonged, arbitrary, detailed interaction, without any inhuman screwups.

Reply on news.ycombinator.com