Comment by ToucanLoucan | Hacker News Reader

Comment by ToucanLoucan | Hacker News Reader

Parent Full thread

> And FWIW, I cannot imagine a programmer, in 2024, that remains dismissive of LLMs or the related tooling.

Hi.

> While they are grossly oversold by the LinkedIn crowd,

That is true.

> and are still a far ways from so-called "prompt engineers" replacing devs,

Also true.

> they're a massive accelerator and "second set of eyes", especially if you're doing varied, novel work.

That is not even remotely my experience.

Like I can envision a programmer who would get benefits from it, but bluntly put, the code I work on day to day is far, far too interesting to be handled by copilot, simply because there aren't nearly enough stackoverflow pages about it to be scraped. Honestly if you found yourself able to automate most of your job with copilot, if anything, you have my sincerest condolences. I can't imagine how utterly bored you are in your day-to-day.

IF copilot could get to a place where it could understand, comprehend, and weigh-in on code, that would be incredibly useful. But that's not what Copilot is because that's not what transformers are. They are fancy word probability calculators. And don't get me wrong, that has uses, but it is nothing I'd be comfortable calling a second set of eyes for for anything, save for maybe writing.

It's interesting that there is this division between programmers who claim LLMs are super helpful, and those saying they are useless.

While it's certainly possible that this divide is based on how 'hard' the problems people are using them on, my current theory is that some people use them like the proverbial rubber duck - in other words, a way to explore the code, and generate some stuff to work on, while thinking through the problem.

Personally, I have not yet tried it, so I'm curious which side of the discussion I'll fall ...

Some people seem to be taking me saying "you must work on boring code" as a judgement against them as developers, and it isn't. I'm speaking directly from my experience: If I asked it beginner-tier questions about how to do X in Y language, it would get those right quite often. I could see Copilot being very useful if you're breaking into a new language, or just knocking the rust off the gears of one in your head.

And like, even for those who write a lot of boring code, like... cool man. I don't judge people for that. We need all code written and all code is not exciting, novel, or interesting and there's nothing wrong with doing it. Someone's gotta.

I'm just saying that the further up the proverbial complexity chain I went, the less able Copilot was. And once I was quite in the weeds, it seemed utterly perplexed and frankly, not worth the time in asking.

Yeah, at least 90% of any job is just making license plates. I have worked on both very complex and challenging code and also very simple and easy code in my career, even within one job.

>as a judgement against them as developers

No one takes it as a judgment, and no one is offended. It's just a truth that when people make such claims, they're often exaggerating the uniqueness or novelty of what they're doing.

You described your work in another comment, and what you described is the most bog standard programming in the field. It's always the case.

This really touches on it. I'm a big advocate of these tools, but they author approximately zero lines of my code, yet I still find them invaluable and a wonderful tool to leverage, and do so constantly. Particularly in challenging projects and needs.

I suspect many who find them useless and decry them were sold an exaggerated utility and then were disappointed when they tried to generate libraries or even functions, then feeling deceived when there are errors or flaws, etc.

Yes, absolutely, they’re an ideal rubber duck and I’ve come to really value them for my work. Checking your sanity, pondering how certain operations might be implemented or how they could be optimized, finding where a logic bug might be in a snippet of code…

> I suspect many who find them useless and decry them were sold an exaggerated utility and then were disappointed when they tried to generate libraries or even functions, then feeling deceived when there are errors or flaws, etc.

No, I suspect the large majority (and this has been backed by surveys) of people that are dismissive of them are more senior and have been working in highly specific problem domains for a long time where this is rarely/never a good "general" answer for a problem, and have spent an inordinate amount of time debugging LLM-generated or LLM-Advised code by their peers that contains nefarious and subtle errors that look correct at a glance. I personally can tell you that for what I work on, in my domain, these tools have been a net time suck and not a gain, and I pretty much only use them to ask questions about documentation, which it often gets incorrect anyway (again, in subtle ways that are probably hard for someone who isn't very senior to detect).

Hope that helps.

I think young programmers who are less heavily invested in their skills and who haven't built a life that's highly dependent on using them are generally more interested in figuring out what programming with LLMs means.

But so are much older programmers who have seen it all, including the obsoletion of many of their skills, and who are not so dependent on continuing to use them as they could retire anyway.

It's more the middle (programmer) age senior programmers who are less likely to see any use.

I've seen the same pattern with artists' interest in generative AI.

But it's complicated because it IS also dependent on what you're doing. So it's hard to know if something is being dismissed correctly due to domain/expertise, or prematurely due to not putting the work in and figuring out what these tools mean.

> It's interesting that there is this division between programmers who claim LLMs are super helpful, and those saying they are useless.

My take is: if the project is doing something that has been asked a thousand times on stackoverflow and has hundreds of pages in the tutorial content mills, the LLM will tell you something reasonably meaningful about it.

I'd hazard a guess that most people overenthusiastic about those tools are gluing together javascript libs.

This is not necessarily a bad thing, we even asked a LLM today at work to generate some code for a library that we didn't know how to use but seems fairly popular, and the output looked like it would make sense. (Can't tell you how it ended up because I wasn't the one implementing the thing.)

However, we also spent 2 hours in a group debugging session because we're working on a completely custom codebase that isn't documented anywhere on geeksforgeeks, stackoverflow or anywhere else public. I highly doubt that even a local LLM would be able to help, and no way this code is leaving the premises.

>if the project is doing something that has been asked a thousand times

There are many billions of lines of high-quality, commented code online, covering just about everything. Millions of projects. All of Linux. All of Android. All of PGSQL and SQLite and MySQL and Apache and Git and OpenSSL and countless encryption libraries and countless data tools, video and audio manipulation, and...

Every single project is absolutely dominated by things that have been done many, many thousands of times. The vast bulk of your projects have zero novelty. They're mixing the same ingredients in different ways. I would think any experienced developer would realize this.

>I'd hazard a guess that most people overenthusiastic about those tools are gluing together javascript libs.

At this point it's comedy how often this "oh I understand that the noobs get value from this, but not us Advanced Programmers". It's absurdist and honestly at this point I just shake my head. My day is filled with C++, Python, Rust, Go, the absolute cutting edge of AI research, and I find these tools absolutely invaluable now. They are a massive accelerator. Zero JavaScript libs or "LOL WEB DEV" programming in my life.

Yes, you mentioned those things that are documented everywhere. I do use LLMs to give me skeleton code for those parts I'm not familiar with.

How about a full equivalent of Qt that is proprietary and has absolutely nothing public in it? How is a LLM going to help with that? There is no public info anywhere.

> the absolute cutting edge of AI research

No offense but there are billions of public pages about "AI" research since it's the new gold rush. Of course LLMs have material about all your libs.

>but there are billions of public pages about "AI" research

Billions? For many of the things I am working on there are zero public pages outside of research papers. I said nothing about working with libs. Again, I'm not asking an AI "here's my project now finish it", I'm working with AIs for the countless little programming challenges and needs. Things that mirror things done in many, many other projects, most having nothing to do with my domain.

As an aside, starting that with "no offense" as an attempt to make it insulting is...weird.

I feel like this discussion is taking place ten years ago. The weird reference to StackOverflow is particularly funny.

There is a lot to unpack here, but is your code so interesting that it defies understanding by an AST? Code models are trained to semantically represent code, so unless you use semantics that exist outside of software engineering the claim that your code is too unique for llm is false.

Maybe you are imagining a case where the entire codebase is generated by a single prompt?

The shorter and also "not breaching confidentiality" answer I can give is we're dealing with setting up custom sockets over incredibly long range wireless connections that require clear and verified transmission of packets of data, rolling both our own messaging protocol and security features as we go.

When last I tried anyway, Copilot was, frankly, useless.

Fwiw, copilot is not a particularly powerful LLM. It's at most glorified smarter autocomplete. I personally use LLMs for coding a lot, but Copilot is not really what I'd have in mind saying that.

Rather, I'd be using something like the Zed editor with its AI Assistant integration and Claude Sonnet 3.5 as the model, where I first provide it context in the chat window (relevant files, pages, database schema, documents it should reference and know) and possibly discuss the problem with it briefly, and only then (with all of that as context in the prompt) do I ask it to author/edit a piece of code (via the inline assist feature, which "sees" the current chat).

But it generally is the most useful for "I know exactly what I want to write or change, but it'll take me 30 minutes to do so, while with the LLM I can do the same in 5 minutes". They're also quite good at "tell me edge-cases I might have not considered in this code" - even if 80% of the suggestions it'll list are likely irrelevant, it'll often come up with something you might've not thought about.

There's definitely problems they're worse than useless at, though.

Where more complex reasoning is warranted, OpenAI o1 series of models can be quite decent, but it's hit or miss, and with the above prompt sizes you're looking at 1-2$ per query.

An abstract syntax tree is not semantics. And language models don’t do this kind of explicit syntax parsing and representation at any rate.

I'll admit I haven't seen the training data but some basic googling shows a subset of the labeling is syntax annotations. I am not claiming LLMs parse code in the way you are suggesting, but they certainly have token level awareness of syntax and probable relations which are the roots of any programming language.

Last time I researched AST parsing was quite rudimentary for LMs. The problem was preserving their structural properties, whereby a flattening approach via node traversal tended to remove. But you needed to do to put it into a format that language models could parse.

>Hi.

I was being dismissively rhetorical. I can actually imagine them because we see them on HN constantly, with a Luddism that somehow actually becomes a rather hilarious attempt at superiority. A "well if you actually get use out of them, I guess you're just not at my superior level of Unique Projects and Unique Code". It's honestly just embarrassing at this point.

>but bluntly put, the code I work on day to day is far, far too interesting to be handled by copilot

Hilarious. It's a cliche at this point.

Let me take it further and turn this on its head: The people who usually think LLMs aren't valuable to coders generally work on the most boring, copy/paste prattle. They stick to the same tiny niche all day every day. They're basically implementing the same thing again and again. It's so rote, and they're so profoundly unchallenged, that tooling has no value.

>with a Luddism that somehow actually becomes a rather hilarious attempt at superiority

>The people who usually think LLMs aren't valuable to coders generally work on the most boring, copy/paste prattle.

Here you are doing the same thing, aren't you?

Instead of calling people names, the biggest tell of a weak argument, why don't you explain the type of work you do and how using an LLM is faster than if you coded it yourself and and/or also faster than any current way of doing the same thing.

I'm assuming you are a senior+ level coder.

But...I'm not doing the same thing. In actuality I'm saying I'm a fairly typical programmer in a common situation: I work across a variety of languages and platforms and toolings and projects, building solutions for problems. The truth is that extraordinarily few programmers are working on anything truly novel. Zero of the readers of this comment are, in all likelihood. The ridiculous notion that someone has so unique of a need that it hasn't been seen is kind of hilarious nonsense. It's the "I'm so random! Other girls aren't like me" bit.

>Instead of calling people names

Who called anyone a name? Luddism? Yes, many HN participants are reacting to AI in a completely common rejection of change / challenge, and it recurs constantly.

>how using an LLM is faster than if you coded it yourself

I am coding it myself. Similar to the other guy who talks about putting an LLM in "charge" of his precious, super-novel code, you're setting up a strawman where using an LLM implies some particular scenario that you envision. In reality I spend my day asking questions, getting broad strokes, getting code commented, asking for API or resources, etc.

> Who called anyone a name? Luddism? Yes, many HN participants are reacting to AI in a completely common rejection of change / challenge, and it recurs constantly.

You should read up on what Luddism and Luddists were actually about. They didn't think the machines were evil or satanic, which is the common cultural read. They assumed (correctly) that the managerial class would take full advantage of increased productivity of lower-quality goods to flood the market with cheap shit that would put competitors out of business, and let them fire 4/5 of their workforces while doing so. And considering the state of the textile industry today, I think that was a pretty solid set of projections.

Luddites didn't oppose automation on the basis that machines are scary. They were the people who worked the machines that already existed at the time, after all. They opposed them on the basis that the greedy bastards who owned everything would be the only ones actually benefiting from automation, everyone else would get one kind of shaft or another, which again: is exactly what happened.

This, actually is incredibly analogous to my opinions about LLM. It's an interesting tech that has applications but is already being situated to be the sole domain of massive hyperscalers and subject to every ounce of enshittification that follows every tech that goes that way, while putting creatives, and yes some coders, out of a job.

So yes, it was name calling, but also I don't object to the association. In this case, I'm a Luddite. I am suspicious of the motivations and the beneficiaries of automation being forced into my industry and I'm not going to be quiet about it.

>And considering the state of the textile industry today, I think that was a pretty solid set of projections.

I think it's just about all industries these days.

Yes, so many quote meanings have been malformed over the years, such as "a rolling stone gathers no moss," is considered good, while originally bad. "Blood is thicker than water," "Money is the root of all evil," etc.

The Luddites were right.

>You should read up on what Luddism and Luddists were actually about.

They were primarily opposed to automation because it devalued the work they did and the skills they held. That is the core essence of Luddism. They thought if they destroyed the machines, automation could be stopped. There were some post-facto justifications like product quality, but if that was true they'd have no problem out-competing the machines.

Yes, it is Luddism that drives a lot of the AI sentiment seen on HN, and it is not only utterly futile and basically people convincing themselves and each other while the world moves on. There is no "name calling", and that particular blend of pearl clutching is absurd.

imho a lot of "luddism" so labeled by pro-ai bros is just people furious about shoddy artefacts that degenAI produce. That compares to original luddism, difference being the original 19th-century opposition against industrial revolution had proven them wrong with improved quality whereas genAI hasn't.

> the greedy bastards who owned everything would be the only ones actually benefiting from automation, […] which again: is exactly what happened.

What also happened is that everyone can buy clothes incredibly cheaply. Which seems like a widespread benefit.

I suspect you're overstating the degree to which an LLM might be unsuitable for some types of work. For example, I'm a data scientist who works primarily in the field of sales forecasting. I've found that LLMs are quite poor at this task, frequently providing answers that are inappropriate, misleading, or simply not a good fit for the data we're working with. In general I've found very limited use in engaging LLMs in discussion about my work.

I don't think I'm calling myself a super special snowflake here. These models are just ... bad at sales forecasting.

LLMs aren't entirely useless for me. I'll use ChatGPT to generate code to make plots. That's helpful.

I would never recommend an LLM for sales forecasting. It's just the wrong tool for that job.

You seem to readily agree that my use case is inappropriate for LLMs, but not ToucanLoucan?

Zero LLMs have been trained on doing sales forecasting to my knowledge (and it isn't the right use regardless). In contrast, many LLMs have been trained on enormous quantities of code, coding languages and platforms and uses. Billions and billions of lines of code covering just about every sort of project. Millions of projects.

If someone says "Well my software dev project is too unique and novel and they are therefore of no value to me, but I understand it works for those simple folk with their simple needs", there is an overwhelming probability they are...misinformed.

Would it help if I said most “normal folk” applications of LLMs are a waste of time and money too then? Because I’m also absolutely a believer that a huge bubble burst is coming for OpenAI and company.

>Who called anyone a name? Luddism?

Sorry, perhaps I misinterpreted it.

>In reality I spend my day asking questions, getting broad strokes, getting code commented, asking for API or resources, etc.

Can you give me some concrete examples? I'd like to use it, but I'm currently of the mind:

  1. If it's boring code, I can write it faster than asking LLM to do it and fixing its issues.

  2. If it's not boring code, like say a rules engine or something, I'm not sure the LLM will give me a good result based on the domain.

I mainly stick to back end work, automation, building WebAPI's and DSS engines for the medical field.

Maybe I'm under and over thinking it at the same time. FWIW, I typically stick to a single main language, but where I usually work, the companies dictate a GP language for all our stuff: C# in my example. I do a small amount of Python for LLM training, but I'm just starting out with Python. I can see it being useful saying, "convert this C# to Python," but honestly, I'd rather just learn the Python.

That was a whole lotta words to say "Nuh uh." and as such I don't really have a response.

Reply on news.ycombinator.com