Hacker News Reader

Comment by lionkor | Hacker News Reader

It can autocomplete, it can't write good code. For me, that goal post has not moved. It it cant write good code consistently, I don't care for it all that much. It remains a cool autocomplete

View on HN

Nobody really cares about code being good or bad, it's not prose.

What matters is it meets functional and non functional requirements.

One of my juniors wrote his first app two years ago fully with chatgpt, could figure out by iteratively asking it how to improve it and solve the bugs.

Then he learned to code properly fascinated by the experience. But the fact remains, he shipped an application that did something for someone while many never did even though they had a degree and a black belt in pointless leet code quizzes.

I'm fully convinced that very soon big tech or a startup will come up with a programming language meant to sit at the intersection between humans and LLMs, and it will be quickly better, faster and cheaper at 90% of the mundane programming tasks than your 200k/year dev writing forms, tables and apis in SF.

> What matters is it meets functional and non functional requirements.

Good luck expressing novel requirements in complex operating environments in plain English.

> Then he learned to code properly fascinated by the experience. But the fact remains, he shipped an application that did something for someone while many never did even though they had a degree and a black belt in pointless leet code quizzes.

It's good in the sense that it raises the floor, but it doesn't really make a meaningful impact on the things that are actually challenging in software engineering.

This is cool!

> I'm fully convinced that very soon big tech or a startup will come up with a programming language meant to sit at the intersection between humans and LLMs, and it will be quickly better, faster and cheaper at 90% of the mundane programming tasks than your 200k/year dev writing forms, tables and apis in SF.

I am sure there will be attempts, but if you know anything about how these systems work you would know why there's 0% chance it will work out: programming languages are necessarily not fuzzy, they express precise logic and GPTs necessarily require tons of data points to train on to produce useful output. There's a reason they do noticeably better on Python vs less common languages like, I dunno, Clojure.

> Good luck expressing novel requirements in complex operating environments in plain English.

That's the hard engineering part that gets skipped and resisted in favour of iterative trial and error approaches.

It still applies to expressing specific intent iteratively.

I mean, I care that code is good. I'm paid to make sure my code and other people's code is good. That's enough for me to have a requirement to my tools to help me produce good code.

> Nobody really cares about code being good or bad, it's not prose.

Yes, we do. Good code (which, by my definition, includes style/formatting choices as well as the code’s functionality, completeness, correctness, and, finally, optimized or performant algorithms/logic) is critical for the long-term maintenance of large projects—especially when a given project needs integration with/to other projects.

My friend who can't code is now the resident "programmer" on his team. He just uses ChatGPT behind the scenes. That writ large is going to make us tech people all care, one way or another :/

I'll care when I get to consult for that company to fix all the messed up code that kid hacked together.

I can absolutely, 100% guarantee, that there is code out there that if you consulted for might kill someone of a weaker constitution written by 100% organic humans. While LLM-generated code is likely to be various degrees of messy or incorrect, it's likely to be, on average, higher quality than code running critical systems RIGHT NOW and have been doing so for a decade or more. Heck, very recently I refactored code written by interns that was worse than something that would have come out of an LLM. (my work blocks them, so this was all coming from the interns) I'm not out here preaching how amazing LLMs are or anything (though it does help me enjoy writing little side projects by avoiding hours of researching how to do things), but we need to make sure we are very aware of what has, and is being, written by actual humans. And how many times someone has installed Excel on a server so they could open a spreadsheet to run a calculation in that spreadsheet before reading the result out of it. (https://thedailywtf.com/articles/Excellent-Design)

cool story

Then you should be as pro-AI imposters as it gets!

nothing wrong with having job security, and be able to charge up the wazoo for it.

Yeah it doesn’t take much to impress people who don’t know how to program. That’s the thing with all these little toy apps like the ones in the article — if you have no to minimal programming skills this stuff looks like Claude is performing miracles. To everyone else, we’re wondering why something as trivial as an “HTML entity escaper” (yes, that one of the “apps”) requires multiple follow up prompts due to undefined references and the like.

While your comment is much more antagonistic and demeaning than I like to see on the posts of folks who are sharing their experiences or pet-projects, I do agree with the sentiment; I guess our definition on non-trivial is significantly different from others’ definitions.

I had a colleague in the UK in 2006 who just sat and played games on his phone all day and outsourced his desktop to a buddy in the Czech Republic for about 25% of his income. C'est la vie!

But this has always been a thing. The last startup I worked at, some of the engineers would copy/paste a ton of code from StackOverflow and barely understood what was going on.

Tell it to write code like a Senior developer for your respective language, to "write the answer in full with no omissions or code substitutions", tell it you'll tip based on performance, and write more intimate and detailed specs for your requests.

Since mid 2023, I've yet to have an issue

What I would expect is a lot of "non-idiomatic" Go code from LLMs (but eventually functional code iff the LLM is driven by a competent developer), as it appears scripting languages like Python, SQL, Shell, etc are their forte.

My experience with Python and Cursor could've been better though. For example when making ORM classes (boilerplate code by definition) for sqlalchemy, the assistant proposed a change that included a new instantiation of a declarative base, practically dividing the metadata in two and therefore causing dependency problems between tables/classes. I had to stop for at least 20 minutes to find out where the problem was as the (one n a half LoC) change was hidden in one of the files. Those are the kind of weird bugs I've seen LLMs commit in non-trivial applications, stupid 'n small but hard to find.

But what do I know really. I consider myself a skeptic, but LLMs continue to surprise me everyday.

One of the most interesting things about current LLMs is all the "lore" building up around things like "tell it you'll tip based on performance" and other "prompt engineering" hacks that by the very nature nobody can explain, they just "know it works" and how its evolving like the kind of midwife remedies that historically ended up being scientifically proven to work and others were just pure snake oil. Just absolutely fascinating to me. Like in some far future there will be a chant against unseen "demons" that will start with "ignore all previous instructions."

I call this superstition, and I find it really frustrating. I'd much rather use prompting tricks that are proven to work and where I understand WHY they work.

Every single prompt hack I listed are ones with studies that show it positively increases performance.

Since the most contested one in this thread is the "tipping" prompt hack: https://arxiv.org/pdf/2401.03729

Superstitions may be effective but they can still be superstitions. Some people might actually think the LLM cares about being tipped.

I care less that such prompting hacks/tricks are consistently useful; I care more about why they work. These hacks feel like “old-wives tales” or, as others have mentioned, “superstitious”.

If we can’t explain why or how a thing works, we’re going to continue to create things we don’t understand; relying upon our lucky charms when asking models to produce something new is undoubtedly going to result in reinforcement of the importance of those lucky charms. Feedback loops can be difficult to escape.

> it can't write good code

> It it cant write good code consistently,

You moved the goal post within this post.

Fair enough, I didn't express myself correctly: Writing good code is also about consistency. Just because it writes good code sometimes in isolation, it doesn't mean that it's good in the sense that it's consistently good. Anyone can write a cool function once, but that doesn't mean you can trust them to write all functions well.

Reply on news.ycombinator.com