Hacker News Reader

Comment by luxurytent | Hacker News Reader

I'm not surprised. A lot of the code I'll write in a day is trivial, calling helpers, writing test scaffolding, etc. I'm still building cool things the other 60% of time, but this job as we know it comes with a lot of repetition. (esp. if you write unit tests ;-))

View on HN

The real question is, why we need to write "trivial" code? I think people have lost the plot there. We should be writing better compilers and libraries, instead of using AI that will write more code to deal with.

The obsession with code generation instead of creating proper abstractions is a scourge of today's programming. It's a regression in software engineering.

I agree to some extent. But there are redundancies that are inherently beneficial and make code more robust and readable, like separating interface from implementation (where some declarative elements have to be repeated), or explicit type annotations instead of full-program type inference. There are also strictly necessary redundancies, like when you pattern-match an ADT, you repeat its declared structure. For desirable or necessary redundancies like that, AI code completion can still save time.

Intelligent unit test creation (if it is indeed intelligent) is also a good use-case. Not to relieve the programmer from thinking, but to save typing and to point out potentially useful tests that you might not have thought of.

This is trivial stuff which is handled well by any competent IDE from 2015, and even a well enough-working set of editor templates.

I don't see where "AI" is required for stuff like this.

I agree, but also, this is actually where AI (whatever it is) would be helpful. Suggesting actual abstractions that could replace boilerplate. Of course, these can build on existing abstractions.

The boilerplate makes it readable by humans.

I have made the same point elsewhere to AI enthusiasts friends of mine.

The fact that defining the same task in natural language to an AI is easier than writing it in formal language, means the formal language lacks expressiveness [0], vocabulary, or more likely, both.

But if they are both at same level of expressiveness and vocabulary, formal language wins, because it can spot and prevent ambiguity.

The question is how scalable it is to implement such vocabulary into formal language.

[0] - The line is often blury. How many languages allow to straightforwardly express `clamp m between 1.0 and 2.0`? You can argue this is vocabulary, or this is expressiveness.

> How many languages allow to straightforwardly express `clamp m between 1.0 and 2.0`? You can argue this is vocabulary, or this is expressiveness.

And I think it would be a great place for AI in programming - automated refactoring. For example, it would see a code that clamps the value and could be replaced with a library function call clamp(m,1,2). (And it would also, presumably, made sure that the code is indeed the same.) So it could help you understand the vocabulary.

Or, if it sees you are doing this frequently, and it's not in the standard library, it would create the function.

But instead, this AI (more like artificial stupidity) will just suggest you the same thing you wrote because you don't know any better in your code base. That's not helping.

Good point. But I think current approach of large scale generative AI models is not sufficient for this.

Sometimes it is just the syntax of your language. We can't all be writing lisp where the syntax is just another abstraction.

> We can't all be writing lisp

Why not? Isn't it easier to learn new syntax than to figure out a correct prompting for AI? (Although actually, Haskell syntax would be something I would prefer to write in.)

> We should be writing better compilers and libraries

this is a bizarre claim to make and just completely ignores reality.

people ARE trying to make better compilers and libraries, and have been for decades. we are we are now after person-millennia of effort, and that still means the optima we're at requires writing some repetitive web template rendering code in JSX or whatever, which Copilot can automate. why didn't some homomorphic-monoid-transformer-auto-renderer win instead?

I think the reality is that "if you build it they will come" is simply not always true, and worse is sometimes better. I don't think we are climbing to an optimum, at least unless it is culturally perceived that good abstractions are more valuable than just generating code.

The auto-renderers did actually win. There were GUI building tools at the end of 90s (like Visual Basic) that still put most web frameworks to shame. But, for various business reasons (mostly pricing, really) they were replaced with "web technology" - something open source but worse.

(And I am not dissing OSS, either. For example, Qt/KDE had some great abstractions that could serve as a basis for building applications. But the urge to reinvent the wheel has won.)

I guess one difference is that Copilot can guess things that it’s not sure about and have you check over it, but a compiler can’t just guess what you want it it would be very unreliable

Compilers do however guess that you don't want something - it's called compiler warnings. Linters do that as well. To me what you describe is more about how we communicate with compilers than an inherent limitation.

I couldn't agree more with everything you wrote. The problem with repetition is not that it's hard to generate, the problem is it's hard to maintain!

I've used ChatGPT to find libraries and to help me line up types. It was very helpful.

unit test definitely a great use of copilot.

What are you talking about?

I don't know anyone who uses it for this; specifically tests are really bad if they're subtly wrong.

Maybe to scaffold the test function, but the actual test if completely useless if you don't trust it.

So like... generate code and have robust tests, or write robust code... but, it's really really daft to generate tests that might hallucinate some random crap (and copilot really does sometimes).

Other people have said this, but copilot is auto complete.

You use it every time you press tab.

Do you accept an auto-complete suggestion 40% of the time? ...mmmm, yes, well, guess why 40% of code is generated by copilot.

It's not because people are generating tests with it.

Most of the time I know exactly what tests I want to write but it's annoying to write them. Copilot autocompletes, I check it and then I use it.

For me the worst part about this is that writing tests tediously makes me reach for layers of abstraction on my test code. And then suddenly my test code is complicated and needs its own test code, and changing a test can often be problematic to the abstractions I foolishly employed.

Being able to churn out the boilerplate for tests, which can make DRY a non-concern, is great. I just hope that if I do this, I never get sloppy, and I always review the tests.

I agree! I try not to make test code too complicated, and prefer repetitive code over abstractions (else you have to test the tests!) so Copilot is useful there.

In my case about 50% of the test code is just boilerplate, so I usually type the test name and generate the rest. Most of the time I have to rewrite the actual test logic (although very trivial tests are sometimes correct), but I probably keep more than 40% if we're talking tests specifically.

> In my case about 50% of the test code is just boilerplate

Why not replace it with a function call?

Some languages are just noisy.

Go, Java I’ve experienced are noisier than others.

A common occurrence is something like a pool of 10 actions where a bunch of tests each do 3 to 7 of them. This is very hard to abstract with a function call.

That makes the test harder to read. Tests should be dumb.

In the case I was referring to, mainly to keep the code consistent with what was already there and the PRs small, as the code base is primarily owned by a different team. It's also pretty innocuous short tests that read well as they are.

I use it for unit tests. That's one of its best use cases. Obviously I read the code it writes down.

In a recent personal C project (a custom archive file extractor), I have used chatgpt to generate tons of unit tests.

And honestly, I was extremely impressed. With a bit of context, it was able to generate almost correct test data for a fairly complex binary data format.

I've also leverage it heavily to:

* generate doxygen comments

* get usage examples of libs I never used

* create a whole bunch of utils functions.

And honestly for all these tedious tasks, it has done a far better job than I would ever had.

Comments were consistent in styling, the base examples were fairly good, and the utils functions were well made, specially the error handling (which I would probably have semi-consciously skipped tbh).

In fairness, I modified most of this code slightly to make it fit my project structure, or tweaked it a bit what the IA didn't quite get it right.

(it never fully understood some offset fields in the file format, but got pretty close to at times. And in fairness my naming for these offset fields was a bit questionable).

Heck, as a test, I even threw at it an RFC-like spec of the file format, and asked him to generate a python parser. The result was not 100% correct, but definitely a good start to iterate on.

In the end, this side project took me 2 weeks to implement with chatgpt probably saving around 1 week of dev. It also greatly helped improving the quality of the project (better doc, better tests).

> I don't know anyone who uses it for this

that would be surprising - it does a fantastic job of producing e.g. dumb unit tests. for instance, Copilot + a Go table testing template means you can churn out the simple "make a request to this url with this data, ensure I get a 200 and the response contains 'id' and not 'error'" extremely quickly. the code is trivial but tedious, so you can quickly inspect for sanity and run to ensure they pass, then commit and have them checking future changes.

> it's really really daft to generate tests that might hallucinate some random crap (and copilot really does sometimes).

variations on this comment are all over these threads, which is bizarre. hallucinating means you waste a few seconds reading the code it produced, not that you commit incorrect code.

this isn't like ChatGPT advising people to drink bleach, Copilot is a dumb tool offering an expert (you) suggested solutions for your expert consideration

I guess this could be true in some languages and settings, for example when a notable portion of your tests verify correct behavior with null values, arguments of wrong type, etc. When talking about unit tests that verify the actual logic, you should naturally become more careful with your copilot suggestions. Especially if copilot also wrote the code it's testing.

Yeah, to me this doesn’t feel that different than using a library from another developer, just slightly more customized. Most code that is run is not written by the final developer, but by people further up the chain.

Reply on news.ycombinator.com