Hacker News Reader

Comment by satisfice | Hacker News Reader

Not really. Developers become TESTERS. Except developers don't want to be testers, and will do it badly.

Testing is hard. No, sorry, I meant to write that COMPETENT testing is hard. Any idiot can do shallow testing and say he tested, and many do. That's not the same thing as doing a professional job of it.

But, the fact is, AI can't be a software developer, because a software developer is capable of being accountable. AI is a machine. Machines cannot be accountable. You can't sue them. You can't jail them. You can't reform them. A machine cannot be a citizen.

A programmer is someone who mediates the gateway between the world of machines and the world of people. This is because machines behave in very particular ways. Just as you should not represent yourself in a court of law, no one who doesn't understand machines should attempt to create a professional grade products with them... Or if they do, expect it to be poor quality.

View on HN

This is the truth. Everyone wants AI to write their tests, but that’s backwards. AI will write the code and programmers will write the tests. The tests are the real business logic - binary assertions of how a system should or should not behave. The internals of how that happens are inconsequential.

Inconsequential until you start caring performance, scaling, reliability, interdependence with other systems, maintainability, or any other things that tests don’t assert.

You can test performance, and you can monitor reliability. Scale and interdependence are spec requirements that are this theoretical AI's job. Maintainability is irrelevant to everyone here except the theoretical AI, as the AI does the maintaining.

This sounds a lot like some kind of government procurement process, which don't have a reputation for producing particularly good IT systems.

I've used ChatGPT to write my tests. It's pretty effective at getting the first 5-6 use cases written out, but in a style that isn't correct. Copy/paste a code module, ask it to write the tests.

Then, in the same context, you correct it by providing it with more information about how you like to structure the tests. It starts to understand the "style".

Then, you start to provide the context with testing concerns. By the end of it, the AI context is pretty good (80-90'ish%).

It's nice, as a developer, to be able to say: "test that asset is deleted when DELETE /asset/:id", and the AI converts that into a proper test case, using familiar variables / test structure / corrects the abbreviated text into a proper test title, etc..

I look forward to writing specifications ("property tests without RNG") in a theorem prover language and some AI writing the program against my specification.

Why not simply use AI to implement monitoring and evaluation so incredibly fine grained any human would go insane if they had to do it?

Strange argument, it's not common to sue and jail developers, and our average software isn't particularly good quality. Companies will use whatever works for less money, as they always did.

I also see very limited accountability beyond firing.

There was some in #Dieselgate (Developers going to jail for their code).

James Robert Liang got jail.

It is quite strange, since the defeat device part was actually quite insignificant to work around a vehicle reset during testing and skip the warm up timer.

The real scandal is that emission treatment in diesels did not work outside of synthetic kinda steady state tests. And that applied to way more manufacturers than VW.

A developer is accountable in the way of being fired. Not accountable in the way of paying for the loss.

In this way an AI is accountable too, I can switch from Claude to Mistral.

Testing is what LLM coding models are born for..

- We want this

- We Had this

- This happened

- Already at work

Reply on news.ycombinator.com