Hacker News Reader

Comment by pjc50 | Hacker News Reader

Software quality assurance can be automated in a way that artistic and textual quality can't.

That doesn't give you good taste, but .. for yer basic line of business or enterprise app, expectations were already low, and most websites have user-hostile design written into the requirements, so the damage isn't too bad.

I do think we've yet to see what the worst case for government contractor software project + vibe coding is. The benchmark is the Canadian gun registry https://calleam.com/WTPF/?p=1949 "In what may be the worst budget overrun in history, the costs to implement a registry of firearms balloons from $2M to $860M". Now add token spend to that.

View on HN

> Software quality assurance can be automated in a way that artistic and textual quality can't.

A lot of companies are creating AI assistants which take otherwise deterministic processes and makes them nondeterministic.

For example, I work in financial services and deal with a lot of data vendors. One of the big ones very recently added a chatbot to their UI, which on the second question I asked, provided entirely incorrect numerical but with confidence and 3-decimal place precision.

So the chatbot makes it "easier" to ask things, because you don't need to know which tab in the UI to use / or code to write in their scripting interface / or function to call in their Excel interface / or parameters to pass to get the correct answer.

Unfortunately the chatbot also may completely mistranslate your question, call the wrong function/pass wrong parameters and feed you back nonsense confidently.

Who is this helping?

Yeah, this is not a good use of AI and the customer service people are going to gradually work that out, especially once people start having the "you know this is a legally binding statement on behalf of your company, right" discussion.

You can't put an AI in a flow which requires reliable results. I don't think people who are used to determinism have coped with that.

> You can't put an AI in a flow which requires reliable results. I don't think people who are used to determinism have coped with that.

The problem is .. what flows don't need determinism? Search results / recommendation engine / ad targeting ?

Arguably the majority of companies & corporate users are using these tools in cases they expect determinism. Email inbox summaries. Search summaries. What is the value of x queries. You'd be shocked.

My favorite Google AI summary bug/quirk that seems to persist (I just tested it again) - "are Lillies OK for cats". The summary starts with "Yes, lilies are extremely toxic to cats. "

I first hit this with another plant (lavender) where the response was much longer and on an iPhone looked like this:

Yes, lavender (both the plant and its essential oils) [line break]

is considered toxic to cats.

> The problem is .. what flows don't need determinism? Search results / recommendation engine / ad targeting ?

That's not the relevant question, because the actual answer to what you asked is "all flows where human judgement is used".

The thing we need to not blindly use current generation AI for is "things where we accept the combination of an untrained (or barely trained) human with no QA".

IMO, the drive to use AI is not only fully automating a lot of things that shouldn't be so, but also revealing how some of them never should have been in the first place. If your human customer support agent makes stuff up and that got your business a penalty fine, you might discipline or fire them; not so easy when it's an AI that replaced a whole call centre in one go, even when the incident frequency is the same.

re: determinism

https://github.com/EricaCoppolillo/LLASP

re: determinism

https://github.com/EricaCoppolillo/LLASP

Interesting thought. So maybe this answers the question of whether coding is art or science.

Well it remains both I think.

The science part is that the outcomes are scientifically verifiable, hence tests.

The art part is that unlike the physical world with the laws of physics, there are nearly limitless ways in code to accomplish those outcomes.

Reply on news.ycombinator.com