Hacker News Reader

Comment by Terr_ | Hacker News Reader

> Welcome to the era of generative AI, where a mountain can have multiple heights, but also only one height, and the balance of my bank account gets to determine which one that is. All invisible to the end user and then rationalised away as a coincidence.

I've always found the idea of untraceable, unfixable, unpredictable bugs in software... Offensive. Dirty. Unprofessional.

So the last couple years have been been disconcerting, as a non-trivial portion of people who I thought felt similarly started to overlook it in LLMs, while also integrating those LLMs into flows where the bad-output can't even be detected.

View on HN

As it turns out, correctness very often simply doesn't matter. Or not as much as one would intuitively think.

How many shops are there optimizing "business strategies" with data that's -essentially- garbage?

> How many shops are there optimizing "business strategies" with data that's -essentially- garbage?

How many of those shops are knowingly optimizing with garbage?

I'd argue that most of this data, which I would agree is garbage, is actually processed into seemingly good data through the complex and highly human process of self-deception and lies.

You don't tell the boss that the system you worked 2 month on is generating garbage, because then he'll replace your with someone who wouldn't tell him that. Instead you skirt evaluating it, even though you know better, and tell him that it's working fine. If the idiot chooses to do something stupid with your bad data, then that's his problem.

For that LLMs are good but I bet some people want to use it for things where correctness is vital.

In that case you use RAG and have it tell you the source.

A RAG needs to be implemented by the LLM provider. The simple end user has no idea what that means, even though he will be (incorrectly) using the LLM for a vital purpose.

ChatGPT does exactly that with its built in runtime and web search.

But the LLM provider doesn’t have to do that. Langchain - the Python AI library - and OpenAI’s own library has support for third party tools.

It’s up to third parties to build on up of it.

Reply on news.ycombinator.com