Comment by Terr_ | Hacker News Reader

Comment by Terr_ | Hacker News Reader

Parent Full thread

> How it is made should be irrelevant in the process.

Is that true? Almost all answers have value based on how they were reached, especially when (like this case) there's no trivial and objective verification step.

For example, suppose you live next to a volcano, and I sell you some software that predicts when it will erupt. One day you dig into it and realize I just hard-coded "not today" [0] which is 99.999% accurate.

I think you would be quite angry, and justifiably so... But why would that be if "not today" was correct, and "how it's made is irrelevant to the process?"

Tying it back to the court case, we expect professional witnesses to look at the details and then reach a conclusion, and to reach out based their own professional expert knowledge of cause and effect... Not to pick a conclusion and try to make the details fit, nor to offload executive function to an LLM. Even if they render the same boolean verdict, we care about what process was taken.

[0] https://xkcd.com/221/

“How” as in “what tool” is a different “how” as in “with what intent”. LLM was used to produce a bunch of arguments supporting a pre-intended outcome. Doesn’t matter much whether it was a machine or a human (and whether it was “expert” himself or a ghostwriter), but it does matter - a lot - how the outcome was the pre-selected and the task was to cherry-pick the facts to support it.

> LLM was used to produce a bunch of arguments [...] Doesn’t matter much whether it was a machine or a human (and whether it was “expert” himself or a ghostwriter)

Even ignoring the intent-issue, the process used matters because we don't just care about whether each argument is individually true, we care about which arguments and claims appear at all. Do we trust that the LLM-brain is just as good at bringing up relevant issues as the brain of a human (real) expert?

Like the blind men and the elephant [0]: They're not wrong that there one part is like a rope, and one part is like a spear, and one part is like a fan, etc... But we'd vastly prefer "this is a large land mammal".

[0] https://en.wikipedia.org/wiki/Blind_men_and_an_elephant

Totally agree on your example:

> One day you dig into it and realize I just hard-coded "not today" [0] which is 99.999% accurate.

It only becomes visible when you do the research.

The justice process in this example just has to check and verify the inputs. There is no way around it. Today it is this expert, tomorrow another one.

When someone is caught that's an ethical issue and there could be more structural consequences as well. Like blocking the person as an expert or other means. As the expert becomes untrustworthy.

Reply on news.ycombinator.com