Hacker News Reader

Comment by lucfranken | Hacker News Reader

How it is made should be irrelevant in the process. There is no way to stop people using LLM’s or other tools to create products like this report.

It should be globally a standard check to verify incoming documents and to determine their validity.

The interesting part on this case is that we see the real human aspect of it. Just asking for a report only finding the positive facts instead of doing real research.

That is an ethics question. When behaving like this (and it happened before LLM’s) it should be seen as an ethics violation. If not done there is no value in those legal procedures anymore.

The facts people have to accept is that the only place to check this is the deliverable. They are lucky now to have found the source, the prompts and the background but so many reports are entered without being so obvious.

It’s a terribly huge and complicated task.

View on HN

> How it is made should be irrelevant in the process.

No? If you are employed (or requested in this case I guess) to offer your expert opinion, the expert they want to hear from is you, not someone else you farm your work out to - whether that’s an LLM or another person

Cynically, the purpose of the expert witness is someone to provide something which is treated as fact but isn't subject to penalties of perjury.

Interesting piece. Seems to be that the expert cannot falsify things but his opinion is an opinion. So there is no good/bad in that.

Not sure how it legally works, I think in the legal system at least in the US it is quite common that experts are hired and giving opinions based on who hires them?

And then the other party can cross examine? Not enough knowledge on that but it’s interesting if legal people could chime in on how it works.

I hope those expert “opinions” are not taken as facts?

How is this any different from any job? Or at least any knowledge worker job like an engineer or some sort? They hired you, so presumably they want you to do the work or solve the problem, not farm it out to another person or an LLM.

Does it matter how it was created? Whether you had a subordinate create it, or an employee, or a contractor, googled it, cited a study, or some other method. You as the expert look at the conclusion and either agree with it or disagree.

I think if people looked at LLM as just tools or interns, which you should always check or question their conclusions, then they'd 1) not believe everything LLMs tell you and 2) they would get better responses after critiquing the LLM response.

You're at work. You need to produce an artefact – a project proposal, say. 100-odd lines of Gantt, costs, that your BD team will send to the client.

You get Copilot to do this for you.

Later at the review meeting, errors are found. A bunch of stuff was missed, the costs were off.

Whose fault is it? Who looks stupid? If it's bad enough, who gets fired?

The biggest problem is not really that chatgpt was used but more than the conclusion is decided first and chatgpt is used to find the arguments to support it.

Normal, an expert should look at the facts and elements provided without a predecided result and forge his conviction based on the elements.

Obviously experts might be biased, especially when paid by the company, but it should still be in the understanding and evaluation of documents. Otherwise they are not an expert. They are not "lawyers" with the task to find incriminating or exonerating arguments.

Their testimony should start with something like "in my honest opinion". That is obviously not compatible with taking chatgpt to generate your testimony by asking to prepare arguments that support your customer.

> How it is made should be irrelevant in the process.

Is that true? Almost all answers have value based on how they were reached, especially when (like this case) there's no trivial and objective verification step.

For example, suppose you live next to a volcano, and I sell you some software that predicts when it will erupt. One day you dig into it and realize I just hard-coded "not today" [0] which is 99.999% accurate.

I think you would be quite angry, and justifiably so... But why would that be if "not today" was correct, and "how it's made is irrelevant to the process?"

Tying it back to the court case, we expect professional witnesses to look at the details and then reach a conclusion, and to reach out based their own professional expert knowledge of cause and effect... Not to pick a conclusion and try to make the details fit, nor to offload executive function to an LLM. Even if they render the same boolean verdict, we care about what process was taken.

[0] https://xkcd.com/221/

“How” as in “what tool” is a different “how” as in “with what intent”. LLM was used to produce a bunch of arguments supporting a pre-intended outcome. Doesn’t matter much whether it was a machine or a human (and whether it was “expert” himself or a ghostwriter), but it does matter - a lot - how the outcome was the pre-selected and the task was to cherry-pick the facts to support it.

> LLM was used to produce a bunch of arguments [...] Doesn’t matter much whether it was a machine or a human (and whether it was “expert” himself or a ghostwriter)

Even ignoring the intent-issue, the process used matters because we don't just care about whether each argument is individually true, we care about which arguments and claims appear at all. Do we trust that the LLM-brain is just as good at bringing up relevant issues as the brain of a human (real) expert?

Like the blind men and the elephant [0]: They're not wrong that there one part is like a rope, and one part is like a spear, and one part is like a fan, etc... But we'd vastly prefer "this is a large land mammal".

[0] https://en.wikipedia.org/wiki/Blind_men_and_an_elephant

Totally agree on your example:

> One day you dig into it and realize I just hard-coded "not today" [0] which is 99.999% accurate.

It only becomes visible when you do the research.

The justice process in this example just has to check and verify the inputs. There is no way around it. Today it is this expert, tomorrow another one.

When someone is caught that's an ethical issue and there could be more structural consequences as well. Like blocking the person as an expert or other means. As the expert becomes untrustworthy.

> It should be globally a standard check to verify incoming documents and to determine their validity.

Part of that process is bringing in expert witnesses and I don't think ChatGPT qualifies.

>It should be globally a standard check to verify incoming documents and to determine their validity.

And how do you do that? Do you ask... An expert?

Try to think about it.

Horribly complex challenge. Totally agree.

The problem is simple: We don't know which sources and tools are used. The experts submits a report, talks about it, answers questions. It is never a hard factual check on where he is coming from.

The intentions, generally based on the party who requested the expert, are many times challenged by the attorneys.

In basis it's a reputation issue, is this a trusted expert. But only that does in my opinion not suffice. How many "mistakes" do you accept before a non-compliant expert is caught?

So yes:

> And how do you do that? Do you ask... An expert?

That's what counter-expertise has to handle. Just one expert claiming something is a hard thing to trust.

It's the same thing that the lawyers in cases found that other lawyers were making claims based. on non-existing laws. Hallucinations by ChatGPT in that case. They did the world to verify. Found the issues. The lawyer gets fined (reputation damage).

There is no way around it, the work of verifying has to be done.

>How it is made should be irrelevant in the process

Bullshit. I dare any lawyer to say that to the face of a client paying 100s per hour for "services rendered."

You say you want respect but honey for what, exactly?

Reply on news.ycombinator.com