This summarizes I think a lot of the challenges with validating LLM output. We hear “humans make mistakes too”, but I would agree with you that our human detection of human-made mistakes and LLM-made mistakes is unlikely to have the same coverage.
I'd rather have a human on my team for whom I can reasonably surmise what tasks they're good at than have a robot who randomly gets shit wrong.
At least in math problems, AI mistakes generally occur more frequently given the difficulty of the task.
The carwash problem is very undespecified. The question should be something like "I am American. I live in a single family home in a suburb. Each member of my family has their own car. All the cars are parked during the night at home. My boss don't authorize me to go during working hour outside the office. I want to wash my car. The car wash is 50 meters away. Should I walk or drive?"
Disclaimer: I am Argentinean. I live in an old apartment building without a parking lot. I don't have a car, but if I had one I'd probably move to a building with parking inside or nearby. The closest professional car washing facility is like 1000 meters away, but the closest parking lot is like 50 meters away and they guys would wash the car for a few bucks. I want to wash my car. The car wash is 50 meters away. Should I walk or drive?
(I don't speak Japanese, I used autotranslation, so it may be wrong.)
There are no "r" in the token of strawberry.
Say you have two assistant employees. One that reads the reports and one that uses AI to summarize the content. You are sitting in a high value meeting. You only can have one assistant with you in the meeting. Which one would you pick, the persons that read the reports or the person who only used AI?
I would take the person who only used AI.
Why?
Your fictional problem has the silent assumption that if you are in a high value meeting, you shall act more conservatively (i.e. reduce risk for reliability, even if you loose chances/opportunities). But this is rather a statement that the respective person acts on a risk-averse profile.
I rather have a risk-affine profile. Because it is likely that everybody else in the meeting has a risk-averse profile, it is very likely that if I act on the "non-conservative" choice and I am right, I will have a huge advantage. On the other hand, if I act on the conservative choice, I won't have any advantage over the other participants.
Since I am risk-affine person, I am willing to risk it. :-)
There was no statement if the meeting was scheduled or impromptu. Client(s) might have saw you in public or at an event. Or at a trade show and there the report only applies to the single client versus to many clients.
There was no statement of report ownership. All parties might have access to it.
There is a chance that AI lied / hallucinated the summary. Only to know would be taking time to read the report and validate it.
AI only consumption has shown to reduce retention of information compared to reading the core content.
Ever have someone tell you why they want AI? I have, it was because they do not want to read the reports.