Asking three LLMs a simple question
sethops1.net
sethops1.net
If I ask a factual question of AI it will issue some output. In order for me to check that output, which I am apparently bound to do in all cases, I must check reliable sources, perhaps several. But that is precisely the work I wanted to avoid by using AI. Ergo, the AI has increased my work load because I had the extra useless step of asking the AI. Obviously, I could have simply checked several reliable sources in the first place. I see this as the razor at work.
It ought to be clear now that the use of AI for factual questions entails that it be trustworthy; when you ask an AI a factual question, the work you are hoping to avoid is equal to the work of checking the AI output. Hence, no time can ever be saved by asking factual questions of an untrustworthy AI.
QED
P.S. This argument, and its extensions, occurred to me and my advisors 25 years ago. It caused me to conclude that building anything other than a near perfect AI is pointless, except as a research project to discover the path to a nearly perfect AI. Nearly perfect should be interpreted to be something like "as reliable as the brakes on your car" in terms of MTBF.
I forget who came up with the idea but we could create a database with functions for every use case with the idea to never have to write something already written but finding the one you are looking for (by conventional search) would take more time than writing from scratch.
AI just provides new angles to attack from. It could save time or take more time, bit of a gamble. Examine your cards before placing the bet.
How is a user with a question supposed to determine if the question is "good"? What should he do if he is not sure? Shouldn't an "intelligent" LLM be responsible enough to tell him if there is a problem?
Being required to only ask "good" questions defeats much of the utility that LLMs tout as being provided.
Your response has a strong vibe of an AI apologist.
The current offerings of OpenAI and Anthropic can be asked to support their claims by for example reaching out to the internet and citing reputable sources. That improves the answer quality for questions like this immensely and in any case they can be verified.
Also the question asked is spurious: It appears there never was a release date for this particular SKU given by Cisco. The whole series (Cisco 1000 Series Integrated Services Routers) was released on 06-OCT-2017.
https://www.cisco.com/c/en/us/support/routers/1000-series-in...
On Amazon available since Sep 2018:
https://www.amazon.de/-/en/C1101-4P-Integrated-Services-Ethe...
But is it the right model? Does the release date actually matter to anyone?
Work with the tool to get best results instead. You wouldn’t do csi style zoom enhance on a jpeg either.
They can and do make extensive use of web search, and since they're pretty good at summarizing structured and unstructured text, this actually works quite well in my experience.
They provide information --- some of which is random in nature and only casually reflective of any truth or reality.
And as this example illustrates, they are far from being trustworthy. Their main achievement is to consistently produce functionally acceptable grammar.
Is an answer that is correct by chance the same as one that is correct by reason?
An unreliable "answer" is really not an *answer* in the traditional sense of computing. It is merely information waiting to be verified.
No one has to verify every calculation in a spreadsheet. If they reasonably needed to do so, spreadsheets would be more like an LLM --- and a lot less useful.
If I ask an LLM “What is the capital of France?” and it answers “Paris.”, then it has provided an answer by any reasonable definition of the term.
This anti-AI weirdness where people play word games to deny what AI is clearly doing has to stop.
It answered "Mbabane" along with information about the fact that the country is now called Eswatini.
There was no mention of the fact that there are actually 2 capitals --- Mbabane (the administrative capital) and Lobamba which serves as the executive seat of government.
The point being --- any "answer" from an LLM is questionable. An unreliable or incomplete answer is information but it is really not an *answer* (in the traditional computing sense) if additional work is reasonably required to verify and prove it as such.
If that’s the point, then you should say that instead of saying that they don’t provide answers. They very clearly do provide answers and this weird rhetorical nonsense is grating.
If a human got the question wrong, would you conclude that “humans don’t provide answers”? Getting questions wrong is normal. It doesn’t mean that the entity class that got the questions wrong is incapable of giving answers, it just means they aren’t perfect.
In the realm of computing, it is not. This is why people use them.
People *expect* computers to provide quick and reliable answers. Or at least they used to --- before LLMs.
I ask an LLM “What is the capital of France?” and it answers “Paris.”
If you see that and say “LLMs don't provide answers.” then you have let your ideological opposition to AI overwhelm your reason and mislead you into saying very silly things that are obviously untrue, and you really need to reconsider your position so that you stop doing that.
You can say that they are unreliable all you want. You can still criticise LLMs! Just don’t get so twisted out of shape that you start speaking utter nonsense.