It baffles me how many people are using ChatGPT to get answers: it's a “language model” folks, not a “knowledge model”.
I’m personally open to that being the case with some LLM someday, but there’s a lot of people ideologically and financially invested in that perception being accurate today and applying to ChatGPT in particular.
People hate it when you point out that they’ve confused the modern equivalent of a Speak and Spell with The Overmind.
Yes, ChatGPT doesn't know what it's talking about. But Google is so bad these days, that ChatGPT in many circumstances can do much better even while being far from perfect.
Can you give some examples of how ChatGPT “does better” than a human searching for something and then using their own cognition to make sense of it?
If Google was an LLM then I’d agree with you, but the idea that googling something somehow skips the step of using critical thinking to understand what is on your screen seems pretty new to me.
…even if it has to invent people, places, things or events to satisfy your focus.
Do you have a particular example where ChatGPT does better than googling something? I’m especially interested in your example since we agree that googling entails using humans cognitive ability to make sense of things.
I do not argue that you don’t seem to get any value out of using ChatGPT, I’m curious about the big difference in quality between the knowledge you get from the ChatGPT page vs what knowledge you can get by googling something.
I'll give you a concrete example where it does better than search: my last Google search was about how to zip a whole folder in the Linux terminal (I must have searched for that dozens of times in my life, but I don't do it very often so it doesn't stick).
I saw the results, and I noticed that they were mostly transcriptions, summaries or extensions of man pages listing all the options, which would take a few minutes to skim through and look for the relevant options (which is what I wanted to avoid by Googling, otherwise I would have used man in the first place).
So I switched to the ChatGPT tab, asked, got the exact command I wanted, done.
This is different from asking it for an explanation or definition of something and not cross-checking every "fact".
While I would definitely, definitely think critically before running a command from the internet, I can see how that use case makes sense for ChatGPT if you’re willing to roll that way.
>…take a few minutes to skim through and look for the relevant options (which is what I wanted to avoid by Googling…
Ah, I see! We are using the phrase “googling” differently here. I explicitly mean googling as a verb that involves reading and parsing information, not the act of avoiding reading or parsing information.
I wouldn't count that as critical thinking, although I guess it depends on how you define the term.
In order to make things friendlier towards search, nigh any answer you find will be drowned in a sea of junk. Eg, Googling sends me to: https://phoenixnap.com/kb/how-to-zip-a-file-in-linux
It does have an answer, around the middle of the page.
https://i.imgur.com/jVEyCZe.png (useful answer on the bottom)
Here's what I actually want:
https://i.imgur.com/tXOEIIX.png
What I actually want most of the time is the closest equivalent to "Hey Bob, what's the command to zip up a folder on Linux?"
Of course I only use relevant keywords in my search. Adding extra words like "how to" will add results containing those words, which tends to mean more SEO spam.
Wouldn’t you have had an identical experience copy and pasting from Google as you did ChatGPT? Did you recognize the correct command immediately in ChatGPT but have difficulty recognizing that same command on Google’s page?
Certainly! ChatGPT outperforms human search and cognitive processing in several ways. For instance, when you require a quick, step-by-step tutorial to fix an issue in an application or program, ChatGPT excels. Instead of searching for an online tutorial yourself, ChatGPT provides you with immediate, on-the-fly instructions. This saves you from sifting through search results and having to verify the quality of each tutorial before attempting them. ChatGPT's instructions are already evaluated based on probability, eliminating the need for time-consuming quality checks. By receiving a concise, step-by-step list, you can swiftly proceed with your task, making it significantly more efficient than relying solely on manual searching and critical evaluation.
A task in this case could be finding out where a particular Vermeer is hung, or which viscosity of oil to put into your car’s engine — a specific concrete task in which ChatGPT provides you with knowledge better than a web search does.
I won’t argue that ChatGPT is better at writing lists than Google is. Google isn’t a website that writes lists so that comparison would be pointless.
Technotroll: How do I enable editing of word document?
ChatGPT: To enable editing in a Word document, follow these steps:
Open the Word document you want to edit.
If the document is in "Read-Only" mode, you'll need to enable editing. Look for a yellow or red banner at the top of the document that says "Protected View" or "Read-Only." Click on the banner, and you should see an "Enable Editing" button. Click on it to enable editing mode.
If the document doesn't have a "Protected View" or "Read-Only" banner, it may already be in editing mode. In that case, you can start editing right away.
If you are using Microsoft Word, you can also check the top-right corner of the window. If you see a button that says "Editing Restricted," it means the document has restricted editing. Click on the button and select "Stop Protection" to enable full editing.
In some cases, the document may be password-protected, preventing any edits without the password. If you're prompted to enter a password when trying to edit, you'll need to obtain the password from the document's owner or author.
By following these steps, you should be able to enable editing and make changes to your Word document. Remember to save your changes regularly to ensure your work is preserved.
Technotroll: I clicked the x on the yellow banner. Now what?
ChatGPT: If you accidentally clicked the "x" on the yellow banner in Microsoft Word, which appears when a document is in "Protected View" or "Read-Only" mode, you can still enable editing by following these steps:
Click on the "File" tab in the top-left corner of the Word window. This will take you to the "Backstage" view.
In the "Backstage" view, you should see an "Enable Editing" button on the right-hand side. Click on it to enable editing mode.
Alternatively, you can press the "Esc" key on your keyboard to exit the "Protected View" and enable editing.
If you closed the yellow banner by mistake, don't worry. You can still enable editing by following the steps above. Remember to save your changes regularly to ensure your work is preserved.
Conclusion: It worked almost immediately. If I were to Google it, I'd have to verify multiple documents before I got to the answer.
https://serpapi.com/playground?q=How+to+enable+editing+of+a+...
It looks like Google’s output is nearly identical information as what ChatGPT gave you. If you chose to trust ChatGPT’s output simply because it was more confident sounding, does that actually qualify as the software doing better?
However, if you do not get a quick guide, or if you're not satisfied with it for whatever reason (perhaps it doesn't work, which has happened to me multiple times), then you'd be forced to browse documents either way. This takes time and energy which I'm not prepared to spend. That has very little to do with how “confident” the information sounds, it has to do with the likelihood that I will expend energy on bs tasks when I can just get the answer right away. Google simply doesn't offer that.
FastGPT[0] is some way towards that: it uses the web to provide answers
Bing is okay, ChatGPT 4 with browe tool is okay and Bard is pretty good, but tends to not use its browse tool enough, and hallucinates mightily.
Good to know ;)
Have you tried Phind?^[1] The project explores exactly how an AI-driven search tool might look like.
The thing that really makes it shine, is that it doesn't only make claims per text, but that it also provides source links that it has already scraped for info.
Not because ChatGPT is an infallible oracle, far from it. It's just that the bar is rather low. Google has become really bad in the last decade or so.
Getting answers out of it is absolutely a reasonable thing to do. Blindly trusting those answers without verification, that's another thing entirely.
There's nothing wrong in using LLMs.
> Getting answers out of it is absolutely a reasonable thing to do. Blindly trusting those answers without verification, that's another thing entirely.
We're basically saying the same thing but with different words: in my wordings, if you don't trust the answers and double check later, it's not “answers” it's merely “hints” or “suggestion”.
But here we have someone that just copy-pasted the response as if they were quoting a source.
It may still be a very useful tool, but not when used this way.
There are many categories and types of questions where you can have a good degree of confidence in the answer. This is especially true for questions where a rough approximation is acceptable.
Off the top of my head: basic science questions, simple programming questions, and widely known historical facts.
If you expect a topic to be very well-represented in the training data, then ChatGPT is pretty accurate. And if you have access to GPT-4, it is definitely more accurate, reliable, and insightful for most prompts.
I prefer using it on questions like this for a few reasons. I find the experience to be so much smoother and more pleasant. The ChatGPT UI is incredibly minimal and clean. And then it allows asking follow-up questions, which is very useful.
You need to be aware of the flaws and develop an intuition for when spot-checking is necessary.
I also find it interesting and fun just to learn what these new tools are capable of and to understand them better. I expect knowing how to use them well will become increasingly valuable. Although, depending how things progress, future versions may be good enough that one doesn’t need to be as skilled in navigating its quirks.
It is, because it can give you correct answers as well as entirely stupid ones (like the infamous “horse eggs” example of a few month ago).
> basic science questions,
For math it's pretty much hit or miss, even for straightforward stuff (and I'm not talking about doing calculation)
> and widely known historical facts.
This is really something you should not use ChatGPT for, because it knows urban legends much more than it does know actual facts (for obvious reasons, because these are repeated as facts everywhere)
> You need to be aware of the flaws and develop an intuition for when spot-checking is necessary.
Every time you don't already know the answer beforehand, that's the intuition you should have. And keep in mind that cross-checking is a much more difficult task that looking up, because we humans are prone to anchoring bias and confirmation bias.
Want to use ChatGPT to learn something? Exploit its ability in understanding natural languages in order to get the actual name of the thing you describe (this is a pure language task, and ChatGPT is an enormous asset in that regard), and then search this.
I think you overstate the frequency of which it hallucinates for simpler questions. I am usually using GPT-4, which is more reliable. For the types of questions I’m talking about, I’d say GPT-4 is at least 90% accurate.
Often times, I’m just looking something up due to curiosity while discussing with friends. It’s replacing a google search. The failure mode is that there is small chance I’m passing along incorrect information to my friends. Not a big deal.
If 90% accuracy is not good enough or if the question is not simple, then I’ll take more care.
Perhaps we are also getting into a semantic debates about what “trustiworthiness” means…
If it outputted "I'm sorry, I don't know how to do that" for every single question, you could consider it 100% trustworthy, even though it would be useless. As it is right now, you can expect it to output correct answers with some confidence, but you absolutely can't trust that a given answer is correct until verified (you should in fact outright assume that it's not until proven otherwise).
The same is true of Google search results and Stack Overflow answers and academic journal articles and so on. No source of information can be entirely trusted and they all require verification.
It still read more knowledge than I will ever be.
And I also often enough believe/ listen to humans who hallucinate.
Just look how many humans believe in a god and a book some people wrote somehow, and I still take some of those people seriously.
I think where it differs for me is humans make stuff up for a different reason, humans invent knowledge to fill gaps in understanding, where invention of knowledge by an LLM is a side effect of it's attempt to complete some text.
It kind of is, though. The field of philosophy called epistemology deals with what knowledge is and how it is obtained, and the field overwhelmingly agrees that it is necessary (but not sufficient) for knowledge to be a (1) belief that is (2) true, and (3) justified. There is some disagreement when it comes to justification, and a lot of nuance in whether every justified true belief is knowledge, but pretty much no disagreement that knowledge must be a thing that is true and involve the mental state of believing that the thing is true.
Language doesn't imply belief (1), and it doesn't imply justification (2), and it certainly doesn't imply truth (3). Language is a medium in which knowledge can be expressed, but not knowledge itself (one can of course have knowledge about language, but that is not the same thing). Language is also a medium in which things that aren't knowledge can be expressed, such as falsehoods, nonsense statements, paradoxes, etc. LLMs generate language without belief, they sometimes do generate statements that are true and justified, but definitely not always.
I think the terms "data" or "information" are better than "knowledge" for what LLMs are trained on and produce.
(1) "The sky is green." Lying in general, are ways to use language to express something that isn't a belief. Also imperatives like "Please take out the garbage" or questions like "Did you take out the garbage?" don't seem to express a belief at all
(2) "There is a planet somewhere in the universe where s'mores naturally occur without intentional assembly by intelligent beings." This might be true, and I might believe it, but there is no justification for it. Still expressed in language. Questions and imperatives come up here too.
(3) See (1), or any other case of lying or being mistaken.
The example I’ve heard of knowledge that is believed, untrue but justified is “porcupines can shoot their quills”.
A tribe that believes this fact is more likely to stay away from porcupines, and even though porcupines do not in fact shoot their quills, over time this superstition is beneficial to the tribe as their neighbors are more likely to get hurt or killed by dangerous porcupine quills.
Judaism taught long before modern germ theory revealed why it was beneficial in a purely rational sense that burying feces and ritualistic hand washing were important to please God.
How do you know if something's true, though?
If knowledge of truth has to be true, how do you know that the true knowledge of truth really is true? Etc..
It doesn't challenge the idea that both a religious believer and non-believer have a certain knowledge of the world. These kinds of conflicts of knowledge are common, not just in religion, there is no single understanding of the world to reference against.