I asked it what my local cafeteria had on its lunch menu today. It answered with full confidence.
It turns out, it was completely wrong though. It had mixed the lunch options and completely hallucinated another one.
Stuff like this just makes me really wary using it. I have to fact check everything every time and for more complicated things I cant be sure if *I* am correct.
It's useful for predicting the future with some level of error that's probably better than you could do on some topic you know little about - and for generating text in the style of someone else about some subject where accuracy doesn't matter.
That's pretty much it.
Until LLMs work different, Chat-GPT - even if it gets to version 9000 - is never going to be able to tell you what's on the menu today at Chez Panisse, unless they build in some API for Chez Panisse to answer that query direction - in which case, you're not really using AI at all...
Please stop with the middlebrow dismissal, doubly so when the dismissals aren't even accurate.
If you don't need correct information, ChatGPT is great.
I've seen this pattern enough times here that it's actually becoming infuriating for how bad faith it is. Look, we both know that there is a gradient on how people use the information they receive. On one side of that scale is how you claim LLM's work, bullshit generators that are wrong so often they are not useful, and so regularly everything you read is presumed bullshit – except applied to everything. On the other side of that scale is homo credulus, a fictional sub species of human that blithely accepts anything they are told without checking it against anything, be it common sense, their own working model of the world, other information, anything. They just take it and run with it.
Neither of these approaches are useful and neither of them match reality.
I am asking, begging you even, to knock it off already. The hyperbole you are spouting is not useful and it is demonstrably not correct.
Probably more than 99%, certainly more than ChatGPT. Haven't used GPT-4 though so maybe that even the gap.
You wouldn't use reddit to ask for straightforward facts that are easily referenced from an official source, if they're important, because you'd have to verify any answers against the official source for accuracy anyway. You would use it for more open-ended questions/prompts, and then you would keep a critical eye out for inaccurate information and misinformation/trolling.
For me is more like 99%, if I search for something and I find the answer, it is correct. Sometimes I just don't find what I am looking for, and this is the 1%.
Using ChatGPT is like using "I'm feeling lucky" feature in Google, but you are only allowed to use it once, and you are stuck with the result you got. You NEVER know if what GPT produced is true or not, and any fact requires double check.
I tried to use GPT-3 as google for some quick searches but I stopped because using standard search was much more efficient at the end of the day.
I question the objectivity of these percentages
Obviously, the prices aren't completely real time. Almost no one has that.
But ChatGPT is just going to be constantly wrong.
The cases of this are endless.
ChatGPT is good at writing you a song in the style of Shakira. It's not good at accurately describing current facts - because it doesn't have them.
Google invented LLMs long before ChatGPT - and never really added them to Search - because they just aren't that useful for the things people search for.
People will start searching for generative stuff. That's a new market.
People aren't going to ask ChatGPT what's the best couch to buy for their living room - because there's a good chance ChatGPT is going to make up a couch that doesn't even exist!
> Chez Panisse is a famous restaurant in Berkeley, California that serves seasonal and organic food1. The menu changes daily and is posted on their website2. Today’s menu for the restaurant (not the cafe) is:
> Fennel and leek salad with rocket, toasted almonds, and salsa verde > Bomba rice cooked with clams and squid; with aïoli > Becker Lane Farm pork loin roasted with Spanish paprika and green garlic; with > braised greens and wild mushrooms > Meyer lemon sherbet with candied kumquats > The price for this menu is $175 per person2.
That seems to be correct.
https://www.chezpanisse.com/1/restaurantmenu/
This is what's there for today:
- Fennel and leek salad with rocket, toasted almonds, and salsa verde
- Bomba rice cooked with clams and squid; with aïoli
- Becker Lane Farm pork loin roasted with Spanish paprika and green garlic; with braised greens and wild mushrooms
- Blood orange and vanilla ice creams with Page mandarins
But at least it gives you the sources, unlike ChatGPT. (And, at least in my experience so far, it is not "often" wrong all. I've had good results.)
But I also find it rather fatiguing to have to check the sources every time.
I also don't know for sure if the times it gave me correct answer were actually correct or if I simply didn't catch the mistake?
Anecdotally: I know that Wikipedia is not always correct. But I feel like I can build an intuition and reason on what pages I can reasonably trust on Wikipedia, since in my experience, the inaccurate bits I have encountered tend to be in certain categories. However it's much harder to feel confident about my intuition about ChatGPTs' correctness, since my exposure has led me to believe that the hallucinations are fairly random, and not concentrated in particular topics. This makes the tool much less attractive for me, as I feel like I need to double check every written word.
Perhaps I should be less trustful of Wikipedia...
Well, as a semi-regular editor on Wikipedia, I'm probably the wrong person to ask.
Checking sources is a great habit to develop!