Hacker News Reader

Comment by LeoPanthera | Hacker News Reader

I've been impressed with Bing Chat, surprisingly so given how much negative talk there is about it.

For quick and simple fact checks, for which I would normally reflexively hit Google, it's a huge improvement. No need to be exposed to clickbait, scams, or excessively ad-heavy results.

Right now it requires you to use either the Edge browser (I don't want to switch), or the Bing app, which I reluctantly do. If they ever make it available to other browsers I can see my Google usage falling dramatically.

View on HN

I have to check Bing's sources when I use it anyway, because what it says is often simply wrong.

I asked it what my local cafeteria had on its lunch menu today. It answered with full confidence.

It turns out, it was completely wrong though. It had mixed the lunch options and completely hallucinated another one.

Stuff like this just makes me really wary using it. I have to fact check everything every time and for more complicated things I cant be sure if *I* am correct.

You can't get correct answers from an LLM for most things you'd want to ask it.

It's useful for predicting the future with some level of error that's probably better than you could do on some topic you know little about - and for generating text in the style of someone else about some subject where accuracy doesn't matter.

That's pretty much it.

Until LLMs work different, Chat-GPT - even if it gets to version 9000 - is never going to be able to tell you what's on the menu today at Chez Panisse, unless they build in some API for Chez Panisse to answer that query direction - in which case, you're not really using AI at all...

This assertion is easily disproven by about five minutes of using even GPT3, and more quantitatively, by 4's documented results on standardized testing. I'm not sure what kinds of questions you are asking in order to make broad statements like "You can't get correct answers from an LLM for most things you'd want to ask it", but this is so far off base from both the research, the documentation, and my own experience that I think we're talking about two different things.

Please stop with the middlebrow dismissal, doubly so when the dismissals aren't even accurate.

If you need correct information - it's not really useful at all if something is mostly correct 95% of the time.

If you don't need correct information, ChatGPT is great.

And what is the accuracy rate of the average Google search? Are you applying this extreme level of skepticism to everything you search for or only to LLM output?

I've seen this pattern enough times here that it's actually becoming infuriating for how bad faith it is. Look, we both know that there is a gradient on how people use the information they receive. On one side of that scale is how you claim LLM's work, bullshit generators that are wrong so often they are not useful, and so regularly everything you read is presumed bullshit – except applied to everything. On the other side of that scale is homo credulus, a fictional sub species of human that blithely accepts anything they are told without checking it against anything, be it common sense, their own working model of the world, other information, anything. They just take it and run with it.

Neither of these approaches are useful and neither of them match reality.

I am asking, begging you even, to knock it off already. The hyperbole you are spouting is not useful and it is demonstrably not correct.

> And what is the accuracy rate of the average Google search?

Probably more than 99%, certainly more than ChatGPT. Haven't used GPT-4 though so maybe that even the gap.

So the more than 99% comes exactly where from? Some gut feeling?

Oh boy how i wish the information i find on Google, Wikipedia, etc. were mostly correct 95% of the time. 95% is actually a fantastic goal to strive for to gain something useful from your own research. Only a fool would assume to have 100% correct information from a quick search.

ChatGPT isn't even at 70%.

I think their point is that GPT is less of a search engine replacement and more of a reddit/Quora replacement.

You wouldn't use reddit to ask for straightforward facts that are easily referenced from an official source, if they're important, because you'd have to verify any answers against the official source for accuracy anyway. You would use it for more open-ended questions/prompts, and then you would keep a critical eye out for inaccurate information and misinformation/trolling.

> Oh boy how i wish the information i find on Google, Wikipedia, etc. were mostly correct 95% of the time

For me is more like 99%, if I search for something and I find the answer, it is correct. Sometimes I just don't find what I am looking for, and this is the 1%.

Using ChatGPT is like using "I'm feeling lucky" feature in Google, but you are only allowed to use it once, and you are stuck with the result you got. You NEVER know if what GPT produced is true or not, and any fact requires double check.

I tried to use GPT-3 as google for some quick searches but I stopped because using standard search was much more efficient at the end of the day.

> for me

I question the objectivity of these percentages

Oh boy, have you tried Googling for stock prices?

Obviously, the prices aren't completely real time. Almost no one has that.

But ChatGPT is just going to be constantly wrong.

The cases of this are endless.

ChatGPT is good at writing you a song in the style of Shakira. It's not good at accurately describing current facts - because it doesn't have them.

Google invented LLMs long before ChatGPT - and never really added them to Search - because they just aren't that useful for the things people search for.

People will start searching for generative stuff. That's a new market.

People aren't going to ask ChatGPT what's the best couch to buy for their living room - because there's a good chance ChatGPT is going to make up a couch that doesn't even exist!

Hm, I'm not entirely sure if you're being sarcastic or not, but just in case, asking Bing about the menu I get this:

> Chez Panisse is a famous restaurant in Berkeley, California that serves seasonal and organic food1. The menu changes daily and is posted on their website2. Today’s menu for the restaurant (not the cafe) is:

> Fennel and leek salad with rocket, toasted almonds, and salsa verde > Bomba rice cooked with clams and squid; with aïoli > Becker Lane Farm pork loin roasted with Spanish paprika and green garlic; with > braised greens and wild mushrooms > Meyer lemon sherbet with candied kumquats > The price for this menu is $175 per person2.

That seems to be correct.

Where do you see "Meyer lemon sherbet with candied kumquats" on the menu?

https://www.chezpanisse.com/1/restaurantmenu/

This is what's there for today:

- Fennel and leek salad with rocket, toasted almonds, and salsa verde

- Bomba rice cooked with clams and squid; with aïoli

- Becker Lane Farm pork loin roasted with Spanish paprika and green garlic; with braised greens and wild mushrooms

- Blood orange and vanilla ice creams with Page mandarins

Bing chat searches an index of the web, then takes that information as context to answer your questions. You don't need a custom API if your machines can read human text accurately.

> I have to check Bing's sources when I use it anyway, because what it says is often simply wrong.

But at least it gives you the sources, unlike ChatGPT. (And, at least in my experience so far, it is not "often" wrong all. I've had good results.)

Yes and I like that Bing gives it's sources.

But I also find it rather fatiguing to have to check the sources every time.

I also don't know for sure if the times it gave me correct answer were actually correct or if I simply didn't catch the mistake?

I have heard this type of reply to this remark (from my side) a few times now. It has made me curious: Are you a type of person that often checks the sources on Wikipedia?

Anecdotally: I know that Wikipedia is not always correct. But I feel like I can build an intuition and reason on what pages I can reasonably trust on Wikipedia, since in my experience, the inaccurate bits I have encountered tend to be in certain categories. However it's much harder to feel confident about my intuition about ChatGPTs' correctness, since my exposure has led me to believe that the hallucinations are fairly random, and not concentrated in particular topics. This makes the tool much less attractive for me, as I feel like I need to double check every written word.

Perhaps I should be less trustful of Wikipedia...

> Are you a type of person that often checks the sources on Wikipedia?

Well, as a semi-regular editor on Wikipedia, I'm probably the wrong person to ask.

when the claims on Wikipedia are outlandish.. yeah i check the source.

> I have to check Bing's sources when I use it anyway, because what it says is often simply wrong.

Checking sources is a great habit to develop!

There's a Chrome extension to trick Bing into believing you're browsing via Edge. Works well enough for me: https://chrome.google.com/webstore/detail/bing-chat-unblocke...

there's also plenty of firefox extensions to switch your useragent

For the lazy, here's the user agent I use:

Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/111.0.0.0 Safari/537.36 Edg/110.0.1587.69

I've used Google today for a quick product comparison search and it's awful. The first three results were sponsored ads and the next pages are just low-quality seo rigged content that didn't answer my question.

Having Bing chat attached to bing doesn’t exactly solve this. It still constantly pulls from low quality sources. I asked a question about Satan and it cited Answers in Genesis. Product reviews regularly return seo garbage.

Product review categories in particular would benefit from whitelisting, by hand, things like americas text kitchen, consumer reports, rtings.

I get the other bits, but what's the issue with citing Genesis? It doesn't use the names for Satan, but the serpent is commonly understood as the same entity. Are you looking for e.g. Job instead?

https://en.wikipedia.org/wiki/Answers_in_Genesis

https://answersingenesis.org/

Yeah, I see why that would be problematic. I misunderstood the original statement as citing the book of genesis itself, which seems a lot more reasonable.

It seems particularly bad lately. The pattern I've noticed is:

* A totally pointless introduction paragraph, devoid of info.

* Big ad

* A sort of teaser sentence in large font so that it appears to be the length of a paragraph. High noise to information ratio.

* Larger ad that loads as you scroll, so you're more likely to accidentally hit it

* Another sentence with high noise to info ratio.

* Repeat.

I think one of the SEO perks of this pattern is how it takes forever to find the information that you know must be somewhere on the website, so users seem more "engaged" because they are scrolling and spending more time visiting the site.

Also generates more ad impressions as you scroll and see these ads.

Looks like Google has turned into AltaVista.

You made my day with that comment!

not yet, you need to 'join in the waitlist' and be granted access to the 'new bing' - otherwise no access to the chat mode.

I wanted to check this point and asked Bing to suggest some good business laptops below $800. It suggested me a $2k dell xps, a $1.5k LG gram and some piece of shit chromebook. I guess we'll never have objective product comparisons free of seo bullshit.

>No need to be exposed to clickbait, scams, or excessively ad-heavy results.

Not yet. But it will be monetized eventually, of course. Most probably through ads. And, as we know very well, big tech corps simply are not able to do monetization in an ethical way.

Me too, though there's an uncanny valley hiding in the spectrum of responses from bing right now. That is, I'd ask it to summarize the difference in views from a novelist, a computer scientist, a media theorist, and a philosopher across more than a half decade and was shocked at how good the quick summary paragraphs were considering my question was something like "compare and contrast the beliefs and values of David Foster Wallace, Alan Kay, McLuhan, and Byung Chul-Han, respectively. Then I asked another question and bing went off the rails getting 3 paragraphs wrong and then going from English into Spanish for no apparent reason other than I presume its source material had parallel language issues. It was surprising how good bing's right answers were given how bad its wrong answers were, especially considering that it had caught the fact that while there's a lineage of ideas from early media critics to present, the tech had changed dramatically underneath those criticisms and bing was aware enough of that substrate to note it in a separate "meta" paragraph about that change of tech, for lack of a better term.

   > Right now it requires you to use either the Edge browser (I don't want to switch)

Or faking your user agent...

https://github.com/anaclumos/bing-chat-for-all-browsers

The ckickbait, scams and ad heavy results will return, don't worry. Microsoft is expert at poisoning their own successes. Just wait for the rampant commercialisation of results, and various efforts to infect the model from SEO types over the long term

The Edge browser is essentially is Chrome under the hood. With cosmetic changes and some Microsofty features. I switched to Edge and would never go back to chrome. Big reason is - the vertical tabs feature and tab grouping. At anytime I have around 300-400 tabs open and vertical tabs + grouping makes browsing through them a breeze and I always know where the tab is I’m looking for. Takes a few days to get used to vertical tabs, but once you do, you’ll question the decision of other browsers to have horizontal tabs where you can’t even read the title when many are open.

> No need to be exposed to clickbait, scams, or excessively ad-heavy results.

That's because it's new, like google search in the beginning. Wait until they monetize it

Change your browser's user agent to Edge.

Huh, this actually works! I never imagined it would be so simple. Thank you.

> Right now it requires you to use either the Edge browser

And we’re back to the Microsoft of the 90s apparently!

> No need to be exposed to clickbait, scams, or excessively ad-heavy results.

Just bot hallucinating, which is way worse than anything you mentioned, as there is now way to determine if GPT is telling you the truth or not without actual manual check in other sources.

Just curious, what browser are you using? I gave up Chrome a while ago with how bloated and invasive it has become. Edge is better at least and I think Firefox is the best choice if you want privacy. Brave and other stuff have some bad reputation...

I use Safari.

Edge and Chrome are two sides of the same coin.

And each side of the coin has different engravings. It is different enough for me to use it and it doesn't seem to be so bloated like chrome.

Not sure how true it is but Microsoft core business is not about ads and selling user data. Google has a lot more incentive to be invasive than Microsoft and it shows in their browser.

> Right now it requires you to use either the Edge browser (I don't want to switch)

More precisely it requires to see Edge in the user agent. Any browser that allows settings user-agent per site (I use Orion) allows you to use Bing chat.

You can also use it on Skype

Reply on news.ycombinator.com