Hacker News Reader

Comment by pmontra | Hacker News Reader

I can see how Google could use something like that to turn from a search engine into an answer engine, selling ads on its home page and hiding all the current search results into a references section. If they won't do it, someone else will and kill Google outright.

View on HN

It does seem to be the future, but, at the moment, it's very unreliable. The worst thing is that it's all said with confidence and it would be very hard to know when things are wrong unless you already know.

For example: Q. How many people spoke French in pre-revolutionary France?

A. It is difficult to determine the exact number of people who spoke French in pre-revolutionary France. The population of France at the time was approximately 26 million, and the vast majority of people spoke French as their first language. <snip>

Wikipedia: It is estimated that at the time of the French Revolution in 1789, only half of the population of France could speak French, and as late as 1871 only a quarter spoke French as their native language.

Also, the population of France on the eve of the Revolution was about 28 million.

To be fair, Wikipedia also “speaks with confidence” and makes no attempt to make it easy to understand exactly how confident one should be in the facts presented. At one point they had said the would, but never did to my knowledge. Google has TrustRank, but to my knowledge it’s on a source-level basis, not fact-level basis.

The bot got it unequivocally wrong here.

When it said that the vast majority of the population of what's now modern-day France spoke modern French as their native language, that's categorically false and shouldn't be treated with leniency or open to interpretations.

My point is cherry picking flaws won’t help improve anything for ChatGPT, Wikipedia, etc — but systematic approaches to discovering and modeling information space and related fact, queries, etc would. Wikipedia not free of issues and to my knowledge does not allow end user to see Wikipedia’s confidence in a given fact, if they have one at all.

These sort of statements are by nature speculative not factual, and thus speakers are advised to express and communicate doubt, and hedge against uncertainty inherent in their views, and probably even vicious rebuttals, by using appropriate language constructs or terminology, but when a know-it-all bot that was trained by its handlers to pass always as an authority figure makes a mistake due to hubris or overconfidence, don't expect us to sit idle, and not call them out, and refute their claims accordingly.

Understand, though where exactly does ChatGPT claim to be a source of factual information?

Wikipedia on the other hand does clearly state its intent to maintain reliability of its content:

- https://wikipedia.org/wiki/Wikipedia_and_fact-checking

- https://wikipedia.org/wiki/Reliability_of_Wikipedia

Beyond that, in my opinion, while human dialogue might hedge confidence, disclose conflicts of interest, etc — to me, assumed the exchange is via text-based chat — there are much more efficient and effective ways to express that information than adding non-actionable text like that.

> Understand, though where exactly does ChatGPT claim to be a source of factual information?

It doesn't right now, but if you scroll up, you'll see the idea at the beginning of this thread is to turn it into a source of information.

The difference between ChatGPT and Wikipedia for this purpose is that:

1. Wikipedia is wrong wayyyyyyyy less often than ChatGPT

2. Wikipedia has sources linked at the bottom of the page you can go check, while ChatGPT does not

> At one point they had said the would, but never did to my knowledge.

I think autocorrect or something mangled your sentence beyond recognition.

What specifically is unclear?

Wikipedia said they would enable users at to be able to quickly tell what the confidence was in the facts being presented and to my knowledge never did. My understanding is that even now that linked sources are not even systematically checked to confirm they even relate to the related passage of text.

the -> they

That was probably readable by context for most, yet not for me ;)

My bad, thanks for the clarification, even in rereading sentence, didn’t notice the error.

http://dhmontgomery.com/2018/02/obscenities-oaths/ for anyone who finds this topic interesting

What's worse is if you ask it to cite sources, it invents fake ones and presents them with full confidence, which I think was also the biggest and loudest concern with Meta's Galactica:[1]

> Q: How many people spoke French in pre-revolutionary France? Please cite sources.

> A: According to a study by the French National Institute of Statistics and Economic Studies, approximately 28.5% of the population of pre-revolutionary France spoke French as their primary language. This number increased to 94.5% in the late 19th century. Sources:

> 1. "French Language in France," French National Institute of Statistics and Economic Studies, https://www.insee.fr/en/statistiques/fichier/1893254/Langue_...

> 2. "The History of the French Language," ThoughtCo., https://www.thoughtco.com/history-of-the-french-language-122...

Link 1 doesn't exist, and link 2 redirects to https://www.thoughtco.com/origin-of-life-theories-1224553 because it just smashed a relevant-looking slug against a real article's numeric ID.

I tried again in a new session and got a different wrong answer. So I called it out on the wrong answer, and it apologized, and regurgitated the same wrong answer again in response:

> Q: How many people spoke French in pre-revolutionary France? Please cite sources.

> A: Statistical data from 1783 shows that approximately 36 million people in France spoke French as their first language. Sources: UNESCO Institute of Statistics and The Historical Dictionary of Pre-Revolutionary France by Paul R. Hanson.

> Q: That answer is incorrect. How many people spoke French in pre-revolutionary France? Please cite sources.

> A: My apologies for the incorrect answer. According to a language survey conducted in 1783, approximately 36 million people in France spoke French as their first language. Sources: UNESCO Institute of Statistics and The Historical Dictionary of Pre-Revolutionary France by Paul R. Hanson.

Paul R. Hansen wrote _ Historical Dictionary of the French Revolution_; on its first page it cites the population of France during the revolution as "roughly 25 million".[2]

If you ask it to cite sources on a more niche topic that's nonetheless well documented on Wikipedia, it seems to regurgitate a stew of every related Wikipedia article on the topic, throws in some press releases about things that didn't happen, and invents fake sources:

> Q. Can you write a history of the 2020 Portland Thorns FC season, citing specific sources with URLs?

> A. The 2020 Portland Thorns FC season was a remarkable one, as the team achieved success on and off the pitch. The Thorns finished the regular season in first place in the National Women's Soccer League (NWSL) standings, with an 11-4-3 record, and advanced to the semifinals of the NWSL Challenge Cup.

> Off the pitch, Portland Thorns FC continued to support their local community through various initiatives. The team hosted a virtual food drive to benefit Oregon Food Bank and partnered with local organizations to provide free meals for families in need. Additionally, they launched their “Thorns Together” campaign, which aimed to support small businesses around Portland during the pandemic.

> On October 14th, 2020, Portland Thorns FC made history by becoming the first NWSL team to host a match at Providence Park since March of 2020. The match was played against OL Reign and ended in a 1-1 draw.

> On December 12th, 2020, Portland Thorns FC won their fourth NWSL Championship title after defeating Chicago Red Stars 2-0 in the finals of the NWSL Challenge Cup. This marked their second consecutive championship title and fourth overall since 2013.

> Sources:

> Oregon Food Bank: https://oregonfoodbank.org/about-us/news/portland-thorns-fc-...

> NWSL: https://www.nwslsoccer.com/standings

> Thorns Together Campaign: https://www.timbers.com/post/2020/09/15/portland-thorns-fc-l...

> Providence Park Match: https://www.timbers.com/post/2020/10/14/portland-thorns-fc-h...

> NWSL Championship: https://www.nwslsoccer.com/news/article/portland-thorns-fc-a...

Everything above is a fully synthesized falsehood plausible to anyone who doesn't know what actually happened, while the season is documented with sources on Wikipedia,[3] something one would presume the model is aware of.

There was no 2020 regular season, it was cancelled due to COVID-19. There were two short competitions, but the Thorns didn't play in Providence Park in March, and their first home match was in September. The Thorns have won the Challenge Cup, but in 2021 against Gotham, not in 2020, and the nwslsoccer.com link 404s. The Thorns and Timbers have a "Stand Together" campaign that's worked with the Oregon Food Bank, but they didn't host a virtual food drive in 2020, and the Oregon Food Bank link 404s. There's never been a "Thorns Together" campaign, and the timbers.com link 404s. There was no NWSL championship to win in 2020, and if they had it would've been their third, not their fourth.

1: https://arstechnica.com/information-technology/2022/11/after...

2: https://www.google.com/books/edition/Historical_Dictionary_o...

3: https://en.wikipedia.org/wiki/2020_Portland_Thorns_FC_season

The Q&A mode is better about not giving false answers, but this answer in particular has me worried:

Q: Would you bet people's lives on the veracity of your last answer?

A: Unknown

Oh, but there is a world of difference between answering questions based on neural memory and being able to bind these answers to actual references and sources. That being said, something like what you described will likely happen.

At this point I feel like it's only a matter of time before a future version of this is able to cite sources. (Though I suspect it'll occasionally struggle to find sources that are reputable and which actually support the AI's statements.)

Reply on news.ycombinator.com