Comment by internetter | Hacker News Reader

Comment by internetter | Hacker News Reader

Parent Full thread

> Where is Wikipedia without all the learning and information from other sources, many of which it put out of business?

Which businesses did Wikipedia put out of business? You will frequently see a 5k word article used for a couple of sentences in a Wikipedia page, with the entire Wikipedia page itself being smaller than one paper it cites for one small corner of said page. When I’m researching events, I frequently go to Wikipedia to find sources as search engines have a drastically larger recentism bias.

> Has anyone researched the relative accuracy of Wikipedia and LLMs

No comparative research on this specific topic has been conducted afaik, and most comparative research is aging (likely, to Wikipedia’s own detriment–general consensus is that Wikipedia’s reliability has increased over time). However at the time of research publication, the consensus seems to be that Wikipedia is generally only slightly less reliable than peers in a given field (ie textbooks or británica), although Wikipedia is often less in depth. The most frequently cited study is a 2005 comparison in Nature which found 4 major errors in both Wikipedia and Británica, and 130 minor errors on Británica whereas 160 on Wikipedia. All studies are documented on Wikipedia itself, see [[Reliability of Wikipedia]]. LLMs… do not have this same reputation.

> Which businesses did Wikipedia put out of business?

Just as a start, other sources of reference, including encylopedias, dictionaries, websites, etc. For example, I'm sure it impacts McGraw-Hill's AccessScience, which likely you've never heard of.

> This is documented on Wikipedia itself

Maybe there's a little bias there? Would Wikipedia accept Wikipedia's analysis of its own reliability as a valid source?

I've heard that claim, but having no knowledge of the accuracy of any particular article, it's not worth very much to me.

> LLMs… do not have this same reputation.

They don't with you, but many people obviously use them that way. Also, reputation does not correlate strongly with reality.

> Just as a start, other sources of reference, including encylopedias, dictionaries

This just seems like healthy competition. I thought we were talking about a situation where Wikipedia’s use of other encyclopedias is an instrument of their demise.

> Maybe there's a little bias there

Paradoxically, I suspect you’d be pleasantly surprised about how tough this article is on itself. A lot of attention is given to bias in this case.

> Would Wikipedia accept Wikipedia's analysis of its own reliability as a valid source?

First, it is not Wikipedia’s own analysis. Editors should not present their own conclusions from research, just what each paper says. See [[WP:SYNTH]]. Second, generally Wikipedia discourages anyone citing it as it is not a stable source of information. Much better is to use the sources the article itself conveniently cites inline. As a general policy citing any encyclopedia is discouraged.

> having no knowledge of the accuracy of any particular article, it's not worth very much to me.

Wikipedia does have internal metrics grading the quality of an article. [[WP:ASSESS]]. In general though, even entirely discounting the Wikipedia component of the británica comparison, based on británicas own failures it seems wise to verify each and every claim in an encyclopedia, which Wikipedia does an excellent job of helping you do.

> They don't with you, but many people obviously use them that way. Also, reputation does not correlate strongly with reality

OpenAIs own benchmarks show much higher hallucination rates than any study on Wikipedia. Wikipedia itself is quite close to a ban on LLMs for reliability issues. If you ask literally any layman “has ChatGPT ever been wrong for you” they will say yes, either in that moment or after only a little prompting. It is much harder to elicit such a response regarding Wikipedia in my experience

https://cdn.openai.com/pdf/2221c875-02dc-4789-800b-e7758f372...

> "It is much harder to elicit such a response regarding Wikipedia in my experience"

You're sincerely claiming that people can't think of times they've seen vandalism on Wikipedia?

Correct. The amount of Wikipedia pages at any given moment with active vandalism is vanishingly small. The only time I have ever stumbled upon vandalism is as part of my work as a volunteer there actively looking for such cases. Looking at my feed of possibly problematic changes at the moment, about 3 entries are appearing per minute with the most recent revert being just 2 entries ago. It is significantly worse while school is in session in my experience, but vandalism very rarely lasts long. Talking to people, people frequently confess to vandalising wikipedia at one point or another. When I ask them "how long did it survive" they tell me answers ranging between "a few moments" to "5 minutes." So to answer your question, I believe it is unlikely the average person has seen vandalism on the site barring those looking at their own shit.

>> Just as a start, other sources of reference, including encylopedias, dictionaries

> This just seems like healthy competition. I thought we were talking about a situation where Wikipedia’s use of other encyclopedias is an instrument of their demise.

Somewhere above, someone complained that LLMs were harming Wikipedia, a source of its information. My point is that Wikipedia did the same to others.

> > Which businesses did Wikipedia put out of business?

> Just as a start, other sources of reference, including encylopedias, dictionaries, websites, etc. For example, I'm sure it impacts McGraw-Hill's AccessScience, which likely you've never heard of.

Your “for example” in response to a question about what businesses Wikipedia put out of business is a business that is...still in business?

I bet your mind is more subtle than that.

Reply on news.ycombinator.com