Citation Needed – Wikimedia Foundation's Experimental LLM/RAG Chrome Extension | Hacker News Reader

Citation Needed – Wikimedia Foundation's Experimental LLM/RAG Chrome Extension

chromewebstore.google.com

153 pointsbrokensegue56 comments

Open articleView on HN

Experimental LLM powered RAG application for checking claims on the Internet against Wikipedia.

You can read more about it at https://meta.wikimedia.org/wiki/Future_Audiences/Experiment:...

Disclaimer: I worked on this.

Any plans for a Safari extension?

Or Firefox

we've discussed expanding to other browsers. the plan is to evaluate the experiment soon and based on feedback decide whether to expand/change

Hey matey, do you folks need any more devs to bring ideas or code?

I’m heavily into browser extension development! I’ve done some insane things with them.

I’ve built about 8 browser extensions in the last 6 months, most of them have thousands of users and one of them had half a million users

Currently building an LLM powered design assistant extension.

If you’d like to chat im reachable by email at “hello[at]papillonsoftware[dot]dev”

> I’ve built about 8 browser extensions in the last 6 months, most of them have thousands of users and one of them had half a million users

Do you have any concerns about your ability to properly maintain so many extensions?

Not really, most of them are “feature complete” as they target a very small surface area or feature.

For instance, one of them reveals salaries on job seeker sites, and is feature complete as it does what it’s meant to do bug free and fast

No web extension is going to remain feature complete or bug free for very long. What happens when the job seeker sites change their HTML/CSS/JS?

I would assume this is an easy update (possibly even doable by an LLM).

Okay mate you’re the expert clearly, you tell me

It looks like this doesn't check whether the article itself cites a source for the claim. Is that why it's called "Citation Needed"? Because it doesn't actually cite anything?

Trust me bro™

Anywhere that gets into technical details and design?

Not the person you've responded to, but I found this: https://gitlab.wikimedia.org/repos/future-audiences/citation...

Can quotations be hallucinated? Or are you using something like "deterministic quoting"[1]?

Disclosure: author on that work.

[1] https://mattyyeung.github.io/deterministic-quoting

the quote cannot be hallucinated but we use a different approach. your work seems interesting though.

I would love to learn more, where would you recommend I look?

That's really cool, do you think this might be the basis for potential natural language navigation? (when going over a document, instead of having to search by keyword or regex, one can search for more complicated concepts using English)

If not, what extra work is needed to bring it to that level?

I think you could get a pretty good solution for that using RAG and some tricks with prompt engineering and semantic chunking. With google's very-long-context models (Gemini) you may also have good results simply with some prompt engineering. Preprocessing steps like asking the LLM to summarise themes of each section can be helpful too (in RAG, this info would go in the 'metadata' stored with each chunk, presented to the LLM with each chunk).

A key engineering challenge will be speed ... when you're navigating a document you want a fast response time.

No firefox extension though.

If Wikimedia Foundation didn't get paid for basing this project on an aggressively embrace&extend browser, they should look into whether they're leaving money on the table, since projects like this help extinguish the browser's competition.

Got it, I can’t use it then.

This is a great idea. Hopefully we also have LLMs fact checking and flagging Wikipedia articles.

Wouldn't that just be checking wikipedia articles against the same wikipedia articles which the LLM originally trained on?

No, citations are supposed to be from reliable secondary sources or authoritative primary ones external to Wikipedia

Many of those sources are not available online (books), point at paywals (research papers) or are dead . Unless you have an API that can bypass these issues reliable you are stuck with a tool that has already landed several lawyers in hot water for making up citations on the fly.

The Wikipedia Library project [0] grants active editors access to a wide range of otherwise paywalled sources. I wonder if it could not be extended to this sort of bot.

[0] https://diff.wikimedia.org/2022/01/19/the-wikipedia-library-...

What LLM will we check those against? How do we trust that its source materials are accurate and correct?

I think the idea is that you feed the LLM the article & the source material (from citations) and it checks if they match up.

Sure, but what happens when the article is updated at a later date, or rescinded, etc.? Should the LLMs be trained to repeat the article verbatim, or to say "according to this article[0], blah blah blah" with links to the sources?

Wikipedia works because we can update it in real time in response to changes. LLMs that need to constantly recrawl every time a page on the internet is updated, and that properly contextualize the content of that page, is a huge ask. Because at that point, it stops being an LLM and starts being a very energy-hungry search engine.

Well, it's just a bot, so no need for it to instantly react to any and every update.

I also have my doubts on whether it is possible to implement efficiently (or at all). I suspect that just yanking in the article and all the sources is non-feasible, and any smaller chunking would be missing too much context. Plus LLM logical capabilities are questionable too, so I don't know how well the comparison would work...

wikimedia's git repo for this extension

https://gitlab.wikimedia.org/repos/future-audiences/citation...

edit: the readme build instructions are incomplete and i don't think hotreload works. use `npm run build-dev` to get a working build.

it's not obvious to me what prevents this from being a firefox extension as well - it might be the sidebar/sidepanel api differences, but i haven't played with those much

Couple this with a search based article generator so you have an article generator and an article checker, and then off you go to generating 1 trillion pages. Could be useful training content for LLMs but also used by people.

Does this only work on Chrome? Can't seem to make it work on Arc.

@wikimedia: Firefox version please

> A chrome extension for finding citations in Wikipedia

In academia, Wikepedia citations are generally a no no. One reason is their unrelaibity (the author is citing a source that they themselves can edit). More importantly, Wikepedia may be a good place to find primary sources, but in itself it is a secondary source.

It really shouldn't, considering they have a rule about not using primary sources.

Yes, they explicitly classify wikipedia as a tertiary source [1].

Wikipedia is good for finding secondary source, and then primary sources by following the links.

[1] https://en.wikipedia.org/wiki/Wikipedia:Primary_Secondary_an...

That justification seems a bit behind the times honestly. We've now seen actual academic fraud on a massive scale with extremely little in the way of a correction to fix this, while at the same time we've seen Wikipedia handle abuse extremely well. The academic fraud is a threat to Wikipedia, more than using wikipedia links is a threat to academia.

This seems no different than it’s always been. Even before Wikipedia you would not cite secondary sources. But you sure would use them to get a foothold on a topic and find some of those sources.

The point is to find the information in Wikipedia which often then has a citation to another source. If you search Google you often find repetitions of the information but most sites don't cite sources.

Yes... this exactly.

> One reason is their unrelaibity (the author is citing a source that they themselves can edit)

There is plenty of reasons why wikipedia is an inapropriate source to cite most of the time in academia, but that surely is not one of them.

Acedemics cite their own papers or other sources they have editorial control over all the time.

Wikipedia prefers to be a tertiary source. They want the references to be to secondary sources, rather than primary sources.

"Wikipedia articles should be based on reliable, published secondary sources, and to a lesser extent, on tertiary sources and primary sources. Secondary or tertiary sources are needed to establish the topic's notability and avoid novel interpretations of primary sources."

https://en.wikipedia.org/wiki/Wikipedia:No_original_research

Asking about itself returns this:

> "Citation Needed is an experimental feature developed in 2024"

- The provided passages do not contain any information about a feature called 'Citation Needed' being developed in 2024.

- Wikipedia

- Discouragement in education

- not be relied

I know I'm not using it for intended purposes but it seemed funny.

I wonder how this will go over politically with the wikipedia community. AI is such a hot button issue, and the risk of hallucinating and saying something wrong seems more pressing in this application then most.

I haven't used this, but reading the description, it sounds like it's primarily a search engine for wiki articles related to selected text. If so, I imagine it wouldn't be super susceptible to hallucinations.

It uses AI to parse the selected text to choose search keywords, and to parse the related Wikipedia article to decide if it agrees with the selected text. It obviously can bullshit in both cases.

Searching for keywords shouldn't be likely to hallucinate. And I would assume they would have a subsequent step to run a quick check to see they're really in the text. And if there is some issue, I suppose we can always fall back to something like TF-IDF.

The second part does seem more problematic, but still, as essentially a yes/no question, it should be significantly less likely to hallucinate/confabulate than for other tasks.

The extension appears to also produce an explanation of its decision, so there is potential to bullshit: https://meta.wikimedia.org/wiki/Future_Audiences/Experiment:...

Also, if you look into their “wrong” example closer, it is a bit misleading, as both sources are correct. Joe Biden was 29 on election day, but 30 when he was sworn in. Understanding this requires more context than the LLM was apparently provided.

Honest question - do you expect there to be hallucinations in this case? I have extensive experience in LLMs and also from talking to peers with similar experience, it is uncontroversial to say that given grounding like this LLMs won't hallucinate.

I am not sure if when people say this they just don't have experience building with LLMs or they do and have experience that would make for a very popular and interesting research paper.

The burden of the proof of no bullshit is on the LLM proponents.

Well wait a sec, you’d be just as guilty of confident bullshit if your claims above don’t pan out, and they didn’t even come from an LLM so it’s worse.

"it is uncontroversial to say that given grounding like this LLMs won't hallucinate"

I disagree. LLMs that have been "grounded" in text that has been injected into their context (RAG style) can absolutely stolñ hallucinate.

They are less likely to, but that's not the same as saying they "won't hallucinate" at all.

I've spotted this myself. It's not uncommon for example for Google Gemini to include a citation link which, when followed, doesn't support the "facts" it reported.

Furthermore, if you think about how most RAG implementations work you'll spot plenty of potential for hallucination. What if the text that was pulled into the context was part of a longer paragraph that started "The following is a common misconception:" - but that prefix was omitted from the extract?

Reply on news.ycombinator.com