Hacker News Reader

Comment by smusamashah | Hacker News Reader

I have not found any mention of accuracy. Since it's using LLM, how accurate the conversion is? As in does that NASA document match 100% with the pdf or did it introduce any made up things (hallucinations)?

That converted NASA doc should be included in repo and linked in readme if you haven't already.

View on HN

Couldn't one just program a linear word matching test to ensure correctness?

People are really freaked out about hallucinations, but you can totally tackle that with solid prompts. The one in the repo right now is doing a pretty good job. Keep in mind though, this project is all about maxing out context for LLMs in products that need PDF input.

We're not talking about some hardcore archiving system for the Library of Congress here. The goal is to boost consistency whenever you're feeding PDF context into an LLM-powered tool. Appreciate the feedback, I'll be sure to add that in.

I don’t think any prompting skill guarantees the absence of hallucination. And if hallucination is possible, you will usually need to worry about it

As soon as you have something else than a paragraph in a single column layout, you will get hallucinations, random stuff, cut off etc even if you say which pages to look at, LLM will just do what ever.

Can you give some examples of prompts that you use that will tackle hallucinations?

> People are really freaked out about hallucinations, but you can totally tackle that with solid prompts.

> The goal is to boost consistency whenever you're feeding PDF context into an LLM-powered tool.

These two assertions are contradictory.

There are no "solid prompts" which obviate anthropomorphic "LLM hallucinations." Also, there is no deterministic consistency when "feeding PDF context" into an intrinsically non-deterministic algorithm, as any "LLM-powered tool" is by definition.

> hallucinations, but you can totally tackle that with solid prompts.

This is so wrong. This so much sound as if you have not used LLMs to do any real work.

Reply on news.ycombinator.com