Hacker News Reader

Comment by carrolldunham | Hacker News Reader

Bing can do this (with GPT-4) but the problem is it has an unacknowledged limit to the amount it's able to read, and so it mishandles summarizing a large document, seeming to read the start and a few random pages. How does yours handle large sources?

Edit: I tried this one with an 800kb .txt and after digesting it, it got two answers wrong (but at least related to the text) and then started spitting out "I don't know". I asked "what is this document?" because I saw with my previous test that it can get blocked and be working with a 404 page ("this document is a page that says suspicious activity request denied") but this time it just said "I don't know."

View on HN

Re: your Edit, it's possible that your questions were follow up questions, which are difficult to make sense of on their own--the service at the moment starts from scratch for each question (has no memory of previous questions and answers).

We'll look into adding memory (either as a default or as an option).

I am on the waitlist for Bing and can't check directly--would it also answer specific questions about the doc? (Rather than summarize.)

Our site is meant for Q&A, and has a layer of tech that finds the sections in the large document that are relevant to the Q first. This will not work well in general for summarization on unstructured content. But most content tends to be structured and in practice we are finding that the approach still works on e.g. news articles, wikipedia articles, blog posts. It's almost as if where it doesn't work, a human would have trouble too. (e.g., on a long rambling HN thread).

>would it also answer specific questions about the doc?

Yes.

There is no waitlist anymore, and this functionality is delivered through the Bing sidebar.

Internally, it has the same privileges as a third party extension with webpage content access, so it cannot access the pdf viewer contents. I believe Edge is getting a new PDF viewer in Canary that might solve this. For now, you can select text 2000 characters at a time and send to chat or give it a URL (assuming that Bing can see it in the search index).

Bing chat is already good at handling recursive queries (with internet access) and processing poorly formatted PDFs from the indexer webcache, so I assume it would do well given the right conditions. It does Q&A really well on GitHub repositories, for example.

> the problem is it has an unstated limit to the amount it’s able to read

Not sure which version Bing uses (I’d guess the smaller), but GPT-4 has either an 8K token or 32K token context (prompt + response) space, some of which is taken up by the hidden system prompt, but that limit is also there using the model through the API.

The 'creative tone' mode of Bing was upgraded to 'large context size', presumably 32K. By unstated (I've changed my post now) I meant that it doesn't report back "that's too long", it just says "sure here's a summary", which is not representative of the document. I saw another "pdf reading" service (not OP one) that suffered this too.

Yeah, GPT itself will do this; you could probably do a separate token count in the app wrapping it to figure out if it was likely to have occurred (or limit the input to a size sufficiently less than the limit to assure that a summary wouldn’t exceed the limit.)

I mean, if you wanted to add no value to GPT, yes. The value of any 'read a pdf' app should be that it traverses the full document as needed for a query, copying into its internal context what may be relevant, re-iterating until the true answer is found regardless of whether it's scattered through the large document. Bing and the OP app both have as author says a "layer" that should do that but currently

- Bing isn't using the layer right - it can search the whole internet to orchestrate context for itself but doesn't seem set-up to so search one doc

- OP app seems to crash after a while

- the third service i forget the name of which has been shown on here (something to do with 'pdf') similarly to Bing doesn't 'search around' in long documents appropriately

The crash was unrelated to tech and a provider billing issue. Fixed. We do traverse the full document, all 2,000+ pages that we support. Give it a go!

Reply on news.ycombinator.com