Hacker News Reader

Comment by yuhongsun | Hacker News Reader

Ya, handling spreadsheets is a beast of its own. We have this simple implementation to cover easy cases for folks but likely it will need its own more involved pipeline for indexing, retrieval, and interactions with the LLM.

Currently with large tables, it's not handled very well either. The more complete approach would be to pass the headers to the LLM and ask it to generate a formula to parse the data rather than feeding the whole table(s) to the LLM directly.

Some of the bigger items we want to target that require special flows are code search, SQL tables, and Excel/CSVs/TSVs

View on HN

most of the files I have, I'm most interested in finding graphs and then updating relevant data.

it looks like the best way to do that is understand the ooxml format in xlsx. it's all fairly easy to understand.

Ya, parsing the file is generally not bad at all. The problem comes with the fact that LLMs are notoriously bad with numbers and formatted data. So the current approach of passing relevant information to the LLM and asking it to generate answers will produce misleading information when larger tables are passed in.

By asking the LLM to generate a formula though, it doesn't actually need to do any number crunching of its own which makes solving the challenge a bit more reliable when it comes to LLMs.

I've limited my expectations for LLM support to file management. locating relevant datasets or filing away things. the interactive QA just don't seem salient beyond some high level.

A lot of people are very bullish on AI, it's very interesting to hear the opposite side. My opinion is that LLMs are very powerful at digesting and distilling knowledge which is why we built this project. I also think that LLMs are terrible reasoning engines and so agent-flows are not quite ready for primetime.

Would love to hear your perspective on the space!

I certainly see the value of large document retrieval and various forms of search.

However, what seems to be the business proposition is giving managers shallow access to documents but won't lead to rigorous information.

There's a few middle grounds where it can yield insights. like regulatory scenarios where you want to understand how public orgs satisfy permits with written plans.

however, what I don't believe will yield is the context size. when I want to explore my knowledge base, I need far more than 128k and there's sever orders of structure that language itself is not going to bridge.

Ya, a lot of knowledge is locked away in metadata, how documents are structured, in non-textual representations, or sometimes even just in the head of experts. We definitely want to be more than a shallow access to documents and we're building with that in mind. We're currently working to include more metadata and organizational understanding, with plans to tackle OCR, NL-to-SQL, code search and knowledge graphs in the future. While we don't have concrete dates for all of these, hopefully this gives some visibility into our vision at least!

Regarding the context size, the research community is doing some stunning work. If you're interested, you should check out the new Mamba (Transformers replacement) architecture: https://arxiv.org/abs/2312.00752

Reply on news.ycombinator.com