Hacker News Reader

This is really cool!

Can Haystack be used to index structured data, or just text?

Is it required to use elastic as the backend, or can you use a simpler file-based or in-memory backend?

Re structured data - in theory, yes :) We have to work a bit more in that direction. Here's the first step - querying table data, which could be really helpful for reports, financial data, etc. In regards to the storage backend - it's currently Elasticsearch, OpenSearch, SQL+FAISS/Milvus/Weaviate (when using dense vectors/dense passage retrieval). There is also an in-memory datastore using python primitives for fast prototyping.

(Also, latest features highlights here https://www.deepset.ai/blog/new-features-in-haystack-v1.0)

Reply on news.ycombinator.com