Hacker News Reader

Lots of focus on RAG here, and rightfully so, but I feel an overlooked benefit to vector databases is the novel visualizations they provide. To be able to plot qualitative data onto a 2d graph with tsne reduction provides a new method with which to draw insights from. I think many companies would benefit from such a visualization tool, especially those in qualitative research.

View on HN

Unfortunately that visualization doesn't really work with vector dbs. Vector dbs normally split their data into separate segments and build indexes on them separately. There is no one overarching index but rather many small ones that are searched in parallel. In addition to this, such a high compression to reduce it down to 2d/3d ends up becoming a giant blob without too much information.

Maybe I'm misunderstanding what exactly a vector db is.

Let's say you have a chatbot and stored in its database is the usual info like session id, timestamp, message etc. To "vectorize" this db then would be to vectorize all the messages. Is this too simple an understanding?

Once the db has been vectorized then we can do semantic search on the messages and create more informative graphs based off the semantic similarity for messages within a given timeframe or other criteria.

Reply on news.ycombinator.com