Vector Search with SQL | Hacker News Reader

Vector Search with SQL

neuml.github.io

36 pointsneumll11 comments

While it presents txtai as a very neat framework for calculating and searching for embeddings the article doesn’t seem to mention anything about a database server, except for the drawing at top.

The code looks like it is doing an in-memory search.

When searching an embedding space, it is quite common to look for the closest angle of the query vector against amongst the search vectors.

That is a calculation I don’t know how to make a sql server perform and given the headline I thought the article would show a neat trick for doing this.

More information on how txtai builds SQL statements can be found here: https://neuml.github.io/txtai/embeddings/query/#sql

There is also an example notebook: https://colab.research.google.com/github/neuml/txtai/blob/ma...

At a high level, txtai uses a similar clause embedded in SQL statements. For example, "SELECT id, text, score FROM txtai WHERE similar('feel good story') AND text LIKE '%good%'". This statement is parsed and the similar clause runs against the approximate nearest neighbor index. The result ids are then loaded into a temporary table and the SQL statement is dynamically rewritten to change the similar clause into "id IN <temporary table>".

I’m late to the party; I just learned about embeddings. I’ve been playing with lexvec this week. I feel like I’ve found a book of working magic spells.

It's certainly a fast moving space. It's not perfect but the NLP scene has made huge strides the last couple years.

In 10 lines of code you can import a transformer and embed your phrase/word. Then you can do prediction or similarity search really easy. It's a superpower, true.

I’m not clear on “where” this sits, is it a framework for turning text into embeddings? Is it the database and serving layer (like Milvus, Vald, Pineconr, etc)? Is it just a Python package for generating and serving vector search in-process?

It's all of the above.

txtai uses transformers to transform data (text, images, audio) into embeddings. Those embeddings are then loaded into an approximate nearest neighbor index for search. On top of that, content is loaded into a relational database to support SQL based filtering. It's trying to get the best of both vector/similarity search alongside standard structured search using a SQL syntax.

This can be run in-process or via an API - https://neuml.github.io/txtai/api/

Components can be split up, for example there could be a server that vectorizes text into embeddings and another server that hosts the indexes.

There is also a pipeline and workflow framework (https://neuml.github.io/txtai/workflow/). This component has modules that assists with splitting data, transforming, summarizing, translating, parsing tabular content. Workflows can be used purely for transformations or as a driver to load data.

Link to GitHub project: https://github.com/neuml/txtai

Looks great. How does it compare to hnswlib? How many vectors can it handle for subsecond queries? Is it approximate or precise nearest neighbor?

Thank you. The indexes are built on top of either Faiss, Hnswlib or Annoy depending on the settings - https://neuml.github.io/txtai/embeddings/configuration/#back...

I've primarily focused on single query times, which have averaged around 5ms - 25ms depending on the size of the index. Queries can be batched so query times wouldn't increase linearly. You could batch quite a few and still get subsecond response times.

All three libraries are approximate nearest neighbors but I know at least for Faiss, it can be configured to effectively be a precise query.

Thank you. I’ll check it out!

Reply on news.ycombinator.com