https://www.pinecone.io/learn/vector-database/
...was less than informative.
https://www.pinecone.io/learn/vector-database/
...was less than informative.
Example:
Let's suppose I've downloaded all the Gutemberg library books.
I can feed a transformer like Bert or GPT-3 to calculate the embeddings of these books.
These embeddings will represent in vector form (an array of numbers of fixed size) the meaning of these books.
I can save these vectors in this database and this database can then calculate the distance between these vectors, so basically how closely related they are in terms of topic.
If I query this database with the embedding of a sentence like "Love story between teenagers from 2 enemy families in Italy, they die at the end", hopefully the best result will be Romeo & Juliette.
I'm no ML person so take my comment with a grain of salt in terms of how well it works. But in theory, that's the goal.
ABCABCABCABC
Vector databases store them like this: AAAABBBBCCCC
This allows faster queries if you just need one (or a few) columns, because unrelated columns don’t have to be processed at all. Caches are more efficient, vector CPU instructions can be used, etc…The downside is that random single row access is more expensive because a row has to be reassembled from many locations.
Row store: “table id; row id; column id”
Column store: “table id; column id; row id”
The difference is the order used to sort the data in storage.In reality, most column store systems use a more complex partitioning scheme with row groups and the like…