The Magic of Embeddings | Hacker News Reader

The Magic of Embeddings

stack.convex.dev

119 pointsgk121 comments

There are often comments on HN about how Google search doesn't work as well as it used to.

It's worth noting that this is probably because this audience wants keyword matching rather than semantic search which Google switched to years back [1].

Curious to know whether embeddings have been a subsequent step in this transition.

1. https://blog.google/products/search/search-language-understa...

Google is pretty bad at semantic understanding too. Try searching for "implementing stackoverflow in typescript". You will get "implementing in typescript" results that come from "stackoverflow". (Then for a good measure try the same in ChatGPT)

Embeddings are the linchpin of semantic search. The embeddings are designed to encode semantic similarity, so the response is formed by retrieving the nearest neighbors of the query, and there are data structures that allow this to be done efficiently.

My hunch is the problem is they are essentially doing both in the same input field.

One of the hardest design problems with search is that the affordances of a search field is largely a mystery. Mixing paradigms does not help.

Yeah, Google uses BERT nowadays.

Given the title, I was expecting a conceptual deep-dive into why and how embeddings are magical. The bulk of the content is more along the lines of a how-to guide for getting started with embeddings.

The best post I've seen so far that really does dive into the magic of embeddings is this: https://txt.cohere.com/text-embeddings/

Great article! Fun, clear, straightforward, good examples, practical.

These type of articles are dime a dozen on Twitter, not clear why this one is on HN front page.

Sure, post some here. Would love that.

I've actually used embeddings at work. I was pretty excited as it was the first of this new AI tech (embeddings, chatGPT, etc) that I used for more than just a query engine.

We offer courses at work and needed a way to see how we compared with the sector. So I wrote a small Python script to scrape the name and description, along with other info, from the websites of other providers in the sector.

The embeddings came into play because our course names don't always line up with other course names. So I used the embeddings for the course description and outcomes to match courses.

The results were mostly good. I haven't done a super deep analysis of the results (got pulled off to work on more pressing issues) but for the most part the courses matched properly.

There were a handful though that didn't really match at all, but I haven't looked into it enough to find out why.

Not in the AI space at all, but this was an interesting glimpse.

I'd like to learn more about the dimensionality of the embedding and how that is minimised - it seems intuitive (not the same thing as "actually correct"!) that keeping the vector length smaller would be more computationally efficient, with a trade-off in (theoretically) worse performance. I'd guess that performance drop assumes a totally "even" distribution of meaning throughout the embedding space, so I wonder if the enhancements revolve around targeting the embedding more towards real data distributions, or something more subtle.

Is there ever a use case for wanting to compare NON-normalized vectors using dot products or cosine similarity? It seems like all use cases always involve first normalizing the values, which makes sense, but wondering when/why you wouldn't want to do that.

Normalized dot product is equivalent to cosine similarity.

Non-normalized vectors are for when some vectors should match less often in general, encoded by length. In particular, vectors that shouldn’t match anything can be encoded as a zero vector. For example, the last layer of a classification convnet is just dot product similarity, and is usually not normalized

The case in which semantic similarity is encoded by Euclidean distance, and you want to perform similarity search (retrieve the nearest neighbors).

But in a model like the article examples with 1500+ dimensions doesnt Euclidean distance suffer the "curse of dimensionality", making it unusable for similarity searches?

That is my understanding. But then again, the angle between of two random points in a high dimensional unit hypercube is 90, so cosine similarity is suspect too. High dimensional spaces are weird like that.

https://martin-thoma.com/curse-of-dimensionality/

Any folks from Convex, do you have plans on adding vector indexes to Convex databases?

Hi there. Another Convex cofounder here.

SiteGuide (https://siteguide.ai/) was the first to do vector embeddings with Convex, built by integrating Convex + Pinecone. This combination has been an increasingly common pattern over the last few months. So we put a template project to demonstrate how this is usually done:

https://github.com/ianmacartney/embeddings-in-convex

We're strongly considering building in vector search a little further down the road, but this is the recommended approach for now.

Convex cofounder here. Initial focus has been low-latency OLTP database workloads, which is pretty tied to the user-facing end-to-end reactivity angle we're pushing. As a bonus feature we also have built-in strongly consistent full text search, but not vector search.

So far our answer for folks who want alternative storage/query engines is to use our streaming Airbyte source connector or write directly to Pinecone, Snowflake, etc. This should work great for most devs.

There are likely always going to be some developers who want to use a particular third party database in addition to Convex, but we plan to expand built-in support for most workloads over time so that Convex is a truly full-stack backend replacement.

low-effort content marketing. did you really lesrn anything you didn't already know about embeddings?

Reply on news.ycombinator.com