Hacker News Reader

Comment by twelfthnight | Hacker News Reader

Are folks typically using HNSW for vector search these days? I thought maybe ScaNN has proven to be better? Especially since it's available in FAISS [2].

[1] https://ai.googleblog.com/2020/07/announcing-scann-efficient... [2] https://github.com/facebookresearch/faiss/wiki/Fast-accumula...

View on HN

Depends... I have a beef with all methods based on "trained quantization". It introduces too much noise in your distribution, suffers from drifts, and makes the method mostly inapplicable for other forms of "Similarity Search" that don't strictly fall into the "Vector Search" category.

Many disagree. Pick whatever rocks your boat, there is a FOSS library for almost everything these days :)

Ah, those are interesting considerations. I don't have a horse in that race, I just had to implement a similarity search algorithm a few years ago and it was surprisingly difficult to find a consensus on what ANN algo to use!

Yeah, SPANN has better f1+queries per second on some benchmarks, but that's a little like comparing sorting algorithms, they're both fast and good.

The database software behind the ANN algo is probably a little more important in practice than the ANN algo itself, unless you're operating at such scale and speed that its an actual issue (e.g. you're google).

Differences between algorithms are a little more interesting when they let you do something totally different, like, for example, minimize the speed hit from doing searches on disk (SPTAG, DiskANN).

ScANN is not available in FAISS, it is Google's version of it.

But FAISS has their own implementation of those ideas ("FastScan") which is what GP linked.

Reply on news.ycombinator.com