Hacker News Reader

Comment by NeroVanbierv | Hacker News Reader

Really love the island map! But the automatic zooming on the map doesn't seem very relevant. E.g. try typing "openai" - I can't see anything related to that query in that part of the map

View on HN

Indeed I've long been intreagued by the idea of rendering such clustering maps more like geographic maps for better readability.

It would be cool to have analogous continents, countries, sub-regions, roads, different-sized settlements, and significant landmarks... This version looks great at the highest zoom level, but rapidly becomes hard to interpret as you zoom in, same as most similar large embedding or graph visualizations.

Ok I just noticed there is a region "OpenAI" in the north-west, but for some reason it zooms in somewhere close to "Apple" (middle of the island) when I type the query

Thanks! Yeah sometimes there are one or two "far" away results which make the auto zoom seem strange. It's something I'd like to tune, perhaps zooming to where most but not all results are.

Often embeddings are not so good for comparing similarity of text. A cross-encoder might be a good alternative, perhaps as a second-pass, since you already have the embeddings. https://www.sbert.net/docs/pretrained_cross-encoders.html Pairwise, this can be quite slow, but as a second pass, it might be much higher quality. Obviously this gets into LLM's territory, but the language models for this can be small and more reliable than cosine on embeddings.

Reply on news.ycombinator.com