Hacker News Reader

At least w.r.t. to the synonym problem, more modern search techniques that rely on language modeling and word representations seem to solve that.

View on HN

Is there a Elasticsearch-like server that uses language models? Or does it require a sophisticated team to tune and host?

There are a few things in this space that might be interesting:

- https://github.com/Hironsan/bertsearch

- https://github.com/hanxiao/bert-as-service

There was another I can't find right now that looked more polished/professional. But, in short, no it's pretty easy to setup. Just need a machine with a pretty big disk and be ok with an index latency. If you're using elastic search then you're already there on both accounts!

Do you have some recent examples?

The most recent understandable implementation of this is something called Word2Vec. It is not the state of the art (it's actually suboptimal now by a lot) but there's a LOT of explanations of it you can find that are great.

A decent video covering it from a programmers perspective (where I'm coming from) https://www.youtube.com/watch?v=LSS_bos_TPI

From a math perspective there's also good materials.

Essentially Apple + Tree != Apple + Computers allowing you to differentiate search like this.

Reply on news.ycombinator.com