Has anyone systematically compared embedding based retrieval against traditional full text search? With all of the focus on vector stores lately, it feels like the whole field of information retrieval has been nearly set aside.
BM25 presents a challenging cross-domain benchmark, and it wasn't till ~2022 that neural methods overtook it. If memory serves, it was the sparse neural methods like Splade, although recent dense models can also beat it.
The caveat is that BEIR is suffering from overfitting at this point.