Show HN: SearchArray – Text Search in Pandas

I've long worked with lexical Lucene based search engines like Solr and Elasticsearch. Anytime I need to experiment with relevance ranking in these systems, I'm exhausted by needing to set them up. It's not conducive to productivity work with something so disjoint from normal data tooling.

Further - the underlying ranking is buried in needless mystique (you know a boolean should query, sums the scores, right?). You shouldn't need to read a book (like Relevant Search ;) ) to unpack mystique that's really basic math.

Why not just let people build ranking systems with vectorized math in a numpy/pandas stack?

SearchArray lets anyone build a relevance prototype in Pandas. Typically building / experimenting with a smaller labeled dataset. If it works out, you can transfer it relatively easily to Elasticsearch or Solr for implementation and testing.

SearchArray is a pandas extension array that creates an underlying search index for BM25 term/phrase based searching.

It's not quite done (will it ever be?) but its getting far enough along to be useful. So feedback is very welcome.

https://github.com/softwaredoug/searcharray