Instacart launches Searchkick - Bootstrap for Search | Hacker News Reader

Instacart launches Searchkick - Bootstrap for Search

rocketship.instacart.com

94 pointsapoorvamehta22 comments

This looks pretty good. Though many of the features are already baked into PostgreSQL's full-text search, out of the box this seems to add enough value to get me interested. I'll definitely be checking this out further.

Upvoted, and I'll pile on. Wordnet [1] already has addons for Postgres.

[1] - https://github.com/taotetek/wordnet-postgres

Can you clarify... does this addon simply store the Wordnet data inside Postgres, or actually let the postgres full-text search do query-expansion-to-synonyms?

Sure. There are functions in this package. This link is better, which explains how the functions will work.

http://wnsqlbuilder.sourceforge.net/sql.html

I recently had to implement search on my product and found Postgres' full-text search to be fairly impressive. More than enough for what I needed.

Implemented through the wonderful textacular gem: https://github.com/textacular/textacular

Interesting. I'll have to look into PostgreSQL's full text search. Does it support token filters similar to Elasticsearch?

To a degree, and is also very customizeable. It breaks things out to the lexeme level, read http://www.postgresql.org/docs/9.2/interactive/textsearch-in....

We looked into PostgreSQL search for a bit, but didn't see a way to train search results from past conversions. Does anyone know if this is possible?

This is nice - We use elasticsearch, but getting good quality has been hard - setting up the tokenizers, filters etc in a good way is tricky, and sometimes our search gives really bad results.

That's exactly why we open-sourced Searchkick :)

Elasticsearch should be the bootstrap of search imo. Afaics the only interesting bit in this project is the reindex technique. The rest is just trying to make a nice rails idiomatic client library (perhaps for ruby too) around search querying. For a python idiomatic client lib comparison, see mozilla services http://elasticutils.readthedocs.org/en/latest/

Training searches from conversions (self-improving searches) is kinda interesting. Elasticsearch is an amazing product but it can be hard to configure, especially for beginners.

Edit: spelling

I appreciate that you decided to open-source this project. I do have a few (hopefully) constructive criticisms to make:

1. Using aliases the reindex with near zero downtime is very clever, mad props to Jaroslav Kalistsuk

2. Tire is excellent, I am glad you didn't try to reinvent the wheel here, please keep it that way.

3. Conversion based score boosting, although clever is flawed. Hardcoding values https://github.com/ankane/searchkick/blob/master/lib/searchk... is convoluted. You should consider providing a more flexible DSL to customize how conversion based score is calculated.

4. Analyzers https://github.com/ankane/searchkick/blob/master/lib/searchk... are poorly named and do not represent what they do.

Again, this is very nice for version 0.1.2

Hardcoding your scoring function here is a bad, bad idea:

https://github.com/ankane/searchkick/blob/master/lib/searchk...

Just curious, how many docs are you indexing?

We're indexing around 100k documents. What line are you referring to, and what's bad about it?

I'm curious about this "bad, bad idea" theory too. For our site, I have a handful of hard-coded scoring functions that the site can select from based on search context, and it's far more supportable and debuggable than some of the dynamic query assembly approaches I've tried in the past.

ElasticSearch is a wonderful product that is a real pain to tune, so I'm curious if there's a better approach I'm missing.

Looks very cool! Going to play around with it some. Is this a first open release of the project or is it in quiet use other places than Instacart?

It's been on Github for a bit, but this is the first big release.

This may be a dumb question, but does Instacart use Searchkick?

Yes, Instacart uses Searchkick.

Thanks! I'll give this a go for sure!

This is really nice. Thanks.

Reply on news.ycombinator.com