Show HN: Fast autocomplete service written in Go and JavaScript | Hacker News Reader

Show HN: Fast autocomplete service written in Go and JavaScript

autocompeter.com

76 pointspeterbe47 comments

I've found Apache Solr the best approach to building a flexible, fast auto-complete service. Solr can spit results back in json so it's dead easy to write a simple API wrapper in whatever language you're using (go, python, ruby).

You can then stick Varnish in front of the whole lot if the search space is relatively static and not ridiculously huge.

Consider using package `log` for logging, `fmt.Printf` doesn't linearize writes to stdout.

https://github.com/peterbe/autocompeter/blob/master/server.g...

Also with `log` you don't need to add a `\n` at the end, it will do it for you.

Thank you!

Clever name, Peter. :)

Btw, I noticed a missing "c" in the "url -X POST" example on this page of your docs:

http://autocompeter.readthedocs.org/en/latest/api/#bulk-uplo...

Annoying. Must be a bug in ReadTheDocs or something https://github.com/peterbe/autocompeter/blob/master/docs/api...

Actually, local mkdocs gets it right. Only on readthedocs does it get screwed up.

It looks to me like it was missing in the source markdown file? I submitted a pull request on Github.

Right you are! I looked at a different `curl`.

Regarding the name, I just desperately needed to call it something when I started the git repo to build the prototype. Then it stuck and I decided to "brand" it as such.

I blogged about it here: http://www.peterbe.com/plog/autocompeter.com

Meh, it's a decent and properly commented implementation, but the interesting stuff is done by Redis, not in Go.

I wrote a similar service (JS+Go) 2 years or so ago using only the Go standard library (specifically the excellent index/suffixarray and gobs for persistence). It typically got ~3ms response timings (locally) with > 1 million records indexed.

I wrote a cache layer in Go (with limits so it doesn't bloat too much) and I could get 22K req/s instead of the usual 4K.

However, the hit ratio is too low to work. It only protects me from the stampeding herd problem at best.

Why does it matter where the interesting stuff is implemented?

Do you have a link to your implementation or comments on how this one could be improved?

I wrote something comparable. It's at github.com/jamra/gocleo. It is based on a Java implementation.

Would you care to link to what you wrote?

I put together an autocomplete service using Ruby and Redis but rather than dealing with the network latency of a remote service I decided to package it up so it could be mounted along side my app: https://github.com/doomspork/autocomplete-me

Yeah, where I work we use another redis + ruby gem that the seatgeek guys wrote called soulmate. https://github.com/seatgeek/soulmate

Cool! It seems a bit more flexible but it requires jQuery and the plugin itself is 11Kb.

The inclusion of a JS library is only a convenience. There more robust solutions for the UI like Twitter's Typeahead.js https://github.com/twitter/typeahead.js

This is awesome, I hadn't seen it before. Thanks for sharing!

Heh, I always feel a bit bad when people mention this since we're not using it ourselves anymore. Though it's a nice and easy way to get going quickly if you're already running redis (and especially if you've got a rails app).

We're using elasticsearch now which is definitely a bit more of an operational headache.

Curious... why use Elasticsearch over Redis? We're currently using Elasticsearch, but were considering switching to Redis, but now I'm thinking maybe we shouldn't. What's the downside?

aw, you shouldn't feel bad... it's been awesome for us! :)

So, it loads all titles into the client?

Sorry I'm not sure I follow. There is no client included within my repo, just the service.

Comments:

1. The focus should be on the search box when the page is loaded.

2. While typing the word "javascript", sometimes the amount of results shrinks and immediately increases, resulting in flickery behavior.

1. Why? That sample on autocompeter.com is just a sample. Consider the search widget on a page like www.peterbe.com instead. Then it shouldn't focus immediately on load.

2. I see. I think it only happens when the client-side cache is empty. And it only happens if you type very fast. I think the right solution would be to not hide too quickly. That's a very useful piece of feedback. Much appreciated!

Thank you so much for that piece of feedback. It really did flicker.

I investigated the problem and found a very good explanation for why it was happening.

Solved now: https://github.com/peterbe/autocompeter/commit/a31c919c1281d...

Would you mind trying again? Make sure to refresh first.

" Fast Redis autocomplete service written in Go and JavaScript "

FTFY

Taglines are hard. I tried to make the tagline more "end-user friendly" instead of focusing on the tech. The people who implement it actually don't need to know how it's made. They just drop in the .js the .css and send their data to the REST API. They don't need to know that the database is Redis and that the server framework is Go.

It would have been nice to give suggestions even if you make spelling mistakes.

Yeah I agree. A Levenshtein distance shouldn't perhaps be too hard.

Also, I think it would be nice to be able to upload a list of synonyms. E.g "go==golang" so if someone types "golan" it could return "Go is a language" for example.

Ok, what `fast` means here? There is no database for the server-side project.

It is fast because Redis, which provides the storage, is considerably fast at the tasks necessary to implement such a service. Go has last little to do with it.

True but it's open source and perhaps people are interested to see something beyond HelloWorld with Go+Redis.

I actually re-wrote it once to switch out Redis for Postgres but it was way too slow.

There is a database. It's redis. It's persistent.

20,000 searches have been made in the last post. I love HN!

Nice! This will come in very handy for me.

I wonder how they can do this for free?

It started as a side-project and still is. I'm the creator of the service and my day job is as a web developer at Mozilla.

If this service becomes surprisingly popular I'll look into ways to monetize but that's unlikely to be the case unless I'm really really lucky.

The origin of building this is that I found a cool prototype of the concept in a blog post, then threw that into my own site and later realized I want to re-use that for a (day)work site. So I though instead of copying it, I'll write a microservice.

Very nice. I hope to use it one day.

Does it support unicode?

In the features list:

> Can find "Pär is naïve" by typing in "par naive"

Yes. And unidecode too :)

and rust ?

Next version maybe.

Reply on news.ycombinator.com