Greplin’s Chrome Extension Now Makes Gmail Search Infinitely Better | Hacker News Reader

Greplin’s Chrome Extension Now Makes Gmail Search Infinitely Better

techcrunch.com

55 pointsssclafani41 comments

I want to love Greplin. I hate gmail search, and would kill for anything better. But giving a startup a complete database of my e-mails? That is a little scary.

Anyone else have the same concerns?

Yes. Scares me too much to even consider the service.

Yep. And above it looks like they use IMAP IDLE to watch for new mail in your account, which presumably means they need your real Google password, and don't go through the OAuth stuff (unless Google's IMAP server has an alternate auth method). Deal-breaker for me.

Having access to all of my data... well... Google already has access to all my data, and they were once a startup. Of course, Greplin could get bought by a company with a poor privacy track record. So there's that to consider. I'm not sure that alone would keep me away from the service, but it might.

Sigh... but it is really cool.

EDIT: Neeeeevermind, just saw this: http://news.ycombinator.com/item?id=2452068

Maybe it would be okay to use Greplin for company mail stuff where technically the company already owns all of your emails anyway -- of course they'd have to deploy Greplin company wide though.

Deal breaker here.

Giving a company, start-up or not, access to _all_ my data just so I can search easily? Not a chance.

People said the same thing about Mint, but things worked out pretty nicely for them..

Same here, I would pay good money for the offline version. Personally this falls into the category of "too much risk not enough benefits"

I hear about these long search waits in Gmail, but I have yet to encounter them. Granted, I only have about 26k emails, but every single search I've made completes within the second.

Also, killer feature question: does Greplin allow search operators, like `label:`, `from:`, `has:attachment` ? And can it search within those attachments (not sure if any desktop clients do this currently, but I wants it, Precious. I needs it.)?

from:foo, has:attachment, to:foo all should work.

The coolest features:

"Once you’ve started searching, you’ll notice that tweaks to your query show up in real-time as you type each character, the same way they do on Google Instant. And it shows results for partial-word matches (“Tech” would match for both “Technology” and “TechCrunch”) — which Gmail doesn’t do."

And "a message with 500k of attachments showed up in my search results within a second of receiving it. Not too shabby."

Full disclosure: I work at Greplin :-D I'm just proud of the very cool stuff my coworkers are building!

How exactly is the data from gmail gathered so quickly? Presumably they are using a gmail API, but does that mean they are polling very frequently?

I guess another way to do it would be to create an auto forward to greplin filter in gmail when a user connects their gmail account but that seems wrong.....

We use IMAP IDLE: http://tools.ietf.org/html/rfc2177

Is it used for indexing or at the moment query hits the search box? (I am interested in how this process works in more words than 4 and less than in that tech spec that i am illiterate about, sorry for that)

For GMail, our syncers hang out on an IDLE'd IMAP connection and process (and then index) whenever a new message comes in.

Edit: So, to clarify, it's used to trigger an update to the index. Hopefully this happens as close to the time the message hits the inbox as possible.

So for every Greplin user you have a TCP connection open to Gmail's IMAP server? How can that possibly scale in the long run?

Data is indexed to Greplin's servers

I've never had a problem with Gmail search and I'm surprised the article writes about 20+ second search times as if it's a common and widespread issue.

I have around 45,000 messages in my Gmail inbox. My experience searching it has felt sluggish for a long time. But really the fatal flaw is that search terms have to be damn near exact. Gmail is a joke compared to Google when it comes to fuzzy searches. I often find myself having to do several searches to get what I want, compounding the issue of search latency. It's so bad I sometimes give up entirely.

I'm at 264,500. And yes, it's about as terrible as you'd imagine.

Ouch. If you're anything like me, around 90% of your traffic comes from mailing lists. Greplin could win big by inferring what mailing lists you are on (along with beginning and ending subscription dates) in order to factor out all relevant storage and indexing.

Do you need to store mailing list email? Usually there's a public archive.

Sometimes it's just easier to search for things in Gmail. But since it is just a mailing list, it's pretty easy to label it and kill it en masse if you need to. At least, that's what I do.

The substring search is HUGE, though. Also, if they support searching attachments, that would be great -- I haven't looked carefully to see if they do.

I think the 30-40 and 20+ second claims were not intended to be factual statements.

If I do a from:, to:, and search keyword, Gmail can often take around 20 seconds; normal keyword searches for recent items take about five seconds.

Gmail slowdowns seem to be common for older Gmail users with a ton of mail and lots of labels; Google has acknowledged the problem and claimed they're working on it, but the only people who have had fixes are ones who've complained loudly enough to get moved around to a newer server.

I appreciate your feedback. I've been on gmail since 2006 or so with lots of labels, but only ever about 500-1000 messages at any given time.

I was trying various searches today, with the from:, to: operators, as well various keywords and the longest lag I had was a little over 2 seconds. I've heard people complain about it, but never to the tune of 20 seconds. I genuinely thought it was exaggerated for effect in the FA.

https://www.cloudmagic.com/

Try this. It builds a local index of your gmail messages. Instant gmail search (and also matches partial keywords like Greplin). No worry about exposing your data to a third party service.

And CloudMagic is available for both Chrome and Firefox. This is important to me because I'm a strict FF fanatic.

"there’s one possible issue for those of you who are concerned about your privacy"

for anyone worried about this kind of thing, there's a program called mairix that will index and search your local maildirs. it's command line, but the results can be used to populate another maildir folder, so it integrates pretty nicely with clients - http://www.rpcurnow.force9.co.uk/mairix/

if you want something more C21 then there's also beagle -http://beagle-project.org/Main_Page [edit: skip that; looks like it's a dead project; recoll might be a good replacement - http://www.lesbonscomptes.com/recoll/ ]

I think all the talk of being faster than gmail search is going to last until they actually get a lot of users. It's easy to make search fast for small datasets.

You're not making any sense. They can shard the hell out of their indexes since all searches are user specific, so that's a non-issue. They can afford to throw enough hardware at the problem if their business model works out and they attract enough paying users. It's presumably not cost effective for Google to offer a super snappy search experience through gigabytes of user-specific data, which is why Gmail search sucks so badly for people like me. (Does anyone know if search is way better if you're a paying customer of Google Apps?)

The existence of a way to shard searches doesn't make scaling real time search on email (hint: do some back of the envelope calculation on how much data that involves) a non-issue.

Fair enough, I too readily dismissed your drive-by objection, however non-specific and snide. Scaling real-time anything is never easy. My main point was that cost effectiveness is what keeps GMail from offering a similar level of search quality. If Greplin can get a large enough fraction of users to pay for their service, the technical challenges are surmountable.

It seems pretty promising so far. I guess my next Emacs hacking project will be a Greplin source for anything.el. Has anyone written plug-ins for Spotlight or Quicksilver?

For someone like me whose attraction is improved Gmail searching, I can't see paying $4.99/month for this. Were my daily workflow based on Google Apps and Evernote, it'd be another story.

How do they access the data used to build their search index?

You have to sign up at https://www.greplin.com, and grant us permission (via OAuth - so we never see your password) to index your mail for you.

See http://googlecode.blogspot.com/2010/03/oauth-access-to-imaps... for some details.

Soon these people are going to discover thunderbird :)

Why does history always seem to repeat itself?

I keep re-trying Thunderbird, hoping it has improved, but keep abandoning it because it keeps annoying me. Especially that embarrassing contacts importer.

But it's still the best I've found on Windows, so I keep installing it everywhere, and helping others do so as well, especially at school (as the school's setup info is about 3 versions of their email server behind).

Isn't obvious way to fix outdated setup guide is to write-edit new? (if it will cost less time to you rather than teach everyone in person)

They won't let me. Despite working for over a year in what's essentially their employee-tech-support / teach-me-how department. But yes.

Care to explain?

I have been using Thunderbird for 3-4 years at this point. I have indexing on and that sort of stuff. But searches that include the body text is remarkably slow. It does not seem to search inside attachments as well.

It would be awesome if I could fix these.

Reply on news.ycombinator.com