Hacker News Reader

Comment by mayank | Hacker News Reader

Search is still up and snappy! I may be biased as an ex-search Xoogler, but Search's resilience is a testament to how battle-hardened a stack it is, considering GDocs and Gmail appear to be hard down.

View on HN

As a present search Googler, it's more of a testament to how easy it is to run a service that doesn't need to accept transactional writes from users, contains little or no private data, and where >1% taint is acceptable.

Search ads require transactional writes. But actually, as far as I remember big outages at Google in the past have been things like "someone pushed the wrong router config" and not "our database got overloaded". Maybe search is just more conservative?

Search ads is a completely different system than search. They don't share tech stacks or reporting chains.

Do ads truly require transactional writes though, rather than eventually consistent log-append writes?

I have the same question. I suppose ads billing might have transactional requirements, but that's separate from search itself.

I worked at Bing/Yahoo.

There are two mechanisms helping keep search up when considering ads:

-> Search engine pages can be served w/ no ads, or even no personalization if components are down or lagging

-> Ad logs can be lost if necessary, since we'd under-bill in that case, which would be our own problem. That happened a small handful of times while I was there.

The billing system is indeed transactional, but has less need for 100% up-time and 0 latency in comparison to search. Not nearly as big of a deal if there's an hour delay on billing ads as compared to search being down for an hour.

Glad to see at least some folks in Search understand this :)

Sorry, what is taint in this context?

Probably, missed elements

Yep, that's it.

No private data? Isn’t search personalized?

I can't really comment on that, except to say that the way it works is a much easier problem than how gmail works.

Big talk! Must be nice being able to serve 99% of the index and not have anyone notice. If the gmail threadlist comes up with 1 missing message everyone will lose their minds. And obviously if user authentication and homing is down then search can serve public docs but gmail etc just can't serve.

I was getting HTTP 500 errors when clicking the search results. The search itself was working, but I couldn't follow the links.

It annoys me a lot that Google doesn't use the "real" URL in their search result links.

Isn't search more or less static? Docs and mail change instantly and users expect instant feedback

Search handles probably an order of magnitude more changes per second than GMail+Docs combined

But if the index doesn't update you won't notice.

Yeah, search isn’t an OLTP system.

I pretty strongly doubt that the arrival rate of new docs at search is an order of magnitude greater than that at Gmail.

Offtopic, but it blows my mind seeing <1 min old StackOverflow comments indexed by Google search, the sheer amount of work going on behind the scenes to make that happen.

I'm pretty sure this only happens when the site is actively pushing updates to google via their webmaster tools. I think your sites importance would also impact how fast google indexes it.

Didn't realize this was a thing: https://developers.google.com/search/docs/guides/submit-URLs

Although I'm not sure if this is what stack exchange actually uses or if there's some private API for big partners.

Search results were either failing or taking a few minutes to load in Hong Kong this morning

Does search actually uses the same platform as the rest of Google?

From what I understand, "mostly". Depends on what you mean by "platform". A couple key differences--search runs at a higher level of redundancy than most other services. And, importantly, it can run very well at reduced capacity.

For example, if you type a search in the homepage or address bar, you start getting suggestions instantaneously. Think about this... if Google decided to only give you suggestions every second keypress, or every third keypress, would you really notice?

Moreover, writes are hard, reads are easy, and everywhere that a write needs to happen when a user does a search can fail gracefully. If this search doesn’t end up in your search history, or doesn’t show an ad, or doesn’t contribute to your future personalization then oh well, you’ll never notice.

Reply on news.ycombinator.com