2013 Founders’ Letter | Hacker News Reader

2013 Founders’ Letter

investor.google.com

55 pointsvinhnx55 comments

Google search is an enigma to me. I oscillate from being blown away by its intelligence and accuracy to being gobsmacked by its stupidity and inefficiency. For all tech related related search terms I usually have no issues with Google, but that is in large part thanks to stackoverflow. As soon as I venture out of anything having to do with tech I find myself lost in a wilderness of content farms a million miles wide and only an inch deep. I'm pretty internet savvy so I can eventually navigate my way to half-way decent answers, but I can only imagine how frustratingly useless google must seem to people who can't recognize the tell-tale signs of junk content.

I find the most effective way to search is to already know where the most knowledgeable people gather around a specific topic and use google to search those sites specifically. I will often search within the context of content aggregators like hacker news, reddit, stackexchange, and other various forums before I rely on a naked Google search but it's a kludge at best and not something less tech savvy people are going to know how to do. And if I'm completely new to a topic it's often a chore just to even find the place where the experts actually hang out.

I think the approach that Stackexchange is taking is going to be far more valuable in terms of search over the long run than Google. Time should be spent on figuring out how to reward domain knowledge experts and making sure they stay untainted and motivated to share their knowledge rather than tweaking a search algorithm in an endless cat-and-mouse game played with content farms.

Google search was an amazing improvement when I found it in 2000, but sometimes I wonder if the post-2005 improvements of search results is nothing but a reflection of human progress digitizing data. There is more data online, and better organized than before. That improves google's results regardless of algorithmic strength.

For any search where the first result is Wikipeida, the following results don't have to be any good, and there are plenty of terms where the next 20 results are just domains with lots of in-links hosting glossary and dictionary definitions.

Tech was the first area to conduct large amounts of our work online and in the public's eye. As a result, almost all our meaningful literature, documentation, q&a (was mailing list archives, now is just stackoverflow) and "here's how I solved X" blog posts are available for google to scoop up.

I agree, Google seems to return things that I already know but cannot remember (i.e. address to my favorite restaurant), but when Im trying to research stuff I know nothing about the results that come back are less than useful.

> The search engine is working on being able to provide direct answers to questions rather than just a list of results

I dont want what they think the answer is to my question. They don't know my thought process or how I like to research information. I want raw data returned so I can sort and process it how I see fit, not them.

I agree. Why can't google just ban the content farms or let users specify it. I can list 5-10 sites that if omitted from all searches would vastly improve my results.

You used to be able to change your search settings to ban certain domains. They removed this feature and instead offloaded that functionality to a chrome extension called Personal Blocklist. It's not an ideal solution:

* It only works on chrome, obviously, because it's a chrome extension. That means you will continue to get crappy results on all your other browsers and devices.

* It modifies the links after you load the page so you see them disappear which can be disorienting. I really don't want those search results even making it into the DOM.

* As of now I'm not able to even edit my list (the options menu item is grayed out) and I have no idea why. The quality of the extension is poor at best.

* It has zero intelligence. What would be ideal is that I could subscribe to a blocklist of well known content farms and allow crowdsourcing to maintain and add to that list.

The clear message from Google is that it doesn't really want to support or encourage wholesale blocking of any domains. Google would prefer to be the entity that makes that decision for you, and since these content farms make Google a lot of money don't hold your breath that they're going to start cracking down in a serious way anytime soon.

Perhaps Google doesn't want a robust blocking system because it would set off a cat-and-mouse game with the content farms continuously evolving to evade blocking? As they are, the content farms are annoying, but they're a relatively stable entity.

It would get harder for Google to crawl new content on the internet because they'd be filtering through progressively more clever and aggressive farms to find real pages. I'm sure is already the case, but aggressive blocking would provoke farms into getting worse.

The highest earning form of advertising for a content farm is Google. The fundamental concept of a content farm is to have mountains of content about any possible topic. The only ad network with inventory deep enough to cover this is Google. The difference being single digit CPMs verse triple digit.

When Google created Adsense it built an ecosystem. While other search engines lost users as soon as they clicked on a link, Google had a fairly high percentage chance that user still made them money.

I don't know if the future looks much better. What Larry Page is saying that instead of display a link of Stack Overflow results he would like to scrape the answer to the query in a way the user never has to go there. The way they will make money from advertising with that model looks even worse.

Greasemonkey on Firefox has a script called "Google Hit Hider by Domain" [1]. The list of blocked sites is not easily edited, but it consistently works. Its overall utility for me is still pretty high and it has a well-polished interface.

[1] http://userscripts.org/scripts/show/95205 (looks userscripts.org is down right now)

> What would be ideal is that I could subscribe to a blocklist of well known content farms and allow crowdsourcing to maintain and add to that list.

You might be on to something here with censoring as a service

Yeah, like a wikipedia for search results! I wonder if anyone has tried that.

It would actually be pretty simple. A search bar. If you search doesn't exist, you're prompted to create a new page and add in whatever links you've found. Users could also vote to merge searches, etc.

You'd need some major fraud detection of course but I think it would be possible.

Anyone interested in working on that?

Wait a second, isn't that doing the same thing that's already automated via combination of pagerank and what google does with chrome and analytics? And for that matter isn't 'censoring as a service' what a search engine does in the first place?

I think that was the dream. But people are complaining about google search. Maybe it's time to try something new like crowdsourced search results.

Also we almost need a search engine that has reverse SEO rules. For example if my search term is in the domain name I can actually be pretty sure it's not going to be valuable content.

However google actually rewards that.

Google places much less emphasis on domain names now.

http://webmasters.stackexchange.com/questions/53703/choosing...

Actually Google still LOVES EMDs in local.

Check several competitive markets in bigger cities. Examples: City Dentist, City Personal Injury Attorney, City Chiropractor. For many the top 3 - 5 are EMDs.

Google just reduced the weight of EMDs, so if thin or spammy content and all they have going for them is the EMD, they won't rank. But an EMD with good content and marketing still ranks well in local.

wow, didn't know that. So does that mean these days it's worth buying an aged domain?

I couldn't tell you off the top of my head. Not necessarily a SEO expert, just recently read about the domain exact match change.

Meh. A lot of the problem with modern search is just that a lot of topics just don't have that many people who are both web-savvy and topic-savvy. Content farmers can be slain in the rankings easily by a superior resource, but that resource has to be attached to some sort of business engine to make it worthwhile enough to create. The hobbyist auto mechanic has less of an incentive to write great articles about auto maintenance than the auto parts corporation does.

Adwords is also excellent, but the skill to use it is not well distributed. Advertisers have more obvious incentives and measurement metrics (return on total ad spent / return on investment) than hobbyists do.

I completely agree. I built myself a metasearch engine that uses StackExchange and other sources in parallel; for example: http://gigglebang.com/?q=postgres+replication+monitoring

It's really basic, but I end up starting most of my searches there, and fallback to Google with a DDG-style !g if I need to.

There must be a massive feedback loop at play. A site that is popular on Google becomes more popular because it is well placed on Google and will then be placed highly on Google as a result. Can a search engine really work when the search engine itself is the greatest decider of quality? This must cause unintended consequences.

But Google is more efficient than the specific StackExchange (et al) search if you restrict the query to that site. I am repeating myself but I think one way to compete with Google is making local search engines work better than Google itself restricted by that domain. The rant continue on Challenging Google’s Search Engine [1].

- [1] http://blog.databigbang.com/letters-from-the-future-challeng...

That's what I meant by "I will often search within the context of content aggregators like hacker news, reddit, stackexchange, and other various forums before I rely on a naked Google search". I add site:reddit.com or site:stackexchange.com to my google search query.

Google is a good search engine so such the question that you query has been asked on Stackoverflow. Otherwise you will go nowhere... And the way they evaluate the "quality" of a site by a simple mathematical formula is a joke. We have the same feeling that Larry about the search engine!!!

I think the one of Larry Page's dreams is very different to most people's dreams, which would be one that respects people's right to privacy.

But you still want Google to deliver useful results.

Let's say I'm in a strange town on a business trip. I've got to eat dinner. I ask Google for a good restaurant in my area. It can just give me a list of all the restaurants nearby, and that's quite useful. But it's tedious sorting through all the ones that I would never consider eating at.

Or, Google could know something about my eating habits, and start off with the restaurants that might actually interest me. That's a more useful result. But to get there, Google had to know some things about me.

Now, may want to be able to choose that tradeoff; certainly Google could do better at giving you controls for this. But the point remains: If you want results that are more useful to you, Google needs to know some things about you.

I think you're being slightly overdramatic. I too wish that Google was more transparent with how they farm our data. But you do have a choice to stop using their products you know

Actually it's almost impossible for a regular user to avoid using Google products any more, e.g.:

- Over 63% of the top 100k web sites embed their web bug ( http://smerity.com/cs205_ga/ )

- I have no data, but a huge proportion of web sites use Recapcha in their sign up process. It's impossible to use some domain registrars without revealing the IP address of the probable domain registrant

- I cannot host a web site without Chrome or Google DNS donating some huge percentage of data about my visitors to Google. I'm not sure the mechanism employed here, but I've set up 2 private web sites in the past year where upon having a Chrome user visit, Googlebot turned up within a day or so, despite all efforts to ensure the URL remained private

- I cannot visit Reddit and many similar sites with a cold web browser cache without revealing my use of the web site to Google, thanks to their AJAX library hosting

- A technical user has no difficulty blocking Urchin, but e-mail is a different issue entirely. I cannot find the article now, but one user reported 40% of all mail in his inbox passed through Google's servers at some point, even though he wasn't a Google user.

On the average day, whether I like it or not (and really I don't), I generate tens to hundreds of new records in Google's databases.

> - I cannot visit Reddit and many similar sites with a cold web browser cache without revealing my use of the web site to Google, thanks to their AJAX library hosting

Not disagreeing with anything, but in case you didn't know, you can actually disable this in your reddit preferences: https://ssl.reddit.com/prefs/

Choose "load core JS libraries from reddit servers" under Privacy Options.

Only if you're logged in, sadly :) Privacy centric folk like myself often have the browser clear all cookies at exit

I submitted a Firefox addon for approval a few years back that introduced a separate cache for all the AJAX libs, but it was rejected for silly reasons

> submitted a Firefox addon for approval a few years back that introduced a separate cache for all the AJAX libs, but it was rejected for silly reasons

I have actually been looking for such an addon. I too clear cookies/cache/etc on exit in addition to using NoScript and a strict Adblock Edge (block social networks/tracking/not just ads). The only area remaining to be fixed would be the AJAX libs to be hosted locally.

Would you consider making a github page or something so you can provide the addon without being in the firefox addon explorer?

I've uploaded what I had to https://github.com/no2weblog/no2weblog/ , but it needs a bit of work to return it to a useful state.. at the very least, updating to most recent versions of the cached libraries.

One of the major complaints in the rejection mail (included) was that the add-on bundled copies of all those libraries. So it seems reworking it to have a dynamic list of URLs, and fetching these once on first boot would go some way towards solving that. Unfortunately this further complicates how the redirection works, so I never got around to investigating it

I tried it and visited css-tricks.com and when using the 'Live HTTP Headers' it still makes the request to Google with the referer. However, the Firebug net inspection appears to locally load it (0ms load time).

Most users are not technically savvy enough to have the choice to stop 'using' Google Analytics. Is it really possible to exercise a choice about something when you don't even know it exists?

On the issue of Google (and especially G+) and privacy, if one is only being even medium-dramatic, that's hardly a criticism.

Not really, Google has most emails to and from me regardless of whether I use Gmail myself

I don't think people realize how much better Google search has gotten over the years. We tend to remember the annoyances or failures, and not remember all of the successes that are second nature to us.

Consider vague long tail queries. Google knocks these out of the park compared to Bing, DDG or others. Let's say there's some old movie you vaguely remember details about:

"movie where evil is in microwave" => Google "Time Bandits". Bing: Nada. DDG: Nada

"dystopian movie with a motorcycle named einstein" => Google "Warrior of the Lost World" Bing: wrong, DDG: wrong

"song about cocaine in california" => Google: Hotel California, Bing, DDG: bottom of page

"star trek episode where spock is possessed" => Google: Return to Tomorrow, Bing/DDG: Spock's Brain (wrong), but nice try"

"previous name of java language" => Google: link to Oak as second item, Bing/DDG: worse result (must click through links to find answer)

"cartoon about plants and a kid with a ring" => Google: Jayce and the Wheeled Warriors, Bing/DDG: Nope

"tv show with scientist named blackwood" => This one is EASY, Google: War of the Worlds, Bing/DDG: Nope

"guy who tried to bomb parliament" => Google: Direct Answer, Others: links

"tv show with alien who has necklace powered by sun" => Google: 1st result, The Phoenix, Others: 3rd or lower

"cartoon car that has auto jacks" => Google: Speed Racer 2nd result, Others: nope

"flying characters similar to thundercats" => Google: Silverhawks, Others: nope

"movie with guy who owns last car" => Google: The Last Race, Others: nope

"commodore 64 game where alien knocks on window" => Google: Rescue on Fractalus, Others: nope

"toy where you program trailer to dump" => Google: Big Trak, Others: nope

You can play this game all day, trying to come up with the vaguest possible query about old things you remember bits and pieces of from decades ago, and I'm often amazed at how vague and obtuse I can get. Google does these queries much better than others. Yes, it fails a lot too, but the instances where it fails, and others succeed are much more rare than the instances where it succeeds and others fail.

But Google isn't an AI yet. And what Larry is speaking about is achieving Star Trek: a computer that can read the meaning of words, understand, rather than just index characters.

Yeah, OK...but they increasingly fail on the stuff I actually search for more than once every few months. Even relatively mundane stuff -- like plant care, for example -- has been completely destroyed by SEO gamers.

That one just comes to mind because it seems like once a month I search for something related to houseplants, and I almost never get good information without having to extensively refine the search, even though the internet is loaded with high-quality sites about plant care, botany, etc.

Ya but some of their other services have dropped in quality, re: google maps http://techcrunch.com/2014/05/10/google-maps-has-forsaken-us...

I think that conflates two things. The Maps Front End, and the Point of Interest database. Both the classic front end and new front end probably use the same DB. So if the query failed on the new UI, it probably failed on the old one, he just never noticed. Is there any proof that "Sambisa Forest" actually ever worked? Otherwise, it is not a regression.

Google apparently has a "Sambisa Forest Reserve" in it's database, but no "Sambisa Forest"

We changed the url from [1], which just copies bits of the original. We also changed the title from the linkbaity (and ambiguous) [2].

Submitters: please do your due diligence and read what you post. An article of cherry-picked excerpts from a more original source is rarely a good HN submission. You should submit the original source instead.

1. http://www.zdnet.com/google-a-million-miles-away-from-creati...

2. ‘Google "a million miles away from creating the search engine of my dreams"’

I'm nostalgic for the days I could click on my Google bookmark and start typing my search without the page missing my few first keystrokes because of slow loading.

At first, I read this as an imperative, and tried googling the phrase, "a million miles away from the search engine of my dreams".

You're not the only one, buddy.

As a search/deep learning engineer I am amazed at google, I think that the more you know about the subject, the more appreciative one becomes. Take, for example, their iOS search app. You just say "ok google" then often before I have even finished my question the results are shown, in that time it did continuous speech recognition, query parsing, intent detection, search, result generation and render on the device. That requires some very serious smarts. All so I can show "show me pictures of c[ats]", results in at "c"...

That's because Google is in the business of adwords, advertising, robots, eyewear & building servers/computers/clusters, not search.

A billion Android devices. That's truly remarkable.

100% agreed. It was a very clever play by a search engine company.

What intrigues me about the Android-equation and the way it entered Googles' playbook is that, indeed, the game is not over. Any other young upstart can arise, almost immediately, and challenge the hegemony .. all you have to do is be willing to commit to a real platform strategy.

I believe the Android vs. iOS challenge for the mainstream is a very vibrant industrial action; echoes of this gameplay can be definitely observed, elsewhere in the F/OSS eco-system. A fundamental difference between iOS/Android: one is closed, the other is open. A new, open challenge to Android could indeed eat its lunch, if the open challenge were won in such a way that it would attract the hunger-economy of the giants.

I personally would like to just have a phone that is 100% open source. I've had both Android/iOS in my life, equally, since inception, and ultimately I find the industrial-baggage load to be pretty high. In the new NSA era, what the hell is the point of trying to hide anything any more .. the new cool is completely open.

I bet it would solve the immense-cruft problem we can easily witness in heft with iOS/Android, as both platforms NIH/NIMBY their way across the 'developer mind-control' table.

Imagine this: someone builds a challenger OS, for free, that runs on everyones' phone, regardless of manufacturer-lockout/lockin. Methinks there are ways to do this under the radar right now ..

Only if you think most of them are Samsung Galaxies or Moto X's. Most of them are just replacements for the low end Nokia's that used to be predominant. A billion cheap phones is not a new statistic.

Well you have to take into consideration that most of the population of this world can't afford to buy a smartphone.

Then you have to think that in 3rd world countries acquiring a smartphone means success... and of course that smartphone is going to run on android.

Stop thinking that Europe + America + Australia have more population than the rest of the world :)

What exactly are you arguing with and why are you telling me to stop thinking things? That sounds rude.

I don't disagree with any of the statements you have just made, and nothing I said in my previous comment contradicts that - so why the attitude?

Ok sorry for my wording. English is not my first language. I wasn't trying to be rude. You are right that the way I wrote the comment sounded a bit assaulting. Sorry about that.

The perfect search engine is fundamentally devastating to Google's business model.

The perfect search engine would always take you to exactly what you were looking for immediately.

Which means, no amount of advertising would ever be useful, since you wouldn't need to be advertised to - you'd type what you wanted and it would show you exactly that.

This is sort of like saying the perfect phone is fundamentally devastating to Apple's business model since it would never break and you'd never need another one. Both situations might some day arise, but until then both Google and Apple have great incentive to improve their core products.

It annoys me that it doesn't search users blogs and things that well. I'm often searching for what real people have to say, not content farms etc I know spam blogs are the cause, but the results are still frustrating.

AI can change the world. I guess this is Page million miles away goal.

Reply on news.ycombinator.com