Google: Hide sites to find more of what you want
googleblog.blogspot.com
googleblog.blogspot.com
I notice that when I block a subdomain (eg, http://answers.yahoo.com/ ) the page actually shows the entire domain (yahoo.com) as blocked.
Is this just an error in the display, or does it actually only block based on domain? If the second, this significantly limits the usability, since I can't block < http://someobviouslinkspam.blogspot.com > without also blocking every blogspot.com site.
Edit: Clicking on "show details" it says that answers.yahoo.com is blocked, so maybe they're only showing the base domain here. Still weird.
On a more serious note though, its nice to see Google validate our assertion, that un-modified Google search results are getting poorer and poorer. Not that they would actually say it directly like that of course. Now lets see if they are willing to drop over a million crappy sites out of their index ...
So you go to start a company and you say that you believe the search experience is getting worse and worse. That you believe the leader in the market place is fighting a losing battle of trying to algorithmically determine relevance in a world where humans will actively work to subvert the algorithms. That the world would be a better place if a more customizable search experience existed.
A common response is "Well if this is such an important idea they would already be doing it."
So as an entrepreneur you deal with the naysayers and you build something which other people have "... been clamoring for a blacklist for years" and yet has never materialized. And you put it out into the market.
Right up until you deliver your product, nobody knows who is 'right' in this back and forth. It is all academic. You've built the best product you know how to build, you offer it up to the world, and when you ship the debate stops being academic and starts becoming empirical.
When the market leader adds features that mimic ones you've launched with several months ago, it can be simply timing (they may have been going to do it all along and it just happens that they did it now), it can be a response, it can be random I suppose. Regardless of why they did it, it seems to be an unequivocal damnation of pure algorithmic search.
As one of Blekko's founding concepts was that the search experience can be made better by human curation, I choose to interpret it as validation. Not that I expect Google to join in and adopt the "web search bill of rights" anytime soon of course.
Similarly, I don't see the ability to choose whether you're doing an image search or a news search as a condemnation of algorithmic search either.
Now they have backed off on that, with reasoning that it is really not truly possible given that the search "Sony XBR TV" might be you are looking for the manual for the TV or to buy the TV or something which isn't known, but the goal is always to have the result you're looking for on the first page. That has been their target and promise for years.
So by creating a feature, and putting it into production, to "block sites" it suggests to me that they are saying "Hey, we know our algorithm is failing and we have given up (perhaps temporarily) trying to fix it. We know it will put sites you don't want to see into the first page. Here is a manual tool to weed those out."
It seems like something of a philosophy change at least. Perhaps Matt will chime in.
Look at this search - "how to get pregnant" - that exact phrase gets 550K searches on google a month. My wife and I did this search last week actually.
Blekko's top 3 results are 1. get-pregnant-guide.com which is an affiliate site that points to pregnancymiracle.com - one of those crappy ebook sites where they have a bunch of testimonials and ask for you to pay $39.95 for their guide, 2. a digg post that has 1 digg which points to a hubpages article that no longer exists, 3. purelyfitness.com/how-to-get-pregnant which is also an affiliate site that points to pregnancymiracle.com.
Google's top sites are the mayoclinic.com, mahalo.com, and howtogetpregnant.net. The least useful to me was howtogetpregnant.net, but they still gave a decent amount free advice. I actually found Mahalo to be the most informative which is blocked from your results. I find Calacanis just as annoying as everyone else, but this page was more beneficial to me than even the mayoclinic.
I'm not sure if "Human curation" is the concept I'd grope for here.
What I'd like is search that uses a "larger context" - what I've searched-for and accepted/rejected, what those similar to me have searched-for and accepted/rejected etc. Being able to implicitly use a lot of information is crucial because there aren't going to be enough curators to everything ever.
Google is already using an incredible amount of implicit information and I expect them to use more.
What I see opportunity as really, is more like "self-curation" than some finite stable of curators. What would be cool is a system to let me keep my list of likes and search-finds - essentially my entire preference-system - and let me edit and send the whole thing to different search providers as well as to friends.
Personally I prefer knowing that those sites are in the index, but don't show up unless I purposefully widen my search to include them (or they appear so low in rankings that it achieves the same effect).
"In addition, while we’re not currently using the domains people block as a signal in ranking, we’ll look at the data and see whether it would be useful as we continue to evaluate and improve our search results in the future."
Sure, but these are minor updates. Look at the major updates. Windows Vista was developed for about 5 years in secret, and when it was eventually released to the world it was a disaster. To keep with the OS comparison, Google has been releasing major OS upgrades every 6 months - 1 year. Android is moving so much faster than Windows or even WP7 and is a huge part of why it's successful. In the time that WP7 has been out (five months? ish?) there have been 0 updates to most phones; maybe a few phones got the small update-to-the-updater. With that same period of time Google has announced and released a new Android version.
Anyway, all I was really trying to say is that Google is, in general, faster to react to market changes, faster to push updates, and faster in developing new features when compared to Microsoft. And that's one of their advantages. Sorry if my specific examples became the primary discussion here, I really only meant the Windows comparison to be an example showing a larger strategy.
It might be the case that different types of products have different release cycles. Not strictly though... IE is way slower than Chrome to rev.
The magical optimization I would prefer would be a non-commercial search. If I search for a piece of gear sometimes I don't want to buy it and instead want to weigh buying it or just look up reference information. For some searches that is tough, and permanently blocking commercial sites isn't an option. (I've occassionally resorted to limiting my searches to .edu and .org domains with limited success). Even temporarily blocking commercial sites might not help, though, since sites like Amazon.com have fantastic reviews on some items.
Crowd-sourced curation -- this must be Google's eventual goal for the "blocking" feature. Can't content farms be thought of as spam? And, if so, don't the same techniques for spam identification apply, especially the "mark as spam" button in Gmail?
[Conjecture and speculation below as there's no evidence Google is going to modify search results based on users' blocking propensities, but I think the possibilities are worthy of consideration.]
I wonder how this scheme could be gamed? Consider how one might game spam filters to cause a target's future mailings to end up marked as spam. To hit a high-volume target, one would have to cause a ton of email addresses under his control to be placed on the senders distribution list. Then, when messages arrived, he would mark them all as spam. Although the total percentage of the sender's mailing list comprised of the bad guy's email addresses might be small, this technique would still be effective because the percentage of "activist" recipients required to flag any sender as a spammer is so low (at least I'm told). To address this problem, Google could certainly apply schemes similar to those used to identify click fraud. Such countermeasures would require the bad guy to make his attack appear more organic. Have there been actual instances of such denial-of-service attacks?
With a similar strategy, could, say, an anti-abortion group enlist its membership to banish pro-abortion groups' websites from search results? What if members were instructed to search for phrases like "How to get an abortion" and block Planned Parenthood, etc.? The difference between websites and email lists is that the owner of an email list has direct control and could take countermeasures like changing mail servers, etc. Also, large bulk mailers (MailChimp, etc.) generally have good relationships with large email box providers like Google and can therefore plead their clients' cases directly to a responsible person. How would pro-abortion groups get attention from Google if their site rankings dropped due to such grassroots organizing? Would Google be able to identify this as an orchestrated effort?
How would the behavioral patterns of pro/anti-abortion activists be any different than this forum's treatment of Experts Exchange? Google will observe that one small segment of its userbase blocks the site in high numbers but another segment, when searching for the same terms (MySQL failure XYZ) click on Experts Exchange results? Wouldn't users actually seeking information on how to get an abortion recognize the name "Planned Parenthood" and click on those same results which were blocked by activists?
I want a one-click way to ban a domain from search engine results because most blacklisted domains in my Wikipedia search engine (http://www.google.com/cse/home?cx=009114923999563836576:1eor...) are porn or filesharing sites. One recognizes such spam in an instant, but it still takes a while to prune down the URL to the right domain, flip to the edit tab, paste it in, flip back, and relocate myself.
I'm not kidding when I say such a one-click button would cut by at least half the time I have to put into cleaning up the CSE results for any given query.
Sadly, it does not. Would have been neat though.
More news from:
NPR
The Economist
Wired News
FOXNews
FOXSports.com
Wall Street Journal
For example, let's say someone has suggested to me to use the factory pattern to solve a problem in a project and I'm not intimately familiar with that pattern. I search for "Factory design pattern" on Google and notice that the first 2 results are wikipedia results. There's some good, basic boiler plate--but that's not what I need (Ok, what I really need is probably a trusty copy of the "Gang of Four" so maybe it's a bad--or at least contrived example).
So can I disable the chrome plugin now? Will it remember the sites I have already blocked?
Note you can only block 500 sites, I should think thats enough though.
Also, the sites that have already been blocked with the extension, will they be auto-blocked?
But in the meanwhile good riddance Mahalo, eHow, Yahoo Answers and ExpertSExchange, etc
As said in another comment I wouldn't be confortable to block the whole expertsexchange.com domain, I get revelant results sometimes, yet some of the results of the same website are so unhelpful I'm sure I don't want to see them again (auto approved solutions and stuff like that).
It also happens when I search for something I already searched for a while back. Some of the results remain irrevelant, but not necessarily the whole domain.
But I'm no Matt Cutts.
Btw, is the link showing up for everybody? It is not for me.
- Yahoo! Answers
- Ezine Articles
- Those Efreedom assholes
- Experts Exchange (your experience aside, this place is crap for most everyone else)
- All the junk sites around appliance manuals
- All the other content farms
I wish we could give more contextual feedback, like... THIS link was helpful/relevant, THIS link was not.
They aren't using these blacklisting as a signal for influencing the google algo, YET... but it seems inevitable.
I'm just trying to say it seems a little too much like an axe instead of a scalpel.
Personally I can't wait to start rage blocking domains that have pop over modals for surveys / advertising, or have sound enabled by default.
OTOH, I imagine that bing will also eventually succumb to content farms and other techniques that will come up, so maybe this is the way of the future.
The conspiracy theorist in me wonders if in a few months users are mysteriously going to have facebook.com added to their block list.
The '-' operator has worked wonders for me for years.
(And to head off the obvious karma steal at the pass - Because sometimes the answer isn't in stackoverflow yet. Duh)
I hope it's propagated to international Google pages soon.