Blocked Sites is discontinued

After you go to that link and read the notification, there is another link to download the blocklist as a plain-text file. Mine contained a number of domains I've added during the years.

It's quite interesting to see all the domains I've blocked so far :)

Bang syntax seems such a small feature to change search engines.

[1] http://developer.yahoo.com/boss/search/

[0] http://www.yubnub.org

If you think it is hard you are either doing it wrong, or it's the wrong job for your personality.

It can sometimes be a challenge, but that is the fun part, not the hard part.

However this explains why Google seems so abrasive... they think it's hard!

Try typing a phone number into google.

Go ahead, try it.

Google has improved in some regards. I recall having to look through 3-4 pages to find a decent result in some queries a few years ago. They're usually on the first or second page now, but there are still these worthless results that end up in the 1-3 spot. That's enough evidence for me that they need expert intervention, not algorithmic guessing.

It's as if something flipped 180 degrees - they went from being the company that wanted to help you find out about everything to being the company that wanted to find everything about you.

One of the interesting things I have experienced in my time at Blekko has been that "growth" on the Web isn't really growing all that much. Sure there are trillions of pages being created but there are only so many things the few billion people in and around the Internet care about. There are 'hard information' places, which are things like libraries where reference searches are common, there are 'entity' places, be thay shops or service providers or SOMA startups, and there are "transient" places where information is current and then stale, to be stored and later reconstructed like coral into a 'dead' (in terms of change) but 'useful' base. Seeing the web from the point of view of a web scale crawler and indexer it starts to be clear that the mantra "Organize all the world's information" is getting tantalizingly close to a dynamically stable froth.

I have to believe that Google has figured this out, some of the smartest engineers I've worked with are at Blekko but Google has its share as well. When you trawl through the fishery and all you get are trash fish you start to wonder, "hmm did we actually catch all the fish there are?" So to it goes with "the Web".

I started doing some speculation [1] on how you could value information that was discoverable on the Internet. And one of the schemes I came up with is how many people would find that information "useful", where useful really means they would have some reason of seeking it out. And then scaling that value by the value to them of having it. So for example if my genome was online, there is maybe a dozen people who would find it "useful", and of that dozen probably on the insurance actuaries and perhaps the occasional researcher who would find it "valuable".

Now one takes that unit of applicability/value and scales it again by the "cost" to acquire it (find it, index it, etc). And from that you can compute the total size of the Web. Well estimate it at least. So far my upper bound (based primarily on the fact that world population is stabilizing) is about 72 trillion documents at any given time.

When you look at it that way you can see that ultimately the spammers lose. They lose because over time the actual information that rises to the useful vs cost threshold is identified and classified, or the legitimate channels that provide dynamic information, or the legitimate archival sources that provide distilled information are all, for the most part known and 99.99% of all your users can find everything they want. And as a spammer you are no longer given the free reign of "appear and be indexed" you have to ask for admittance though some form or another. And the level of new credible sources that are created is inherently a function of the number of people that exist, and the number of people that exist is stabilizing.

When Yahoo started with its human curation it was vastly better than anything anyone else could do, and then it was overwhelmed by a combination of growth and algorithms that could much more rapidly infer curation from the social signals of bookmark pages and article reference. Curation has come back into favor, and it combined with machine learning algorithms will create what is essentially a stable corpus of documents[2] known as "the web".

Wikipedia is a great analog for what is happening world wide. All the reference articles they want to put in are nearly done, the number of editors required has gone down not up, and the future of web search is, in my opinion of course, similar. The only new ground on the web is social networking and Google understands that it seems. Its fortunate for them that only deep pocketed entity that is possibly a near time threat there is Microsoft, and since to date Microsoft is trying to do exactly what they did, they benefit from having already been through that part and know exactly what Microsoft will have to do next.

[1] My side hobby is attempting to discern the economics of information.

[2] Documents are just that, pieces of information, I hardly count every page rendered by Angry Birds in a browser as a separate document, in fact applications are them selves a single "document" in the since that some number of people will seek them out, and connect/consume them over what we think of as the "web" interface.

¹ not original, but works better than Tineye, in my opinion

² granted, only for a limited number of people

Maybe if it was as simple as clicking a "never see results from this website again" it would've seen orders of magnitude more use and been far more useful. Perhaps an upvote/downvote system? Seems like these are relatively tiny time investments by a couple of engineers and would serve to massively increase the quality of results. Not sure why throwing useful features away is a better decision.