How Ray Kurzweil Plans To Revolutionize Search At Google
forbes.com
forbes.com
Google search returns approximate results, related results, results with your words split (or joined) and other nonsense to drive ad views.
Even using the so-called "power tools" like allinsite: and the (barely functional) quotation marks, you still get very shoddy results.
Anyone who ever searches for very specific groups of terms knows exactly what I am talking about: enter search terms, click on result, search in page for term, doesn't exist.
Edit: Thanks, comex. Made google.com become: http://www.google.com/webhp?tbs=li:1
Search is too big a problem for just a couple people at Mountain View to be working on. And at this stage with the amount of data being generated, no one really has the infrastructure to compete.
Hope they do it before the regulators make them. If Apple has succeeded at building a market place around their closed platform Google can too.
Another example is targeted search. For example, if I'm searching about mental health stuff I do want good quality sources, I don't want tabloid gossip about celebrities going into rehab.
Or sometimes I want to break out of the SEO trash, and have a bit of serendipity. Search for a term like [spectacles cases]. You get a lot of shops selling pretty much identical cases. What you want is a nicer way to preview those shops (because often the websites are god-awful and their own site searches are much worse than anything Google provides.) WAIT: I just tried this to make sure I was right, and Google have changed the way they show results like this. You get the same ads at the top, with heavily SEOd content links below, but now at top right there's a "shopping" section with links to different cases in different shops. So, that's much better now than it used to be.
Still, serendipity is fun. I remember when you used to be able to use Google to noodle around and find cool stuff. Now? Not so much. That's not Google's fault. The modern web is very different to what it used to be, but it'd be great if there was some way to get access to those smaller sites.
Just look at their menu bar...images, videos, flights, blogs, shopping, books, patents, apps.
Is that it?
Not to mention random. So we just sit around waiting for some benevolent god in Mountain View to say, you know what now let the mortals have...recipes.
If they want to expand that list to the infinite domains it should be covering, it is never going to happen with the resources they have. They need to open the index to tap into its full potential.
Identifying what a recipe looks like and then providing a search interface that can figure out which of the millions of variations of some soup recipe is what a person is looking for and is more authoritative than others (and not some blog spam with minor (but random) alterations, or written by an amateur with no business in the kitchen) is a hard problem. Crawling isn't really the hard part. It takes a lot of hardware and time, but then you have all this data...that's when the hard part starts.
I'm interested in what others think a useful "index" API would look like, though.
I find Google Search to be incredibly frustrating to use because it chooses to ignore search terms even when pages exist with those terms. Google better be really careful because crippling the experience in order to sell more advertising reminds me why I switched away from Altavista.
1) The web is just much bigger and noisier than it ever was. There are so many SEO-bait sites out there now, it is a wonder search still works at all. I can't really fault Google for this part.
2) Changes Google made to make search more accessible to the mainstream user. Google search now tries way too hard to be "smart" about what the user meant to ask for instead of what he or she actually asked for and this can be a huge negative if the person is looking up precise information and knows how to use a search engine.
They somewhat recently added a "Verbatim" option to search that can help you avoid some of this too-smartness, but even with that enabled Google is still inferior to what it used to be when I'm looking for very targeted technical information that I am sure exists out there.
Sadly, this sort of thing is a trend impacting not just Google. The success of Apple has created a culture of creating things for the mainstream consumer user which often comes at the expense of the power user. I get why this is done and ultimately it is the right call for any business that could potentially serve the mainstream, but I do wish more companies would leave in the highly technical expert options as settings for those who are comfortable using them. I feel that in recent times most software in general has swung way too far on the pendulum from being too hard for normal people to use to being totally gimped for experts and feeling like a toy more than a tool and I wish attempts to try to support both sets of users became a "thing" instead of constantly hearing the mantra that "options or settings are bad, no options or settings for you"!
Thankfully you can still tell it to use Google.com and in english.
Sadly, you are in a minority. Google's results are excellent for most people.
I used to use + often. We know that the + operator was rarely used, and most of those times it wasn't used correctly. (Of all searches, only 1 in 600 were correctly using the + operator.)
Being outside Google (and not able to see their data) is frustrating, but they have a lot of numbers and they do this stuff because they can show it helps most people.
Sadly, you are in a minority. Bing's results are excellent for most people. Slap a Google logo and I'm willing to bet that most people wouldn't tell the difference. It's the Google brand that is golden, at least so far. But lately people are talking and questioning their honesty.
[1] http://blog.paulnshapiro.com/bingiton-google-wins/
[2] http://www.punditpress.com/2013/02/the-bing-challenge-yes-it...
You may want to check this http://searchengineland.com/figz/wp-content/seloads/2013/04/... from http://searchengineland.com/users-prefer-google-even-when-15... It's impartial, afaik, and almost 70% like Bing results with the Google logo vs around 30% o Google results with the Bing logo. Amazing since that case shows that people liked the Bing results and had their preferred company's logo on Google. (Not sure if Bing had anything to do with that study though)
Either way, if Bing narrowed the preference to 10%-20%, it's a great feat. Personally I find Bing lacking on obscure terms but most people wouldn't care.
In my sentence "Google's results are excellent for most people." I could have said "Modern web search engine results are excellent for most people" - I didn't because this is a discussion of how Google (and only Google) is failing for a number of people.
As has been mentioned, you should probably set your default search to verbatim mode.
Search for "vertx play framework" and the last two search results don't have the term vertx despite pages and pages of results with the term. I don't understand how dropping search terms can make the results more relevant.
Also you can't set the default search to verbatim. You can only hack the query parameters in specific browsers.
Fair enough. Like I said, it can certainly be improved, and perhaps recent changes have gone too far.
> Also you can't set the default search to verbatim. You can only hack the query parameters in specific browsers.
Don't the kind of people that frequent this site near universally use their browsers to search, rather than wasting keypresses navigating to google.com?
Ridiculously, Safari, which I use, requires an extension to use an arbitrary search engine, but I can't think of another browser that doesn't support it.
Not 100% proof but ad views and clicks seem to be increasing by 20% to 30% each quarter. Google tests everything to death and they please wall st with those ad click increases.
Besides, without any other data or even a clear mechanism of action, you could just as easily say they're just better achieving their stated goal of only showing ads that the people who see them are interested in seeing. That's supposed to be one of the benefits of the auction-based approach of adwords. 20-30% is a lot for that, but who knows?
Really, I think you're seeing a pattern there that you want to see, and I'll second comex's call for real evidence.
But growth in places where Google makes its real money (US, EU, Canada etc) has plateaued for a while, IIRC. Growth in ad clicks mirrors growth in revenue and we know a click from Guatemala is not the same as one from NYC.
>>Besides, without any other data or even a clear mechanism of action, you could just as easily say they're just better achieving their stated goal of only showing ads that the people who see them are interested in seeing.
Maybe, but if you read the comments, people are accusing Google of making results worst to boost the ad click rate. That's wrong on so many levels.
Actually, average cost per click has been going down for a while now. I don't see any breakdown by geography, and wall street had been all worried about what that means for an ever-more-mobile world, but, again, the data isn't there to back up your supposition. A large part of it could be growth in regions that bring in lower ad revenue.
> Maybe, but if you read the comments, people are accusing Google of making results worst to boost the ad click rate. That's wrong on so many levels.
Er, what is wrong on so many levels? I can't tell what you mean from that sentence construction. You say, "maybe", so you don't disagree with me, but you backed up the only other "accuser" in your post above, so I don't think you disagree with him...
In any case, the considerably more obvious and likely explanation is the one I think most people have tended to assume: google disambiguates terms automatically because it's what works in the 90% case. It's annoying for power users, yes (even back when it was less of an issue, at least we could +terms, so I miss +ing terms like crazy with today's google), but it makes no sense to attempt to optimize for ad revenue that way. Extraordinary claims, extraordinary evidence...or in this case, any evidence.
Google's revenue is rising by what % a year?
Google's user base or searches are increasing by what % a year?
Is there a difference and what makes that difference?
Google's recent stock growth and revenue also match Google's highly publicizes Panda and Penguin updates. Or supposedly fighting "spam" but in reality also ruined a lot of small businesses. My educated guess is that Google is sending a lot less "free clicks" to other sites.
Cost per click can be brought down by a supply gut or a poor ROI for the advertiser.
>> In any case, the considerably more obvious and likely explanation is the one I think most people have tended to assume: google disambiguates terms automatically because it's what works in the 90% case.
In any case, the considerably more obvious and likely explanation is the one I think most people have tended to assume: Google makes sure that any algorithm changes (at least) don't hurt their Adwords business. That's their bread, butter and dessert.
So I consider this a feature not a bug...
Alta Vista years ago had full logical search but Google beat them with simpler searches having greater relevance. Google's searches are now moving towards even less specificity. How much could greater language understanding help this?
How many people want to go to the trouble of writing out a full sentence to specify exactly what they want? How much is it worth accommodating them? etc
I think Google (and Larry Page) is betting that they will be able to separate out this mixture.
One thing about good ideas - is not so much the novelty, but more-so ideas that are far enough out there that they are semi-novel, but based on an extrapolation of the current state of things such that a path to getting the idea implementable is possible.
Sometimes, this means that you're able to foresee the trajectory of industries and technology that others miss or can't.
However - its also important to have the resources avaialble to be able to implement/execute/foster that which is needed to make those ideas become reality.
In this case, he has immense resources which can make the ratio between rubbish and great ideas shift significantly.
Its easy to simply be a daydreamer - even if a greatly informed day dreamer - when you have no resources.
For one, Kurzweill DOES have "grand" ideas. Extravagant visions of a future with technological immortality, singularity, etc. Hofstadter does not. He merely examines some things, like cognition, and proposes some theories about their workings. Like, you know, every scientist.
Kurzweill comes out as grandioze, obsessive and deluded, Hofstadter like a normal writer, no more or less strange than, say, Marvin Minsky.
Pragmatically speaking, Kurzweil is miles ahead of Hofstadter in terms of putting his theories and ideas to practice.
It's very easy to criticize visionaries, until they achieve something. Then nobody holds the critics accountable for their negative attitudes towards the visions, and the critics probably would claim that the progress was obvious (in hindsight everything is obvious).
Why would he have to accomplish anything else? He is not an inventor, he is a writer.
>Pragmatically speaking, Kurzweil is miles ahead of Hofstadter in terms of putting his theories and ideas to practice.
I don't think so. He merely invented some low hanging fruit in early computer science, like OCR and text recognition. Things on which other people worked and had results too.
And things that, even now, 3 and 4 decades after his inventions, are miles BEHIND of his expectations of them, and somewhat of a disappointment still.
>It's very easy to criticize visionaries, until they achieve something.
And it's equally easy to be a "visionary", if you don't have to also achieve those visions. Visionaries are a dime a dozen, especially in California.
It only looks like low hanging fruit after it is done.
>>And it's equally easy to be a "visionary", if you don't have to also achieve those visions. Visionaries are a dime a dozen, especially in California.
What exactly is your overall point? Most visionaries will fail because that is just how things are. Leonardo da vinci was a great visionary that could not fulfill any of his visions. Visions that were fulfilled hundreds of years latter. Eventually somebody will fulfilled those visions. Again, I don't understand what you are bitching about.
All this irrational hate against the guy for daring to dream what would be one of the greatest achievements of human history is unnerving. Eventually we WILL have strong AI, we are living proof that it is possible just as birds were living proof that things could fly.
Kurzweil has already done enough to prove he's not simply deluded. It seems like every week there's an article that comes across HN about how all great innovators have a capacity for self-delusion. Kurzweil has consistently shown himself to be a great innovator, and yet people dismiss him as a crank because he has the gall to continue to reach for "impossible" goals.
That's a Hollywood cliched idea of creativity. The "mad scientist" idea.
People have created great things without being "grandiose, obsessive and deluded". Genius and/or hard work will do.
Einstein wasn't either grandiose, obsessive or deluded. Maxwell too. Feynman was mostly playful and humble. Turing. The list goes on.
Great scientists and inventors are not necessarily of the Emmett "Doc" Brown type.
And you're listing people who have made discoveries. Kurzweil isn't trying to discover an equation or a law of nature, he's trying to build strong AI. That's a different category of endeavor. It's the work of Henry Ford or Steve Jobs rather than Einstein or Feynman.
They were rather pragmatic and market based -- not foaming at the mouth for some pie in the sky tech of 2100.
It's easy to forget that sometimes a discovery will languish for a long time (or even be forgotten and rediscovered) before someone invests time and effort in determining how it can usefully be exploited.
Who deserved credit "discovering" the assembly line? Adam Smith, who wrote about division of labor in 1776, or Eli Whitney, who implemented it manufacture muskets, or Henry Ford for using it to such great effect that he changed how industry operated? Or maybe the first Chinese Emperor, who's creation of a terracotta army is said to have used techniques reminiscent of assembly lines? Or perhaps the Venetian Arsenal? How many times in history was it discovered again, but never put into practice?
Another example, I really don't care who discovered it was possible to go to the moon, I do care about the people that acted on that discovery and actually did it.
All that said, I think Hofstadter's "mixture of good ideas and rubbish" description isn't a good characterization of the thinking behind The Singularity. Rather, Singularity thinking is extrapolation of growth trends which might easily be false but since it is made on a broad level, it is not easy to formulate why it is false. And Kurzweil, in particular is popularizer of the singularity with a particular version of it.
One could argue there are limitations to the expansions that Singularians have been extrapolating from but characterizing these limitations is itself quite a tricky problem.
I think the best refutation to the Singularity idea is the argument of Paul Allen, that software and the understanding of intelligence simply haven't been amenable to increased computing power.
http://www.technologyreview.com/view/425733/paul-allen-the-s...
It's a good argument but I feel the tone isn't quite fair, the sense that you could more fairly say the singularity looks plausible until you really focus on the software barrier.
For example, "Carolina beat Duke" follows from "Carolina defeated the Duke Blue Devils once again". I have a demo here: http://ec2-23-22-22-135.compute-1.amazonaws.com:8001/demo
Here's an example app that uses entailment recognition to answer natural language polar (yes or no) questions: gavinmh.github.io/HelloTablet.
I'd be happy to talk to anyone who is interested in getting involved.
I use a simpler alignment representation, and use semantic role types in the predictions.
By rigorous hand-waving, as he mostly does things the last 1-2 decades?
His basic argument of the inevitable advance of technology to true AI, or ability to upload consciousness or integrate computers with brains, rests on priors that are not clearly relevant.
With his latest title, "How to Create a Mind: The Secret of Human Thought Revealed", how can anyone take him seriously? Scientists (practicing scientists) have been writing books like that for decades, and the lesson should be that unless you've demonstrated that you can build an AI (real AI, not soft AI like machine learning), you have no business using a title like that. If real AI is simply a combination of the right machine learning tricks, then the trick is in the combination, because nobody has figured it out (publicly) yet.
How does this one-liner contribute to the conversation at all? Your negativity is unenlightening and obnoxious.
No, he did it to get some PR of Google as a mad-genius storage. He also did it with tons of other semi-retired Comp Sci legends. He might also like the guy, or believe the Singularity mambo-jambo himself.
There's a line going around about Google being where "old computer scientists go to retire".
>How does this one-liner contribute to the conversation at all?
By giving a summary of the whole thing? A very succinct one at that? If it's also accurate (which you can argue about), then it's a perfect contribution to the conversation.
The guy isn't an idiot, and has managed technical teams his entire life.
I know he takes like 100 pills every day
Him getting 100 pills every day: I don't think is very relevant. Mostly delusional. If that worked, pharmaceutical companies would have made it into one mega-pill, and sold it for a fortune to rich people years ago.
As for your argument against 100 pills, that only holds if it's well-known or well-established that it works and works safely.
Or semi-safely. The rich are also "early adopters". Also see tragically failed plastic surgeries and BS like cryogenics.
But still, that it's not "well-established that it works and works safely", is exactly my point about his use of the pills. Mostly wishful-thinking on his part.
If it's a truly competitive advantage, wouldn't offering it for startups/other-businesses in return for equity would offer bigger profits?
Do you then also know about the cost per query as compared to a Google search? Or alternatively, how much concurrent traffic can they handle with that server?
I think there was a query on quora about this.
and Google Search is basically Watson.
Basically, by reading and understanding large amounts of text, it can help you validate an hypothesis or even lead you to new hypothesis.
I am not sure I want that. At least not as much as Google does. Kurzweil is going to help me click on more ads. Shreds any sense of him being a visionary.
Clippy told people how to do their work more efficiently, and because people don't like to remember new ways of doing work if their old habits sufficient, people hate him. Clippy was like computerized supervisor, slave driver, who looked over their shoulders and told them "You selected one menu item 5 times, better use keyboard shortcut to mine more uranium for your Master."
Google wants to tell people what to do. He is like missionary, every layman would respect him for making life easer. People would love Google for this.
I'm totally OK with searching "#{my_city} taxi" instead of "taxi" if it means when I search "abortion" I get a sampling of search results that has the greatest co-citation... instead of something that myopically focuses on Google's perception of my preferences.
At the least, I'd like to be able to switch off the personalization. But anyway... Beating a dead horse here in this community, I'm sure.
https://www.google.com/search?q=abortion
https://www.google.com/search?q=taxi
Or if you were willing to type more than 2 words into the search box before blaming Google for failing to read your mind, while you simultaneously tell Google to stop trying to read your mind.
He seems to be saying that he'd be okay with typing in his city name to get local results in exchange for not having any personalization. In other words, it's beyond local vs non-local.
http://www.businessweek.com/magazine/content/11_17/b42250609...
... land, get off that plane, take a cab, and have a polite but blunt conversation with the founders about how to fix their company. (What did you think I meant?)
Artificial intelligence is nice, brahs, but natural stupidity in the form of closed allocation and Enron-style performance reviews are putting that company at 10% speed. Clear out the latter and you'll have plenty more muscle for the former.
I'm sure Ray Kurzweil will do amazing things, but he'd do even more amazing things if the company still had the machinery (e.g. open allocation, a culture of human decency) to bring talent to him properly. The whole reason he is there is to work with great people that the company is supposed to be better equipped to find than he is... but how will that work, given that the company sold off its ability to reward and recognize talent just to appease McKinsey?
I tried doing it a bit with my last project. My results were basically terrible. It turns out that getting useful results out of heterogenous, vague, fuzzily-specified real world data is really hard.
I'm tight with a few folks in Search Quality whose job is that sort of data-scientist machine-learning work, and they're really good at it. Y'know what 90% of their daily time is spent on? Compiling golden sets. Labeling training data. Running MapReduces to collect basic statistics about their data set. Running MapReduces to identify representative members of their data set, and outliers that should be excluded. Shoving data into R to visualize it. Futzing with numeric coefficients. Building webpages and tools so they can visualize the data and results, futz with the numbers online, and get feedback in real time. Collecting test sets and running your algorithm against them, and then trying to figure out why your losses are losing.
Machine-learning from a practitioner's POV is not at all like the textbooks and theoretical papers suggest. I'd estimate that less than 10% of one's time goes into the "fun" part of machine learning - brainstorming new signals and writing the code to extract them and feed them into your classifier - and 90% is on the kind of grunt work that hard science grad students do all the time. You get paid well for it, but that's because a lot of the work is really boring and time-consuming. I suspect that I get a far more frequent rush of accomplishment as a mostly-UI guy than the data scientists in my department get.
It's a tool. It works well in some cases, but it can take a lot of effort to get it to work well.
Part of what makes data science fun is the machine learning itself. And an equally interesting part of it is is that it involves so many other parts of computer science that a typical corporate job would call "too hard" and isolate you from.
At least based on what I've seen, data scientists are respected and this means they get dibs on the most interesting projects. However, the interesting projects themselves involve a lot of detailed work (that's the nature of technology) and if you're that interested, you're going to want to do it yourself, at least until you really understand the problem (at which point, you'll automate the dull stuff).
I wonder how easy would it be to actually do so?