Not only is it possible to beat Google, it could happen sooner than we think
quora.com
quora.com
1. The post assumes that the search result quality is not good. This simply is not the case. Users say the search results are good. Could they be better? Sure. Are they terrible? No.
2. The post assumes users want transparency on the search algorithm. This clearly is not the case. The whole reason that google became what it is today is because of the lack of clutter and good search results.
If you remember Altavista then you remember users constantly having to fidget with all these knobs in the advanced section to get relevant search results. Lycos, excite, etc weren't much better. I remember using Google for the first time and it just worked.
3. The post assumes that users want deep customization. This again is not the case. Users want less clutter, less knobs, and a faster search experience. Google is delivering that.
4. The post assumes that any startup with a better algorithm can beat google. However this is not the case. Google has deep search integration with mobile providers such as it's own Android OS and also iOS. it also has it's own browser (Chrome) and has paid deals with Mozilla. Obviously, both browsers default to Google when you type something in the search field.
Not to mention, the leading IOT devices all connect to Google to find results when it needs to search the web. And for safe measure, Google has come out with its own voice assistant device, Google home, to safe guard against any shenanigans from Apple's Siri and Amazon's Alexa that might happen in the future. It is heavily subsidizing the price to make sure it has a strong horse in the voice game.
Integration aside, it also has scale. It crawls most of the internet several times a day. That's no small feat and it is quite hard for a startup to just replicate that with funding. I mean, even that won't cut it.
Google is now even using machine learning to understand relevant pieces of information on a webpage so when you query it by typing on your keyboard, by voice, or future mind control it delivers the answer you were looking for.
The only valid point the post makes is that Google has squashed competition by buying any startup that remotely could threaten it's position...for safe measure. I'll give the author that.
Definitely good points, but I disagree with them.
1) The issue is not with search result quality for a given query. In fact, I agree that such quality would be difficult to match. But this is a very specific, very polished feature. It is by no means the only way to deliver a quality user experience.
Rather, the issue (and perhaps this is just one issue with many more I am unaware of) is that the search space of possible queries is limited to a small number of dimensions.
2) Definitely under no illusion that there is a demand for transparency on the algorithm. However, if that information is a successfully-guarded secret, nobody else is going to create an incremental improvement over that specific type of feature.
3) You have two incorrect assumptions here. Not all users want the same thing. But more importantly, there is absolutely no reason why more customization means that the UI must become more complex. A company like Google is smart enough to see that. If you have more customization, it can be offered through an API. Even if not offered through an API, you could see open-source browser plugins to create a custom page, with no clutter and no knobs. And likely a fast search experience with some effort.
4) Absolutely false! The point of my post was to illustrate that the failed strategy of "imitate the UI with a better/different algorithm under the hood" will never work.
Your point regarding scale is a good one, and it is exactly part of the reason why I believe that peer to peer blockchain technologies will be the likely place for this dethroning to occur. Users can collectively contribute their own little piece of the network, sometimes even being paid by doing so (refer to Steemit for a proof-of-concept). There are a lot of interesting ideas in some of the Ethereum distributed app whitepapers. Most of those ideas will never be successfully built, but there are enough of them where we will see some successes. Then the industry will learn from the mistakes that were made, and we will see a new paradigm. I think this will happen rapidly over the course of a few years. I already have specific conceptual ideas for how a search engine might operate (URLs can be associated with a reputation rating, various tags, and so on, and not bound to a centralized server).
1. How else would you like the results to be delivered? It is delivered in browser's search bars. It is delivered by visiting Google.com. It is delivered by voice. It is delivered by mobile devices. It is delivered inside my Nintendo Wii, XBOX, PS3/4, and my Nintendo Switch. It's delivered on my crappy Vizio TV via smart apps. It's even delivered in my car. Google search results are everywhere.
2. I think that is kind of the point. However, you assuming Google is being evil by keeping it a secret. That's not fully the case. A lot of SEO companies would love to see how things are ranked so they can beat it on behalf of their clients.
3. They already have an API and if you really want to customize the query, you can add all kind of query modifiers in the UI.
4. As a developer myself in the crypto space, blockchains make a poor curated database for the very reasons why Google keeps things a secret. Anyone with money can pump the reputation of an entry on the blockchain to get better rankings thus lowering the quality of results.
Second, most blockchains can't handle the query load.
Three, dapps cost money to run. I can see a market for users submitting entries. If there are users using it, it might catch on in the submission department. However, this brings me to my next point.
Four, the chain would not have all the websites on the net. Kind of expensive to get it all on there and not scalable.
Lastly, someone has to pay to run the query side too. Would users pay to run a search query and get gamed incomplete results? No.
Regarding your comment about blockchains and query load, you also seem to be unaware as a developer that blockchains are a source of truth, not a front-end. A user request does not cause a query to the blockchain. A network of server nodes can exist, with cached results. The servers themselves can have their own reputation system, or the entire front-end experience can be managed by a single centralized entity without compromising the use-case of decentralization. Specifically, if the owners of the front-end were to act in bad faith, it would be much, much less expensive for someone else to spin up another identical front-end (most likely open-source as there would be a lot of early criticism if it wasn't).
Regarding expenses and costs, solving that problem is part of the hard work. If I were building the system, I would probably not start out with the entire Internet being the scope. Perhaps as a proof of concept I'd start with Wikipedia only. I know for sure that paying for each search query is a non-starter, as that has a psychological effect of being punished for using the service, when you want it to feel rewarding instead.
Why? Because the instant I search once, twice, three times and find nothing. I am abandoning the system.
Blockchain aside, it doesn't matter. Whatever system you build has to be 10x better than the current system. Nothing you have said, blockchain usage included, comes close to getting there.
https://www.quora.com/Is-it-possible-to-beat-Google/answer/M...
And I don't go to page 2 anymore because Page 2 will always have garbage on it. I remember in the old days, in the 90's, I could search and get 5 or 10 pages of unique pages. They might not have all been entirely relevant, but they might have been interesting reads nonetheless. With today's Google search, I get about 3 or 4 typical relevant results, often from the same websites vomited out from the top 50 from Alexa 500. And then a lot of spam, computer-generated web pages after that.
There is no hope for anyone who wants to independently publish "a web page", anymore. I realize that this isn't something easily solved due to the fact that the Internet is vastly more populated now. But I don't think the system that we have in place, with Google, is anywhere near the level of sophistication that we can achieve as we approach the 2020s. We have this narrative that Google is such a vastly powerful collective, and that this is the best that they can come up with, and therefore that's what we have to work with, as Internet users. My post was not to show that I have all of the answers, but to illustrate the fallacious reasoning which deters people from even attempting to solve the problem.
Folks, as long as we continue that mindset, that's how things are going to continue to be.
/rant
But in general I agree with your rant; please experiment! I just do not believe it will take hold easily. And they article seems to suggest so.
(3) No, I'm pretty sure users want more knobs or at least filter options. Hell, even a simple domain blacklist feature would be amazing so that I could get the woowoo psuedo science cruft -- which has been SEO'd to the front page -- out of my results, so I didn't have to put any mental effort into visually filtering out multiples from the same site. Hell, they could even collate results from the same domain into a single grid folder to optimize the time I spend searching through the search results.
Don't conflate Google's lack of innovation for an optimized experience simply because it's Google. I love Google, but there is a massive amount of improvement that can go into their search user interface. Same for every other search engine, but Google is the leader in this space, so they get the brunt of the criticism, and should they change, then others can follow suit.
You can put w3schools in /etc/hosts to an IP like 127.0.0.1. You still have to visually parse the search results, but if you accidentally click, you’re not giving a pageview by mistake.
I use this Google plugin just for this:
https://chrome.google.com/webstore/detail/personal-blocklist...
Nah, 90% either don't or wouldn't use them if they were there.
Sure, if you ask people directly if they want more options, they'll say yes, who doesn't want more choices? But that's a different thing from actually using them when they're available. The vast majority of users for a utility service like a search engine just go with the defaults.
Well, a year after we implemented 2FA, I queried the database for users with 2FA enabled, 3 people had it enabled.
Aside, only somewhat related to the previous story, the idea that users want a bunch of knobs and levers is just an example of the HN delusion where HNers regularly think they represent the population, like the person using the Lynx browser.
The first few times I saw it, I thought they must have been joking or trolling.
I don't even have time and budget to make my site work well on older versions of IE. It would be insanity to optimize for the 12 people that use lynx.
Even Facebook used to work with Elinks (don't know if that's still true).
Don't let your supposed experience as a professional techie give you the false impression that you know everything there is to know about tech.
First day in my Intro CS class at college, a dude asks about whether the class website supports Lynx.
-site:badsite.comStill possible though: set your browser to use as a search engine Google with those terms already supplied.
Which point out that at this point not only Google has a good search algorithm, but users are trained to help it do better. A search startup has to overcome that as well.
I stopped using it because it ended up being extra time spent running my google queries.
Whatever it was at the beginning of the switch I used to use Google on a daily basis when DDG results were not good. Then in few months it pretty much stopped.
This is so annoying; I don't understand what the country toggle is actually for, since it doesn't seem to correct this?
One can interpret this (and android) as google desparately spending money whereever money can give some competitive advantage, no matter how small, because google is acutely aware of how easily users can switch search engines. It's not a super clever strategy, but just: what else can you do with all this cash?
But if there are technological shifts that invalidate the investment Google has made, then there is a chance.
I think you're spot on about why Google is investing in the Assistant, but I think you underestimate the threat from Alexa. If Alexa gets a foot hold and manages to use the fact that you're having a dialog to deliver commercial queries better, Google is in a lot of trouble.
Vertical-specific commerce sites are also a threat, if you no longer search Google for insurance, but go directly to an insurance aggregator, Google's search tech is irrelevant to capturing that advertising revenue. I think Craigslist is undergoing this problem since they've basically refused to invest in better UX, vertical specific sites are eating their lunch.
But using a voice system to buy new stuff just isn't a good experience. It's like the old days of catalog shopping with a phone except there's no catalog.
People need to pull up and see, look at reviews, watch YouTube review videos. Audio only just doesn't do that and is a step backwards.
I don't even use Alexa for reorders because I want to make sure I'm still getting a good deal everytime I order. And for those that don't care about the deal, they can just setup subscription purchasing.
He also seems to not understand the value that search advertising provides. Search advertising is a win-win for businesses and consumers. If you see it as something parasitic that needs to be fixed, I can't take you seriously about "how Google will be beat".
Then I read the last sentence, quoted below. Sigh. Nothing serious here folks, carry on.
> I predict this will become more of a real possibility under decentralized smart contract systems such as the Ethereum blockchain, where value exchange itself can happen on a peer to peer basis (via a synergistic ecosystem of content curators, content providers, and content consumers).
People would fix Google if it had an API but it would mean censorship; I would add a ‘credible sources only’ button which is not good for the web but it is what I want. I do not care for traveldiscounters’ SEO or SO or Wikipedia scrapers; I care about the originals. That would greatly improve the experience, but in general it is bad for the internet because someone decides who is credible. Now there I see a place for Blockchain.
Ethereum, in my mind, is merely a way to help automate, standardize, and enforce similar systems. If it can be achieved without any sort of blockchain, like in the case of Wikipedia, then great. But perhaps the barrier to doing so is just out of reach without the help of the technology. Yes, having a central authority decide who is credible is exactly bad for the Internet, and Ethereum is a possible means to facilitate alternatives. What other alternatives would there be?
And the market does this in a relatively simple manner: by letting merchants bid on search keywords, and showing the site of the merchant with the highest bid, you force merchants to either lose money or be relevant.
In other words, the most relevant merchants to a search phrase will have the most money to bid up visibility for that phrase because they make the most money, on average, from users looking for this particular thing.
Especially given the amount of vitriol I see on Hacker News regarding decentralization technologies (not surprising given the heavy ties to Silicon Valley's centralization culture), your condescending comment wasn't helpful to the discussion. But I appreciate you giving it a full read nonetheless.
I think you're missing some important details in your definition of centralization. It's not who uses it, but who controls it.
That is the power of decentralization, regardless of the naysayers and corporate shills who continue to dismiss the whole concept, by citing examples of alpha-quality solidity contracts. Ethereum, despite being itself barely beta-quality, and highly primitive technology of its kind, is still a disruptive technology that threatens the status quo. Creating an app that has the strengths of Wikipedia, but functions like Google, Facebook, or Twitter will be an order of magnitude easier with the help of this technology, and the right team. Maybe not enough to disrupt things outright, but certainly enough to be a large threat, and the next generation of that type of technology will have a near-guarantee to bring that disruption. Obviously there is a lot of vested interest with deep pockets, toward discrediting it. The resulting narrative, combined with the consumer culture/sentiment toward the products by the same companies, is what makes up the centralization culture. I think we need a catchier phrase, maybe even a buzzword to describe it :P
Don't get me wrong, I think many crypto enthusiasts out there are insufferable man-children with criminal records and an axe to grind with "the man". So blockchain's culture might be the problem in the sense that some of the users are giving the technology a bad name.
Open-licensed content cannot be said to be controlled by one party, as anyone is legally entitled to create a fork. Thus, Wikipedia is not controlled by the people who run the servers; the infrastructure may be centralized, but the project (the compilation of knowledge, and the articles produced) is not. This aspect of control is what I said you were missing in your assessment, and it seems I was right.
Ethereum is a platform for creating decentralized applications, and it too has technology and policy that is managed by a central foundation. That doesn't mean that the ecosystem is centralized.
Zooming out, we can see that the Linux Kernel development itself has a centralized development model, even with a dictator in charge. But the Kernel is a platform for which anyone can create and ship a distribution on top of. As a result, I am able to enjoy the use of three or four distros (or more, if I wanted) that are most tailored to my use cases, instead of being forced to choose between Pepsi and Coke (Windows and OS X). And many people can get into operating system development as a hobby, with a much more achievable criteria to do so.
We can zoom out even further: The FOSS movement itself might not have existed without the relentless efforts of a few individuals over a span of many years. But from day one, the ecosystem it spawned was decentralized by design.
Going back even further, we can show that market Capitalism allowed for a rapid and diverse pace of technology and business development. Less barriers to entry. Less red tape. Obviously we have had governments, both big and small, to facilitate the economies of a nation.
What do all of these things have in common other than incorporating applied forms of decentralization? It is not just a buzzword. It is no coincidence that these frameworks have (or will) foster significant societal changes, mostly in good ways. That's not to say that the success rate is high for any given effort, but it is not a concept to be written off, especially not because of a lack of understanding of it.
The problem is that most people, even here, cannot visualize this concept in ways that they have not seen working examples of. Not a problem, not everybody is exactly a visionary in every way. But when people are actively trash talking ideas that they don't understand, without offering constructive opinions toward an understanding of it, then as technologists we ought to be taking a stand against that behavior.
Instead, what I see in many cases is that many engineers, for whatever reason, will write off certain technology ideas, believing that they understand it, when they do not. I was one of those people, for years, so I understand the mindset as good as anybody. Or perhaps they saw too many uninformed comments coming from non-technical crypto enthusiasts, and therefore have written off the idea. Folks, if this describes you, I encourage taking up the study of critical thinking techniques. The world deserves better than to have such a large number of engineers not giving the discussion a fair shake.
Its absolutely parasitic and not a win for consumers. If it were a win for consumers then Google would not be so fantastically profitable.
Search results on Google are great provided you don't use a search query that indicates an intent to buy something. If you do that then the results turn terrible. Or profitable. Depends upon whom you ask.
Would users be willing to learn a more powerful interface, if they consider current search engine results “good enough”?
Could Wikipedia or Archive.org address a subset of the search market, since they offer user-focused services? Archive.org already crawls a subset of the web.
It doesn't need to have a huge amount of users, it doesn't need to get the majority of the market share, and it doesn't need to earn more revenue, or even any revenue at all, in order to deliver a better user experience. Such things are not mutually exclusive.
I believe this is most likely to happen with the help of peer to peer technologies, combined with a healthy synergistic ecosystem of content curators, content providers, and content consumers. Steemit is a crude proof of concept for such a system. In Steemit, comments and posts are scored with a reputation algorithm, and assigned a tag. There's absolutely no reason why web links can't be associated with an arbitrary number of contextually-weighted tags, and a contextual reputation algorithm.
Everything that Google does could be done in a peer to peer manner, with each user bearing a small piece of the responsibility. Wikipedia is a great example of this, with many users contributing to the system. No single company could ever produce the amount of content that the Wikipedia userbase has created. Even though Wikipedia uses a centralized foundation and centralized server, the editing culture itself is decentralized, with a set of principles being the main driver of decision-making.
I bring up blockchain technologies, much to the disgust of many users on HN, but only because I see those technologies as lowering the barrier to creating systems such as Wikipedia, through automating, standardization, and enforcement of some of the mechanisms. It also creates a guarantee that a central authority doesn't exist and therefore cannot be manipulated in subtle ways by those in positions of greater power - That is a big appeal factor for a significant number of users, even if it is not the majority.
Yes. Google cannot be beaten.
I don't think that any domain has more money to offer than advertising/marketing and certainly not easier. Then the real time auction and pricing systems that run Google are truly incredible.
Business wise, google is milking the perfect cow perfectly. There isn't a better cow or a better milking strategy.
http://www.independent.co.uk/life-style/gadgets-and-tech/new...
I think it folded after a year.
Cuil, SearchMe, and Blekko didn't fall into the UI trap that he outlines. None of us succeeded that well, but that's a different issue.
I'm also curious if you have kept up with other web search technology efforts since then or whether you moved away to other things. There's definitely some things that don't add up.
I am sure bing/msft also does everything they can to 'understand' me, but at least they don't see my browsing history or emails.
Now back to beating Google - I do believe it's possible, but Google will soon catchup with any potential competition. They will catchup because it's their bread and butter and they have perhaps the worlds biggest concentration of technical personnel. The best way to beat any product/company is to be considerably better than it in at least few areas, given the scale of web indexing, that is a very very tough job only few companies can pull through.
Microsoft has/had a good chance to beat Google, but Microsoft is considered a bigger evil/corporate than Google, so they have a slim chance to emotionally overcome Google, not to mention technical capability. But they can perhaps open source bing in an effort to beat Google, provided they have the will and moolah to do so. When I mention open source, it's not the code, but the underlying search index, exposed as an API, free of use for anyone/any product. That will dramatically shift the emotion as well as the usage towards bing. I even mentioned this during an internal bing meetup event (I am part of a bing beta testers group), but of course no one listens.
A startup can also try to beat Google in their wildest dreams if they can associate themselves with the younger crowd like snapchat did and the younger crowd somehow finds it hip/cool to not use Google and use that service instead, and provided they get a decent web search data/index from established competition like bing or they manage to index themselves. These are few of the possibilities.
Also, the image of a "hip" Google competitor like Snapchat seems tremendously unappealing, personally.
It's not just about consumers, but a plethora of other startups who can then use the index to build something creative on top of it. For e.g., you currently have bing/google customise certain type of websites with custom view for quicker access (IMDB, Wiki etc). However there is a limit a big company can be creatively about this.
Imagine a search engine where any user can submit a custom widget under a market place and consumer users can then install that as a plugin. For e.g., stock brokers can install stock widget, programmers can install document/stack overflow search and so on as a quick widget to see the top result without opening the link.. since it is community maintained, it will be proactively maintained by the community itself. That alone can topple something like Google if there is enough momentum (E.g. see VSCode overtaking Sublime text.) That is just one possibility. And it can also improve the search results as users may voluntarily come up to improve the result accuracy.
Other possibilities include voice based search, gesture search, integrated search within any mobile or smart watch app etc. With current licensing model, other startups have severe limitations in obtaining search results from the wast internet. This could explode and perhaps take over everything.
Also,
> I predict this will become more of a real possibility under decentralized smart contract systems such as the Ethereum blockchain, where value exchange itself can happen on a peer to peer basis
...
Results from Altavista, Bing and DuckDuckGo are so much worse.
Note, it's not that Google has personal information on me. I'm not logged in, my cache is cleared, I use a VPN and google.com/ncr when it was available. Not sure how they do it. Perhaps their tracker sees how long people have visited a page and then they give those pages higher priority.
Perhaps other search engines can do a better job, but they had over a decade to improve their results and they are still far off.
I saw an article from 2014 saying ghostery works to defeat it, but haven't looked further and I don't use ghostery, so not sure if that's true still.
You can test it here: https://browserleaks.com/canvas
[1] https://arstechnica.com/information-technology/2017/02/now-s...
For example, if you search for some programming topic, it will generally pull results more towards the top, if they reference the current or a recent version of whatever programming language/tool you're searching for.
That helps, if you just want to type in "how do i do this? [programming language]" or similar.
In other search engines, you'll instead first search "how do i do this? [programming language]", then possibly not really get any good results and then instead search for "how do i do this? [programming language] [current version]".
This is how Google worked as well, some few years ago. Knowing what keywords to type in to get the right results was an actually valuable skill ("Google-fu"). Today's Google is instead designed to minimize the need for skill in phrasing your query. Great for the average user, but in my experience, if you do still possess Google-fu, then this interpretation actually hinders you.
With other search engines, you get the results that your query asked for. You know immediately when your query was ambiguous or just bad and can adjust accordingly. And with that, you can more easily narrow down the results, given that you know how to narrow down the results.
So, maybe give other search engines another try, while keeping that in mind. I personally very much prefer other search engines, because of that, even if I sometimes have to type one word more.
Doesn't surprise me coming from the sort of person who submits their own Quora ramblings to HN.
The post got a lot of views, so I would call that a success. Not sure what point you were trying to make there.
Cuil tried to compete with Google. Their idea was that their system was an order of magnitude cheaper to run than Google's. But they never developed a revenue model. Cuil went bust.
I tried a system which trimmed most of the ads out of Google result pages. That got very little use.
Google does have search customization. It learns from your queries what you ask about. At least if you're logged in, and possibly if you keep the same IP address.
Google is already moving past search to question-answering. They're not very good at that yet, but they're getting better.
When searching for answers which Google has no answer for the answer should be “this is ogooglebar”. So you don’t have to spend hours trying to refactoring your query for hours. This would be a great improvement and a great user experience.
This is Google’s akilles heel - never able to admit that they don’t know.
I mean, someday, we will be encasing entire stars in artificial shells and creating black holes of our own. What’s a single corporate entity that managed to get a lead in barely the first stages of a relatively new industry?*
How about incorporating search tech into web servers themselves? Move it to the level of DNS And other core protocols. Make individual servers automatically index all content that passes through them. They could then populate special caching servers with the indexes and results. When a user searches for something, the query gets sent to successive search-cache servers, kinda like a DNS lookup, until match is found or up to a browser-specified depth.
—
* People generally did not care much about search engines at the beginning; they usually just had a small list of sites in their memory or bookmarks that they visited regularly, and found others through magazines, links or word-of-mouth. Search engines were merely a hit-or-miss convenience, often used for porn or “wares” I’d wager, then that convenience emerged to become the core way of discovering and interacting with the internet.
Does anyone here remember the point when they “switched” to using Google? I personally can’t recall myself ever doing that. I think it just happened casually, through a browser that defaulted to Google. I remember thinking how ugly Google’s UI was, and preferred other engines for their “looks.” But at someone point, Googling just became faster than any other way to get to what I felt like looking up.
For now, I do wish browsers would start offering Wikipedia as an option for the default search. :)
Try Firefox: it's right there in about:preferences#search. :)
iMho this will be the natural progression. So if you are creating a better google search please stop you will be out of business before you start. This is no longer about standalone search but a larger ecosystem around the information needs.
I want to know: "In Rob Fergus' paper intrinsic motivation and self-play, how is it that the reward for agent A can be based on the time taken for agent B's task, when agent A and B are run sequentially, and agent B runs after agent A?"
Can Google answer this? No. Is NLP smart enough to answer this? Probably not. But the question itself is hard to write in one sentence. Complex queries are best structured as conversations, with clarifying back and forth, Socratic method style, like if you're asking a professor a question about hard science.
YouTube and Gmail substitutes and then I am Google free.
I have started to trust Apple maps - not as good as Google though.
Hope this gets commodotized though.
For maps, check out openstreetmaps. Not quite as good as google, buy pretty solid. And you can run your own instance.
Heck, you could also run your own email without much hassle.