Google to divide its index, giving mobile users better and fresher content | Hacker News Reader

Google to divide its index, giving mobile users better and fresher content

searchengineland.com

76 pointskjhughes43 comments

This was publicly announced more than a year ago, but still isn't live: http://www.thesempost.com/google-working-completely-separate...

This title seems misleading, especially the "better" part. I think what's going on is a lot of the web has bifurcated into mobile versions and desktop versions of the same site. Mobile versions are often stripped down for speed reasons, but mobile searchers can still land on a page based on a search query that only exists on the desktop site. Google is fixing this in a pretty sensible way (although I don't get why they are explicitly making the mobile crawler more active or "fresher").

This won't affect sites that serve the same content on mobile and desktop. I suppose now the concern may swap over to sites that offer more content on mobile, but I don't know why anyone would do that.

Reality is much more complex than that. Google's index has been tiered for many years, see e.g.

https://search.googleblog.com/2012/05/search-quality-highlig...

• New index tier. [launch codename "cantina", project codename "Indexing"] We keep our index in “tiers” where different documents are indexed at different rates depending on how relevant they are likely to be to users. This month we introduced an additional indexing tier to support continued comprehensiveness in search results.

This just means how often they scrape a site. Reddit gets scanned more often for changes than someone's personal website that only gets a few hits.

You can test this. Posts on Reddit show up in search in minutes.

It's not just a matter of freshness. They are also served by different backends with different trade-offs.

I may not understand the impact of this, but this sounds like "there is a limit to Google-scale solutions and we have hit it"? I know this is a gross over-simplification, but I'd appreciate if someone might offer more depth to this? FWIW: I have liked the duckduckgo approach of separating the "space" of the search, but I always figured this was a stop-gap until Google could grok the context from your situation (e.g. 4 searches in a row related to motorcycles and the 5th for AMA shouldn't give me anything on the American Medical Society, but instead prefer American Motorcycle Association)

Now I'm wondering if there may be some limit to Google's ability to hit this in a timely manner with the amount of content available. Insights very much appreciated.

As someone who only uses my phone to keep up on things between desktops, it's pretty awful for me to hear that Google is going to update the desktop index slower than the mobile one. If I'm searching for something, I'm on a desktop. I don't care about which websites are best optimized for mobile or which websites use Google's latest attempt to move the entire web to their infrastructure.

More and more, Google is demonstrating they're no longer willing or capable of meeting my needs as a search provider or a service provider, and I'm glad I don't use it anymore.

what is it you use now for search?

I'm not the parent, but I've been using DuckDuckGo heavily, primarily because of its bangs[0].

[0] https://duckduckgo.com/bang

I use Bing and DuckDuckGo somewhat interchangeably. Bing is particularly useful at work since I'm usually looking up Microsoft support issues (and they pay you to use their search in Amazon gift cards, which is nice), and DDG is obviously much better for privacy.

Not sure why this is being downvoted. "Microsoft Rewards" is apparently a real program (discussed Here https://www.reddit.com/r/MicrosoftRewards) which even I as a MSFTie knew almost nothing about. (Not speaking as advocation or any official/unofficial capacity yadda yadda)

I also find myself "OK with" bing for when I'm doing work searches; but it does seem to be more miss than hit for my more general tech questions. Like with browsers, multiple search engines seems like the unfortunate current state of the world as each has its own pathology.

I still remember when Google was pretty much a "grep for the Web", and wished it would stay that way. But now it seems to be a pseudo-search that second-guesses your queries, deindexes or only partially indexes old sites (which are still very much alive), censors, and does all sorts of other things which I find unpleasant and frustrating. Finding popular and new content is the easy part; unfortunately Google has only focused on that and severely neglected what I think is the main use of a search engine: finding the obscure and rare content that exists somewhere, but is hard to get at.

Unfortunately, DDG, Bing, and the others are still far behind the last time I used them. But maybe not for long...

While they can sometimes frustrate me when I'm running a very specific search, there are some features that I often like, such as the one that includes close synonyms of search terms, and includes both plural and singular forms. Makes it more of a "perl for the Web", which has its place.

https://xkcd.com/208/

I've never found a drastic difference between Google, Bing, and DDG except in one category: Searching Google websites. Bing's YouTube results are frequently terrible, for example, where highly popular trending videos are not in the top results when searched for on Bing's video search. Google indexes Google+ posts well, nobody else does. I'm not sure if that's a fault of the other search providers, or Google unsurprisingly making it difficult for competitors to index websites that they obviously can index quite easily using internal APIs.

I felt really weird the day I realized I don't use Google to "google" anymore.

I use DuckDuckGo, when it fails I'll fall back to Google, but even then it's getting to be pretty useless compared to how it was.

For example, I recently wanted to find a reference for an obscure quote that included the word "squaring" but all the search results from Google were for "square". So frustrating.

You can put "squaring" in quotations and then google will search exactly for "squaring".

I've had mixed results using their modifiers. Remember when you used to be able to do something like -something and it would filter those out? Hasn't worked for me in months.

can you remember the query you tried?

No, but "squaring bucky fuller quote" just now showed the problem.

and what is the correct answer?

Never did find it. In one of his books he points out that associating the second power with the area of a square (as opposed to some other base shape, like a triangle or circle) is a choice we make.

Consider triangles (hallucinate the bases):

    Side  Area

     1 /\ 1


       /\
    2 /\/\ 4


       /\
    3 /\/\ 9
     /\/\/\

it wasn't hard to find once i read your explanation

https://books.google.com/books?id=03fSBwAAQBAJ&pg=PA20&dq=bu...

First of all: THANK YOU THANK YOU SO MUCH!!!! It's been years, over a decade. This is awesome. Thank you.

What did you do to find it? (Is Google "punishing" me? My Google-fu used to be strong but I could not find this.)

Sure :) You mentioned it was in a book, so the first thing I did was go to books.google.com; after that, I just typed in some keywords that seemed right - you can see them at the top, I searched for [buckminster fuller triangle squaring power]

Thank you. :)

DDG failed every single technical search for me. I was putting !g to get google every time. It was at that point I realized Google really still doesn't have competitors in this field. Any search engine can bring up common stuff, say articles about the current election or celebrities, but all the others I've tried with technical content (not just software based) bring up pretty bad results.

Yeah that matches my experience, but Google's gotten much worse in the last 3~5 years in my experience, while DDG has slightly improved.

Very simple decision for them. They care about android far more than desktop stuff.

The mobile market is bigger and still growing. Desktop is shrinking. It's a simple financial decision.

I don't get it. It's essentially free to give desktop users just as good of results as mobile users, isn't it? How does intentionally crippling desktop help mobile?

Simple. Google makes money from search ad.

Desktop users block ads. Mobile users don't.

> Desktop users block ads. Mobile users don't.

I do. Anyone who root's their phone can easily block ads.

You're confirming that >90% of mobile users cannot block ads at all.

Firefox + ublock. I feel blocking ads on mobile is crucial considering how small data caps are.

I did not realize that FF had mobile addons. Time to switch.

It's pretty slow tho.

Got it. It's not direct cost then but more like opportunity cost. Thanks.

Everyone I know with iOS blocks ads now. Ads are far more intrusive on mobile than they are desktop. Those full page takeovers that come up as you scroll and the ads that bind to any touch event to auto play a video are the reason that push most over.

oooh, this is good.

From firsthand experience I can tell you the intent is worse than most people realize. The discussion is usually "well of course <company> wants to prioritize mobile, that's where the money is." That's small thinking. Those companies want to kill the non-revenue generating options in favor of the revenue-generating ones. In other words, Google isn't trying to de-prioritize desktop, they're trying to force you away from it (because they make so little money there) onto mobile. Once you realize that their strategy makes a bit more sense.

That would make sense if Desktop didn't make them more money per user/click.

Do you mean that, in their ideal world, anyone who was sitting at their desktop computer trying to do actual work, and needed to search for something online, would have to pull out their phone and search Google with it instead of with their desktop browser? And then do some silly dance of manually typing URLs from their phone's screen into their desktop computer?

Because that's patently absurd. That would cripple their own use of their own search engine.

However, that's not to say that there aren't necessarily some..."people"...at Google who would try to pursue such a patently absurd outcome. Maybe we need to give more serious consideration to these "distributed search engine" projects...

I don't get it either, but the reason I don't get it is it's free to give a tiny link at the bottom of the page letting people go to the other version. On mobile you don't see how many pages match your search result, so for googling an uncommon spelling you're guessing at, you don't know if it's 37 results on the entire Internet or 737,328 results. It would be sooo easy to add a tiny little link.

So why don't they do that? It should be super simple to do -- just do it! Likewise, here, it should be easy to get to the other link. I don't get their thinking here. (Plus, mobile versions of browsers should be basically as functional as normal browsers these days, so it's not like that's a good reason. Mobiles run desktop versions of sites just fine -- wasn't there just an article saying that the iPhone 7 is faster at single-core CPU tasks than any Macbook Air ever made?)

How Providing New & Fresher Content To Mobile & Small Device Users Will Play A Beneficial Role?

Reply on news.ycombinator.com