Hacker News Reader

I have to call bs on Matt's response for two reasons.

Digging a bit deeper I found that the bottom two links you mention Matt are both scraped from a site called freshome.com. This raises the question should sites/brands be worried about content that gets scraped and put on low quality sites? That sort of "manipulation" seems quite inorganic.

My second question is what about Nest posting on Freshome.com as well found here:

http://freshome.com/2011/10/26/greening-your-home-with-a-sex...

THEN we see the scrapes on the same sites found here:

http://doyoulovewhereyoulive.com/archives/greening-your-home...

http://arch.itect.us/2011/10/26/greening-your-home-with-a-se...

Notice how both NEST articles are found on doyoulovewhereyoulive.com AND arch.itect.us?

Case in point, Nest has links from the EXACT SAME SITES and didn't get penalized. This raises a huge red flag. If these links are the reason that Vivint was penalized, I'd love to hear how Nest somehow managed to escape a penalty.

Maybe NEST disavowed the links using the disavow tool? Whatever the reason, raising concern is appropriate, but calling bullshit without all the information is overreaching and inappropriate.

The problem is that they won't say. We're here, discussing of a potential violation - voluntary or not - of antitrust laws. All we have is a muddy answer by the lonely marketing gunslinger Google has put in chief of public replies to search issues.

How is this possible? Matt is the ultimate super-nice guy, and I think he's sincerely helpful and motivated, but he's nonetheless the keeper of a smoke screen.

I think this way of handling (better: brushing off) issues is unacceptable for a giant corporation whose search branch personifies the Web for a way too large amount of users.

Google is in a position of monopoly, at least on a cultural basis. Even if they don't exploit their power with evil intentions, they still retain it. They corporate mission may state what they want, but I don't get why that should be enough. It's a Public Company, and nobody should ever be satisfied with answers of the "take our word for it" kind.

And that "don't be evil" mantra? I've seen much too shit in the last few years to still believe it's not just an empty marketing motto.

Matt Cutts is "head of Google's Webspam team"; in other words he spews disinformation so that Google can have more influence over their own results. Very similar to the President's Press Secretary or the head of Propaganda. Web site owners are best off to almost completely ignore what he says and instead focus on what really is happening.

Do you have any idea who Matt Cutts is? He's owned the spam problem at Google since practically day 0. He's not some PHB.

Thanks, Matt. I always appreciate your input here. Your response sounds legit. In general, I still think that G is in an awkward position. Essentially we have to believe that Google, a company whose primary goal is to make money, can effectively police itself and operate on a level playing field. Yet there is a lot of secrecy in the things they do and just like any other company they deserve that right (I guess). I wonder if 'we' can do much else at this point except _trust_ what you are saying.

Google’s mission is to organize the world’s information and make it universally accessible and useful.

There's nothing in the mission statement about making money...

Are you joking, right?

No. If maximizing next quarter/year's profits is Google's primary goal, then the logical thing will be to slash R&D, cut back on benefits, eliminate CapEx, basically what Mark Hurd did to HP.

If you mean Google has to make money to survive in the long term, then that's absolutely true, just as it's true for any other company (Microsoft, Yahoo, DuckDuckGo etc.)

I don't see any evidence that Google is more profit centric than any other company.

You don't need to be profit centric to insist on a path you think will lead to a greater good. That's enough to scare me, frankly. And, I don't understand why a multimillion public company should be ever trusted just because, well, they say so.

Paving the way to hell have always looked great from the outside, you know, because of all the good intentions put towards it.

You can do something besides just trusting Google, and that is using and promoting alternate search providers. Google is approaching monopoly status in the search space because we're allowing it. With a bit love, DuckDuckGo and Bing could make dents.

I think the other problem is that search doesn't matter as much to anyone else. Yes, Microsoft made Bing, but Bing is essentially a side project for MS -- their crown jewels are Office and Windows, and everything in the company revolves around that. Just like everything MS does ultimately comes back to Office and/or Windows, everything Google does ultimately comes back to search. If a company with resources, like Amazon or Microsoft, made search a top-tier priority, they could take marketshare from the big G.

One potential move that would greatly assist Bing, for instance, would be for Microsoft to make Android phones and put Bing as the search provider instead of Google. They could team up with Amazon and use Amazon's app store. Instead of doing this, Microsoft released a totally separate platform. Why did they do that? Because Java is competition to C#. Who cares about C#? Microsoft cares, because C# (and Xbox, and...) is about keeping Windows the dominant platform for any and all software, because that's how you keep selling Windows licenses and keep Windows shipping on every PC sold. It's all about protecting their lock-in. If MS embraced Android, they may help Bing, but they'd be undermining the Windows lock-in strategy.

Every company comparable to Google has similar problems. DDG is noble but nowhere in the type of ballpark where they can realistically compete (for now).

I've got to say I've got a definite softspot for DuckDuckGo, but I always struggle to see bing as a contender. Now it might be for the reasons you mention (Microsoft not giving it enough TLC) but it does make it difficult to switch.

That being said I make sure that DuckDuckGo is always one keypress away, I still haven't managed to make it to a full switch but I try and use it more and more.

You do know that most of the results served by DuckDuckGo come (via APIs) from Bing?

Matt,

I have tried reporting a website that buys 100% of its links (I used to work for them and have tons of proof) and is being rewarded with 2.5-3mil hits a day from google and yet no one seems to care! Is there a way to get my point across or do I have to write a blog post that gets picked up by HN to get any action taken?

Well, the best way to report spam is with our spam report form at https://www.google.com/webmasters/tools/spamreport?pli=1 because then we can prioritize the report along with other complaints. But you can also post in our webmaster forum at https://productforums.google.com/forum/#!forum/webmasters or catch us during webmaster office hours--see https://sites.google.com/site/webmasterhelpforum/en/office-h... for the schedule of upcoming office hours. Or catch one of us at a search conference (I'll be at SMX Advanced in Seattle in June and at Google I/O later in June).

If none of those work, I often pass on spam reports that people tweet to me (or when they do a blog post and tweet me that link). You can also tweet to @googlewmc (for Google Webmaster Central). Or if you add an email address on your HN profile, I'll drop you a note.

Thanks Matt,

I have tried the first link several times with no luck. I've heard that it's a myth that it actually gets checked but seems that may be wrong :)

I will try all the other avenues mentioned and worst case will fly to Seattle to report this site in person. Site that is blatantly spamming is plaintube and is ranking for an insane amount of adult terms. I'll try to get my point across via the methods you described. It would be wrong of me to hijack this thread to get my point across anymore. Thanks for your time.

Email added in the profile. I would be more than willing to show how you guys are sending millions of hits (daily!) to a site scraping content and buying cheap backlinks.

I'm surprised that such tactics work in 2014, and am just as surprised that I got your attention. Hacker news is a powerful place!

Hey, I didn't see an email address at https://news.ycombinator.com/user?id=r0fl ? Maybe the page is cached and only gets rebuilt every so often, but you might want to double-check?

Sometimes people make a mistake and just put it into 'email' (which is non-public) and not into 'about' (which is public).

@natsu was right, I only had the email in "email" and not "about". Just fixed that, sorry about that.

What is the site?

http://www.plaintube.com/ apparently. very much NSFW.

just mention that this site is competing with one of Google products/acquisitions.

Thank you Matt. Agree, even a quick look from a not-so savvy SEO at their backlink anchor text you can see that they were intentionally building a ton of links.

I think the only thing that made it a tad fishy was the timing. It just seemed like the acquisition happened, then the penalty tight after.

In defense of the OP, the Google spam-punishing procedure is not purely objective or run as an unsupervised automation...it seems that Rap Genius would not have even been discovered had a blogger not made a relatively innocuous inquiry about RG's very public solicitations...which was then upvoted to the top of HN and presumably to Cutts' attention. And then after RG was reinstated, there were complaints that had it not been such a high-profile startup, the Google team would not have reinstated it so quickly (but in the defense of RG, they described and released the source code to the program they used to hunt down the bad links, which made it clear that RG handled the process in a way more efficiently than most other companies might have).

So Google's banhammer is wielded by humans with biases...that much is clear...but do we really know the order of cause-and-effect here? Perhaps this suspicious timing was an inadvertent screwup of Vivint's...that is, right after Google's acquisition of Nest, someone on Vivint's marketing team thought, "Oh shit, better step up our back-link game"...and then took it to a level that triggered a Google investigation. That chain of events could also explain the coincidental timing...and also could mean that both Cutts and Vivint's CEO are both telling the honest truth, as far as they know (not all CEOs are privy to the actual workings and details of their marketing teams)

In this case, we started dissecting this particular spammy guest blog posting network in November of 2013, and Google didn't acquire of Nest until January of 2014. So Vivint was link spamming (and was caught by the webspam team for spamming) before Google even acquired Nest.

Unless details of when the due-diligence occurred for the purchase, this point is really moot. I hardly doubt your board decided to drop a cool 3.2B the night before the Nest announcement.

They certainly didn't tell the eng staff about it until then.

So in other words you got interested in Nest competition in the middle of Google-Nest acquisition negotiations?

The strange thing to me is that when we took action, our notice told Vivint it was for unnatural/spammy links to their site. It's really not hard to find these spammy posts, so I'm not sure where the disconnect happened.

Regarding timing, Blekko did a neat site called http://www.spamclock.com/ that claims a million spam pages are created every hour. It's safe to assume that Google algorithmic + manual protections against spam have to operate at that scale. As Del Harvey pointed out in a recent TED talk at http://blog.ted.com/2014/03/19/how-to-keep-240-million-twitt... , "[At Twitter], a one-in-a-million chance happens 500 times a day."

And since Google operates on an even bigger scale than the volume of daily tweets, you end up with these sort of occurrences. For example, it's virtually guaranteed that some website's rankings went up today, and they happened to start using AdWords today. Likewise, it's virtually guaranteed that some website's rankings dropped today, and they happened to start using AdWords today. There's no causality there--buying AdWords doesn't help your Google ranking--but to that particular website, it might look like causality even though it's just the law of large numbers. I think the logical fallacy is called "Post hoc, ergo propter hoc"?

The strange thing to me is that when we took action, our notice told Vivint it was for unnatural/spammy links to their site.

Yeah, but the perception of a conspiracy is the price you pay for handwavey/non-specific guidance about what's wrong. If your notices are not specific, then it's hard for people to know definitively whether or not they are in compliance. The flip side of that, of course, is that if you are fully specific then bad actors will exploit that in order to game the system. But that's the price you pay for ubiquity. Taking your power for granted may backfire on you if a regulatory agency or a jury decides you are not as objective as you believe yourselves to be. Intent is less important than how it manifests from the perspective of a disinterested bystander.

> our notice told Vivint it was for unnatural/spammy links to their site. It's really not hard to find these spammy posts.

Why dont you simply include these links in your notice, so that there is no ambiguity?

Because then spammers would become more effective at not being caught.

> our notice told Vivint it was for unnatural/spammy links to their site. It's really not hard to find these spammy posts, so I'm not sure where the disconnect happened.

the disconnect probably happened because you feel that telling people through an internal tool is adequate.

Great post Matt. Thanks a lot. Vivint is a super shady company in just about every way possible. It's no surprise that they resort to shady SEO.

If only Google could get rid of their Doucebag salesman as well...

I have a website 5000best.com/tools with a ranking of web tools, and I received so far about 40-50 link removal requests. I rejected them all. First I answered to those people, later I stopped answering, because that's waste of my time.

The reason of all that nonsense seems to be this article: http://googlewebmastercentral.blogspot.com/2012/07/new-notif... and notifications in Google Webmaster Tools that tell people that spammy links point to their website, but do not tell them specifically which links Google does not like. I understand that Google uses link data mainly to estimate popularity of a website.

I don't understand why so large company, a global monopoly with so large revenue, and so much data gathered, can't figure out which sites are more popular than others without wasting webmasters' time. What's so difficult about that task? Why shift any work burden on website owners?

If this task is too difficult, maybe it's time to support the competition, or create a serious competition. Maybe Google does not deserve to be the largest search engine and get all the profits.

Why dont you just "nofollow" the links? Everyone should be happy

I "nofollow" the links. Also that is additional work and unnecessary burden for webmasters.

Wouldn't that break the whole point of links and page rank? I link to things I genuinely like or find useful. The only things I nofollow are things I bitch about, because I am petty and don't want to give them any google juice. Other than that, a link is an endorsement, and if Google doesn't like me "liking too many things", that's Google's problem for not understanding websites that are the work of love.

It's actually much worse than that. Plenty of the links that those removal requests are all about were placed by spamming comments, creating millions of bogus accounts across as many services in order to boost PR. That was done automatically.

So now, after the penalty hammer comes down the automatically placed junk supposedly has to be manually removed by the website where the spam was posted, usually accompanied by some vaguely threatening words to the effect of 'if you don't comply we'll use the disavow tool'.

I wish them good luck and refuse to honour any of these requests, they can disavow until they're blue in the face.

See, sending out those vaguely threatening emails is still almost at 0 penalty (it is an automated action) but using the google disavow tool requires a human.

Preferably google should add some really nasty captcha or hard to solve puzzle there for every link disavowed.

Why did Google ask all those SEO companies to send e-mails to me? Why did Google give them a possibility to blackmail webmasters (disavow tool)? The Internet was not supposed to work this way. I should be able to link to whatever I want, and care only about the users, and not care about how competent or incompetent the search engine creators are, together with the SEO industry.

Matt, it looks like you and Google are clearly in the right here, and in fact you work with a very high degree of integrity as a rule. I really admire the job you're doing. However, there is an inherent conflict of interest, or, at least, the potential for a conflict, when the company that regulates and provides "fair playing field" oversight, Google, is also the company that is fighting competitive, commercial battles on so many fronts because of its many, many forays into paid services and products. Again, this is not a criticism of what you and your Google search brethren are doing – I'm with many people when I say I'm in awe of the job you're doing – but to point out the conflicted business model that Google is pursuing. Best, Joel MaHarry

Your examples don't strike me as low-quality or spam articles. Maybe if they were in Nature or Science or IEEE Spectrum. I think you should have a wider perspective, plenty of lifestyle sites I go on have poorer-quality articles that I choose to read, and I can certainly see paid relationships which are pretty transparent when I do so (as can most readers). I actually pay money to buy these magazines, and read articles like this (on many different subjects), so it is fair to say that I would pay to read content like this. I found them interesting.

I wouldn't have paid to read your comment though, probably because I don't care about SEO. (Not a reflection on your comment.)

Wait, wait, wait. Didn't your bosses recently complain about the EU directive to protect privacy as being "censorship"? Yet Google feels it has the right to determine what is "low-quality" or "spam" articles in determining web rankings? How's that not "censorship"? Also, do you ban everything that shows up on say, Buzzfeed, since by any measure that's low-quality and spam.

Thank you Matt. I used to be a Pando reader, but in the past months, for some unknown reason, they started publishing vindictive articles. No proper research, random arguments... while seeing themselves as 'true-seekers', in a very Machiavellian way. Now, as a former Pando reader I'm better off not visiting their site.

I might understand your action if you removed their company from search results when people searched for related keywords. But to remove them when someone searches for their company name is just wrong. You have enough engineers and resource to implement a better solution than to just wipe a company from the Web.

yes, this is a very good point and should be easy to differentiate between a branded company name versus (emd or other spam profile)

also, they continue to sell ads for these branded company searches to other competitors

Truce.

Hey dude3, I mentioned this lower in the thread, but we started dissecting this spam link network in November 2013, well before Google even bought Nest.

Truce

dude3, you know Larry Page doesn't have me on speed dial for companies he's planning to buy, right? No one involved with this webspam action (including me) knew about the Nest acquisition before it was publicly announced.

Truce

You flush a company from existence and then offer them zero help in getting their existence reinstated. This is something an asshole would do.

But this is par for the course when it comes to small businesses interacting with Google. They don't give a crap about helping. They will gladly take your ad dollars, but anything outside of paying them is seen as an offense that can result in your company being banned.

Google is evil. Let there be zero doubts.

Thank you Matt. Right from the man himself. I think we can put this so-called controversy behind us.

Google sometimes seems to be the sacred cow of HN...