Google Has Officially Penalized Rap Genius For Link Schemes
searchengineland.com
searchengineland.com
Justin Bieber All Bad Lyrics
Justin Bieber Confident Lyric
Justin Bieber Heartbreaker Lyrics
Justin Bieber Memphis Lyrics
Justin Bieber One Life Lyrics
Justin Bieber All That Matters Lyrics
Justin Bieber Hold Tight Lyrics
Justin Bieber Pyd Lyrics
Justin Bieber Change Me Lyrics
Justin Bieber Recovery Lyrics
Justin Bieber Bad Day Lyrics
Justin Bieber Roller Coaster Lyrics
Justin Bieber Lyrics
Most surprisingly, even
Justin Bieber Heartbreaker rap genius
They were using a wide array of questionable SERP optimizing techniques (http://www.rocketmill.co.uk/hideous-seo-strategy-rap-genius among others). One described here is very interesting: https://news.ycombinator.com/item?id=6958883. Apparently that's an old one, but it's first I'm hearing of it. From all of this, to their rape jokes and everything else in-between, you can't say they didn't have something like this coming.
I understand and agree with google's desire to punish spammers but their effort should not have such a major impact on users who obviously know what they are looking for.
Doubt they intended to penalize them quite this drastically. Its likely that they algorithm is just being overzealous some keywords being blacklisted. Quite similar to how the UK porn filters are filtering out Claire Perry, the most prominent campaigner for the filters[0].
> I honestly had no idea how severe these violations were until now
The violations are pretty serious and have been for the better part of the last (half-)decade. RapGenius should have known better. For me the worst was when the pretty much said "But everyone else is doing it".
Too bad this will obliterate them.
Unintentionally though, going forward RapGenius will be the poster child for what happens to children on the naughty list.
[0] http://www.forbes.com/sites/timworstall/2013/12/25/the-best-...
This looks like a -50 penalty. This is both good and bad for Rap Genius.
Good in that it will be temporary: specifically 30 days. This is better than a fundamental algorithm change like Panda or Penguin because there's no coming back from those.
The bad is that -50 penalties effectively destroy 90+ percent of your traffic. It's a very harsh lesson to be taught. You also don't want to have it happen again because the 2nd time is 60 days, and the 3rd time is permanent.
I don't think RapGenius will be doing something like this again.
From RG's perspective it's also frustrating because they'll see competitors do scammy things and get away with it. But that's the dual edged sword of such public personas. With RG getting called out so publicly Google had to do something about it. It's also good for Google that this penalty is getting a lot of publicity since it's proving an example to everyone else.
This was many years ago though (> 5) so likely the numbers have changed since then.
I ask because I have a site that had its Google traffic cut suddenly by over 75% in October. This lasted 64 days, until it increased just as suddenly last week, though not to the same traffic volume as before. I'm trying to figure out if it came back because an unseen penalty expired, or if there was something I did to trigger more conventional SEO rules.
Good in that it will be temporary: specifically 30 days.
This may take 90 days as it took JC Penney[3], 11 days as it took Interflora[4] or even 30 days. However, it should be relatively quick and straightforward for Rap Genius to recover from this penalty because, the penalty is based on links they've built themselves through their "tweet-for-links" system meaning that identification and removal of links should be relatively quick and straightforward.
This is better than a fundamental algorithm change like Panda or Penguin because there's no coming back from those.
In order to fix a Google Panda penalty you need to completely overhaul your site to remove any rubbish content as well as, to improve the content. In cases where you cannot improve the content to add unique user value, you should completely remove it from the site.
In order to fix a Penguin issue, you should see my overview here: https://news.ycombinator.com/item?id=6963785 although essentially, you just need to identify the links which need cleaning up, try and get those links "cleaned up" yourself, submit a disavow file (if necessary) and then get Google to reconsider the site.
[1] https://support.google.com/webmasters/answer/35769?hl=en
[2] https://support.google.com/webmasters/topic/2604771?hl=en&re...
[3] http://searchengineland.com/90-days-later-google-lets-j-c-pe...
[4] http://searchengineland.com/interflora-gets-google-rankings-...
Previously (like 5 years ago) you'd just see a massive hit to your rankings and would be scrambling to try and fix anything and everything you could think of that may cause it.
Like scoring an own goal, doing a victory dance, and then sprinting back out to score another. Just amazing. I wish I could find a violin tiny enough to appropriately express my sympathy for these clowns.
Google and Cutts love to preach this "Create value and quality content and you'll rank high" bullocks but it doesn't work like that. When you have a high number of well funded competitors who are all playing dirty, there is NO POSSIBLE WAY to compete with them with white hat tactics. You either play hardball yourself and risk a ban or you play nice and get creamed in the SERPs.
It's really a rock | hard place situation in some niches.
Oh, and I just saw the founders made this statement:
"We are working with Google right now to resolve this. They've been really great, helping us identify changes we need to make, even on Christmas. We’re working on it as fast as we can, and expect to be back on Google very soon."
WHAT? What makes these guys so special that they actually get help from Google. I think that pisses me off more than anything. There are a ton of webmasters out there trying to do the right thing SEO wise who get banned or penalized but could never, ever, get personal help from Google. We don't even get the courtesy of knowing why we've been banned most of the time. (Although to be fair this has gotten much better in GWT).
This is the worst Christmas present possible. Well, Google had no choice and the apology was not a stellar one (not that it could have mattered).
If I recall correctly, Google once penalized Google Chrome for doing shady SEO [1]. Although it was not as brutal as this which sends them to the equivalent of Siberia.
My Questions.
1. I normally search "<song name> rapgenius". Now, it is no longer on the first few pages. Is that not against Google's goal of giving the searcher what (s)he wants?
2. Probably the only way for RapGenius to counter this would be to go heavy on AdWords. Is that not a bit conflicted on Google's side?
Hopefully this punishment will not be permanent. I would hope they have learned to tone things down sometimes.
This may be a lesson in disguise as they would be forced to think of how to survive without SEO. If they can survive now, they will be doubly badass when this penalty is lifted.
--
Edited to include source
[1] http://searchengineland.com/google-chrome-page-will-have-pag...
It's not an answer to whether Google is giving users what they want, but you can just create search engines in Chrome[0] for the sites you search frequently. Make sure you set a custom shortcut for it that's only one or two letters. When I want to search Rap Genius, I type 'rg ' (note the space) and I get this: http://i.imgur.com/At9KxBr.png
I have this for at least 20 different sites, so I almost never use Google if I know what site I want to search, a prominent exception being if the site's search really sucks, like reddit. In those cases, I have custom search engines set up that will search Google without whatever I searched, plus "site:example.com" appended to it.
You can also hack this functionality to set up shortcuts for frequently visited sites ('hn' for Hacker News, 'gm' for Gmail, etc.) by creating a search engine without a '%s' (which is where whatever you search is substituted in). So I type 'hn', hit Enter, and it takes me to http://news.ycombinator.com
If you're finding that you continuously make such searches, it might be worth taking the time to set up your shortcuts. I have them set for Amazon, YouTube, Wikipedia, EtymOnline, etc.
Edit: That being said, searches on various sites often suck. My default search engine is Google's Feeling Lucky, so if I want to search and IMDb entry for example, I just type in `imdb <name of the movie>`. If I'd had imdb as a keyword for IMDb search, I'd go to the IMDb's search results and would have to click on the result. With feeling lucky search I end up on the entry directly. Same goes for Wikipedia and a few others. If I want to see Google results, I have a keyword for that - g.
Right, that's why I, if I were a frequent IMDb user, would have 'imdb' set up to do an I'm Feeling Lucky search with "site:imdb.com" suffixed to whatever I'd entered. Pretty much the same as suffixing 'imdb', but the "site:" syntax ensures that only IMDb results actually show up.
> Same goes for Wikipedia and a few others.
I actually find Wikipedia to be very good about sending me directly to the page of interest, without having to click through from a search results page. I use Wikipedia for movie info (rather than IMDb), and it works very well.
definitely. That should tell you just how pissed off Google is - they are willing to go so far as to give their users an inferior search experience just to punish RapGenius. This isn't about being fair or getting even, this is about setting an example. If they have to give a few people inferior searches for a while, i guess they feel that's a worthy compromise in order to discourage this kind of SEO tactics in the future.
> they are willing to go so far as to give their users an inferior search experience just to punish RapGenius
Another way to look at it is that RapGenius was willing to go so far in their mindless quest for bling and Bieber fans that they overstretched the rubber band. Google basically gave them what they begged for, and now all that is left is HN to get over it.
and none of these guys are geniuses either
and justin bieber cannot rap, so google is correct in this case
One tip if you want to search in a site you can always use the "site:" trigger.
Example :
we will rock you site:rapgenius.com
------
Even if the site is penalised it isn't de indexed ( I guess ) so you still can do a site search.
Thats how I search every website. I never/hardly use any website's search, most times its awful.
>If you go to Google and search for [rap genius], rapgenius.com will not be found on the first page
That's pretty crazy. Whenever I put up an obscure site, I notice ranking on the domain happens almost immediately with no effort.
[edit] Holy shit, I didn't realize quite how awful this is. Very often, I search for "<song name> rapgenius" because I really enjoy the annotations; that's not even on the first page of results. Making it worse, the first page (even when searching for a song) contains stories about them making rape jokes and spamming. https://www.google.com/search?q=today+was+a+good+day+rapgeni...
There are two lessons here: (1) don't do shady shit, (2) Google should only be one of many sources of traffic.
And the greatest gift of all was to be schadenfreude.
For people who act like arrogant jerks were punished severely that day.
For many of these high quality content silos (for lack of a better word), the PageRank paradigm seems pretty broken. People don't feel compelled to link to well-known sites precisely because they are well-ranked. And when they do, it is in form of viral posts in social networks, which is not really a great input to determine site quality.
I wouldn't be surprized if Google's real secret algorithm nowadays consists of millions of special cases maintained by thousands of poorly paid interns.
StackOverflow does well despite few people linking to them because the search terms are so niche, if I search for a fairly generic programming term I maybe get one StackOverflow post in the first 10 but if I search for an error message, they'll all be SO. Other StackExchange sites maybe don't benefit quite so well from that long tail, though. Sites like Quora presumably have the same problem.
It's interesting how sites like these, and in turn mobile apps, are turning information into vertical silos (which want you to stay within their bounds) rather than the horizontal web of equality we're used to.
(Also, the interns are pretty well paid, but yes. They start out as autocompleters: http://www.youtube.com/watch?v=blB_X38YSxQ)
But of course Google's algorithm is way more complicated now, and takes into account all sorts of things. They of course have all sorts of smart experienced people working on improving it, and apparently still determine that links should count for a lot.
I'd guess there are actually lots of links to stackoverflow.
That is, if page rank still works like Google claims. I'm pretty convinced that there is a lot of manual curation going on. At least for popular terms and for the first page of results.
Really? What makes you claim that?
They could have built a nice "Link to your favorite lyrics" widget for people to include in their "homepages". Make the appearance customizable, jumble the HTML, and include some stock bla bla to make it "individual" and to appeal to google.
Then announce a competition. We'll showcase the best homepages that link to our lyrics in the most creative way. Maybe give away some silly prizes. Completely legal, nothing for Google to complain about, but gives you the same effect very cleverly.
Instead they did this brazen crap... makes them look very amateurish. Technically not very different, but the tone is completely different, and Google had no choice but to penalize it.
(http://searchengineland.com/google-is-looking-into-rap-geniu...)
I do suspect that because of the attention this once case received, and since Matt Cutts was personally looking into the company's SEO tactics (talk about bad luck), they issued a public, but temporary, slap. That would serve as an eye opener to Rap Genius about how dangerously they're playing the SEO game and a warning to everyone else tuning in.
In any case, I find it hard to shed a tear for these guys, they either knew full well that they were flagrantly disobeying the Google rules or they are incredibly stupid. And unlike some others I'm not so surprised that their previously high ranking might have been gamed because they are one of a set of companies I know of that has huge mindshare with the HN/startup crowd, but virtually none with people I know who aren't part of that first group.
Google wants to make a point, they turn off the traffic faucet, Rap Genius repents, and things slowly return back to normal.
They don't even rank for "rap genius" right now. Their Facebook page does though.
Rap Genius alleges that their competitors are using the same tactics. Will those sites also get the same scrutiny? Perhaps now would be a good time for a new lyric site run by some nice bland people to go on the offensive and rise to the top.
One last thing to keep in mind: Rapgenius was an unlicensed lyric site. They weren't playing ball on either side of the fence. Let's hope that a lyric site emerges that provides lyrics with at least acceptable, non-distracting ads, and serves licensed content. I really think merely going by this strategy will get you ranked high on Google.
Rap Genius was originally called Rap Exegesis, and while it has a lot of rap lyrics it is a community supported text annotation site, with an ambition far beyond a lyric repository. They have a great deal of content beyond rap, or even lyrics in general. They have poetry and literature, announcements, and press releases (I'd love to see them make a Rap Genius page of Ulysses with the contents of Ulysses Annotated). They have a platform and are building a community. That's why it's attractive to VCs, and that's why it's something beyond the eyesores one mostly lands on searching for lyrics.
These companies(RG, azlyrics, etc) are making money where their main product is the creative work of someone else. Damn straight that other person deserves a cut of the profits because without them there is no site.
No, that's exactly how the world works for luxury items like music; demand drives, not supply. What the seller wants to charge means absolutely nothing if demand is elastic and consumers can simply stop buying. Demand sets the price, not the seller. Creators can set whatever price they like, and consumers can and will ignore them; if you want to sell something you have to set a price the market will actually bear.
> Damn straight that other person deserves a cut of the profits because without them there is no site.
No they don't; they got paid to write the song, they're done. The notion that a person deserves to be paid forever in secondary markets where people trade their creations is a perverse and absurd notion that exists only in the messed up modern world; it's wrong and it's stupid. You as an artist do not deserve to be paid every time someone makes a copy of your art; period. If I paint a copy of a Picasso, I don't owe the Picasso family anything and the same should apply to any other creation. Copies are not theft and the artist doesn't deserve anything.
>No they don't; they got paid to write the song, they're done. The notion that a person deserves to be paid forever in secondary markets where people trade their creations is a perverse and absurd notion that exists only in the messed up modern world; it's wrong.
Who is this magically being that pays the writer to write the song exactly? It isn't a secondary market it's a primary market. The song writer has created something(A song!) and the site wants to use that song. They are dealing with the song writer or copyright holder that is a first sale.
That person deserves to be paid for what they have created which is what this is. Your job wouldn't exist without that belief and neither would mine.
I never said they did, please do try and comprehend before simply repeating yourself.
> That person deserves to be paid for what they have created which is what this is.
False.
> Your job wouldn't exist without that belief and neither would mine.
This is a belief you hold, not a fact.
Whether or not a person deserves to get paid forever for their work is irrelevant to this discussion... The point is that someone is getting paid forever in this situation -- either the original songwriter or the owners of the lyrics site. I'll accept for the sake of argument that it shouldn't be possible for a person to monetize his or her work in perpetuity -- so then why the hell should it be allowable for an unrelated third party to do so?
Ok, fair question. I didn't claim a person, be it first or third party, shouldn't be able to monetize anything; I said they don't deserve it, not that they can't do it. I'm disputing the sense of entitlement displayed by the use of the word deserve. No one, whether the creator nor the third party, is entitled to consumers money.
In the vast majority of cases this is wrong. Most songwriters do not get paid to write songs. They get paid only when they receive royalties on the reproduction of their song.
This is good for society because it means that the songs that everyone likes will provide the most reward--so songwriters have a strong incentive to write the best songs they can.
Clearly SOME people are willing to pay for the right to use lyrics, ie. singers, record labels, etc. Just because a single individual, ex. rapgenius, doesn't want to pay for the music/lyrics, which they legally are obliged to do, doesn't mean that music/lyrics have no market value. There is a market value which has been established, and which people pay for. It just means that if they don't pay for the lyrics, they are stealing it.
Well since I didn't say that, you're wrong.
We agree.
> When someone else takes what someone does, it diminishes the value of that opportunity.
No it doesn't, those people wouldn't have paid anyway. Copying is not theft. This is the same tired argument the MPAA's been trying to make and it's just a sign of the establishment not adjusting to the new reality that distribution is not a profitable industry anymore.
You can't know all of the ways they will possibly make money from the content. Justin Bieber may want to charge Apple more for including vetted lyrics in iTunes. But why would Apple give Justin Bieber money when they can just copy the content from Rap Genius? Or cut a deal with Rap Genius for a feed?
> distribution is not a profitable industry anymore
This is where you're missing the point. You see the price as paying for distribution, but it's really about the value of the content. Historically, the two were inextricably linked by a physical medium. Now they are completely separate things with different prices and costs, and are almost unrelated to one another. Just because one goes close to zero, doesn't mean the other does, too.
Art doesn't have to be a business and art will be created regardless of whether artists can profit in perpetuity from their work. The notion that ideas and words are property and can be owned is perverse doublespeak meant to trick people into transferring their gut instincts about physical property onto things that quite simply are not property.
Performers should make money from performing, not being paid forever because the government created artificial monopolies on words and ideas.
Commenting on established artworks with a cute little javascript popup doesn't add anything to the original art. If I want to check lyrics, I am looking for the lyrics, not a popup that shows a picture of a ham when I hover over 'ham'. RG doesn't even have good content, just a bunch of smartass nonsense and obvious comments.
Never said they didn't, try understanding what was sad before flying off the handle.
(Seriously though, this is something absolutely different)
Is that close enough for you to see the similarity, or do I need to explain how songs and poems are functionally indistinguishable and further how poetry and prose is functionally indistinguishable; and therefore how written works are functionally indistinguishable and if you support copyright in one but not the other your perspective is logically untenable?
Or perhaps you'd like to explain to me why a poem should lose its written copyright rights when someone sings it.
We live in the AOL dark ages and most don't even realize it.
This would be the equivalent of Walmart/Safeway refusing to carry some company's products because of {false/mis}-representation.
What I'm saying is that when there isn't a lot of competition in the search space, businesses will use underhanded tactics to game a certain search engine. Having so much depend on a single entity is antithetical to the web and good, competitive business practices.
"RG clearly broke the rules and forced google's hand here"
One company's rules not some standard organization. Its Google's site, but they wield more power than Microsoft ever dreamed of.
Google sets their own rules for their own product. They don't run the Internet.
These RapGenius guys consciously decided to try to game Google and lost. Boo hoo. Can't win every game you play.
Yeah.. it really doesn't. The DoJ would be all over their ass if there was really a case to be made there.
> Therefore it's a rule for the internet.
That doesn't even logically follow if we assume the first. It is more than possible to use the internet without giving a shit about google's product. Even if it were a monopoly this would be true.
Google changing how their product reacts to RapGenius does not effect anything other than how their product reacts to RapGenius. Just because RapGenius cares a lot about how Googles product reacts to their own does not mean that the rules of Googles product are rules "of the internet".
If Google had the power to set "rules of the internet", then the MPAA/RIAA would have won their battle when Google changed how their product handles search results for torrents. Of course they did not, because Google cannot.
This just shows that you don't understand what a monopoly is. Monopolies are not in themselves illegal, so there is no reason for the DoJ to get involved unless they break one of the laws regarding what monopolies can and can't do.
And, of course they are being investigated for doing exactly that in Europe.
Can you hear yourself?
I'm not defending RapGenius - I think what they did was slimy. That doesn't change the fact that it's Google who decides the rules and polices them.
They definitely also work on improving their algorithm; it's just that often the algo does not automatically penalize or ban a site, but rather flags it for human review, which certainly makes sense to me.
google.com - 35.7%; youtube.com - 4.4%; google.co.uk - 4.0%; facebook.com - 3.9%; google.de - 3.5%; google.ca - 3.1%; google.co.in - 2.6%; google.fr - 2.5%; google.com.tr - 1.5%; google.es - 1.1%;
Except for facebook all it top sources are Google owned(58.4%). This will truly be disasterous for their revenues.
Considering they rank 59th for "rap genius" and 51st for "rapgenius" on Google.
There are tons of lessons to be learned here. Don't depend on a single source for traffic and most of all don't excessively try gaming the system, the system usually catches up.
Sure , now tell me what other search engine has the same market shares as Google. Answer : none , there is no competition in SEO, there is only Google. If there was some real competition , Google would not be so arsh.
There is only one lesson to be learned here : Since one depends on google for SEO, one should not try to game Google. because today basically , if you are not in google results you dont exist on the web.
Btw., duckduckgo still shows rapgenius on top.
In the short term. In the long term, this move probably has prevented a lot of black hat SEO and spam from happening on the web.
I really hope someone new comes along and makes a better site with the same premise (annotating lyrics).
For instance, this has happened in the past to well known brands such as J.C Penney through NYTimes expose[1], Interflora[2] more recently and a lots of others.
An apology which RapGenius offered [3] doesn't fix this either.
Is it fair? Yes and No.
The only reason it isn't fair is that the site disappears from Google for the BRAND term e.g. [rap genius]. My personal belief is that, devaluing the site for the BRAND term e.g. [rap genius] actually devalues Google's quality. On the other side of the coin, if someone searches for [X rap genius] whilst they are under penalty its fair that they do not rank for that either. However, there are obvious reasons as to why the search quality team have decided to do this.
How RapGenius can fix it / How you can too if your site gets a penalty:
First of all, RapGenius if they are doing any link building now they should pause it immediately until they’re out of penalty.
Secondly, in their apology [3] said:
"With limited tools (Open Site Explorer), we found some suspicious backlinks to some of our competitors"
Anyway, they should download all the links in WMTs, OSE, Majestic etc (although it looks like they only have OSE[3] so they should just download them from WMTs and OSE) and then remove the duplicates.
Once they’ve done this, they should flag every single link which, they believe is causing the penalty.
After identifying all the links which are causing the penalty, they should create a Gmail to outreach to all of the sites to remove the links. They should outreach to all these sites and documents all the sites they’ve contacted, status – still live/nofollow/removed/requested payment/no response etc.
Having got some links removed/nofollowed etc, they should then disavow all the other sites that have requested payment or not given them response to the removal. Personally, the disavow(s) that are done by myself are usually done at the domain level although, there are reasons to do this at the URL level as well (Rap Genius needs to make the decision which one to disavow).
After submitting the disavow they, should submit a reconsideration request which outlines, the work they have done – from the spreadsheet – and also offer Google’s Search Quality Team the login to the Gmail to show they’ve tried to get the links removed and that some people have asked for payment etc.
The Google Search Quality team will review the site then, they’ll either flag more links to be removed or they’ll get out of penalty – after which Rap Genius will start appearing for the BRAND term again and other results once the Google Algorithms trust the site again.
[1] http://www.nytimes.com/2011/02/13/business/13search.html?_r=...
[2] http://searchengineland.com/google-says-no-comment-on-why-in...
[3] http://news.rapgenius.com/Rap-genius-founders-open-letter-to...
So if instead of being rapgenius.com, they had shelled out big bucks for the domain lyrics.com, they would continue to rank for searches for "lyrics"? Our would someone at Google make a subjective decision about what terms are unique enough to their brand for them to not be penalized for?
Either of these seem much less fair to me than the status quo that all spammers get penalized for all search terms.
Google actually have done an EMD update to devalue sites trying to rank for generics using a generic domain.
As you can see using Google Adword Keyword Planner[1] for all locations in English the average searches are:
Lyrics - 1.2M avg. searches/month
Lyrics.com - 110k avg. searches/month
Could you possibly give me a tldr on this story?
I recall the exposé talking about how they were trading twitter links for keyword link building. Yet I'm having a hard time working up the requisite outrage.
It's not spam - the people linking back aren't being coerced (i.e. spam comments), and the content it's being linked from is legit (i.e. not a crappy link-farm site void of content).
Is the reason that this is Bad(tm) because the link back is not "organic"? It strikes me as being identical to say, paying every a thousand bloggers writing about Bieber to link to RG - except that last example is impossible to detect. They exchanged a small ad for twitter inbound traffic.
Is there just a blanket ban on trying to divine how the algorithm works? It's a commonly accepted practice to pay other people to promote or write about your product/service/brand.
Buying or selling links that pass PageRank. This includes exchanging money for links, or posts that contain links; exchanging goods or services for links; or sending someone a “free” product in exchange for them writing about it and including a link
However, Penguin is not the only way which Google identifies people being involved in these practices as they also have a place to report the links[3].
This is one of Google's Manual Actions[4] that webmasters receive, when Google believes you are not providing the user with additional value and/or are trying to manipulate the results.
They cover everything from Thin Content (mainly through Panda) to Hacked Sites to User Generated Spam to the recent Image Mismatch Penalty etc. You can see them all here: https://support.google.com/webmasters/topic/2604771?hl=en&re...
[1] https://support.google.com/webmasters/answer/35769
[2] https://support.google.com/webmasters/answer/66356
[3] https://www.google.com/webmasters/tools/paidlinks?pli=1&hl=e...
[4] https://support.google.com/webmasters/answer/2604824?hl=en
How is that different from paying bloggers to write a product review with a link to a product? Why is not that considered a "manipulation of SERPs" - you exchange (money/tweet) for a link.
Those types of links reviews for a link, are considered advertorials which Interflora were "famously" penalised[3] for and something Google specifically identifies within their Link Scheme examples[2]
[1] https://support.google.com/webmasters/answer/35769
[2] https://support.google.com/webmasters/answer/66356
[3] http://searchengineland.com/google-says-no-comment-on-why-in...
I was about to ask if the lesson to draw here was to make it look organic.
But mostly serious question here: how does TechCrunch have any Google rank then? How do most tech publications, for that matter? PR hits are extremely common; they're the name of the game when it comes to cheap content.
It's okay if it looks like it's mutual self interest, and not if it overtly competes with Google's advertising platform?
TechCrunch does not post advertorials, they post content for free and they actually posted on their site to reaffirm this fact[1].
Sure there may be some PR firms etc who might get paid to get that content on to TechCrunch or another tech publications but TechCrunch writers are not directly compensated for doing so.
As a result the content they write from a startup launch to a new feature etc is considered "natural" by the search engines as they're choosing to write about it.
Moreover, those publications get the majority of their search engine traffic through being in Google News ala. posting about "Twitter" and getting inserted into the "Twitter" SERPs within the "News" section. Additionally they also leverage internal linking to boost their SERPs potential e.g. whenever they talk about Zulily (they're in #9 for me in Incognito mode although they might be higher/ on Page 2 for you) instead of linking to the site they'll reference the "Tag URL"[2] as well as referencing Crunchbase[3] (although CrunchBase isn't really 'internal' as its an external site). Likewise, Google loves "fresh content" so they will naturally be inserted with little/no links within the top 7ish results for something generic (although this does not always happen) such as "credit card numbers" when they do a post about "credit card numbers" and will naturally lose search engine positioning for that term over time.
[1] http://techcrunch.com/2012/11/08/we-are-worth-at-least-3k/
However, they're acceptable if they do not carry any link equity (aka. use the no follow tag) on the link.
[1] https://support.google.com/webmasters/answer/66356
[2] https://support.google.com/webmasters/answer/35769?hl=en
Typically they don't know what nofollow is, and their sites are run by people who don't really know much about websites.
I can't educate the entire sector. Is it better to steer clear of link building with this broad swath of sites? Most of the other sites in my niche have built links through these sites.
There is a shortage of followed links in my niche as most content creators in my narrow field are commercial and don't link to competitors.
Which is sad.
SEO is a cancer and, like cancer, there is no good SEO. Google and their broken search ecosystem have spawned a vast, useless, derivative "industry" that needs to die.
I say "broken ecosystem" because SEO should have a value that is inverse to how well google indexes. That SEO exists at all shows the extent to which it is broken. Website death penalties (or whatever we call this) don't solve the problem.
SEO indeed is like what you described, but there is a fine line to start doing blackhat or stay whitehat.
Now the problem is, the line is so thin you end up crossing it at times. And everybody does. The way Google works you need to do both somewhat in a way to get the balance and stay top.
For example, link-building its one of the worst areas where defining white-hat and black-hat is really difficult. One way is doing guest post is allowed and great but paid posts aren't. There is no guarantee a guest post is not a paid post.
There are 'digital marketing agencies' which at times are so ridiculous that the backlink included in the guest post doesn't have any value in to the post as such but anchor text magic does all the work.
SEO isn't broken as such but there are too many backdoors to doing the white-hat SEO methods in the concealed black-hat way that it isn't realistic to blame he search ecosystem.
I agree, website penalties is not a good solution to the problem.
You use sites because they used good SEO. You can find StackOverflow posts ranking at one, because they used the good SEO that the good content deserved.
Anyone with an internet business taking your comment at face value will find herself ranking not in the top 20, no matter how good their content.
Good SEO has worth, has value. Even Matt Cutts promotes good SEO and sees the worth. Without SEO analytics you are sailing blind. Without knowledge of how to interpret them. Without knowledge of the guidelines you get gaffs like these.
There is good SEO and there is vandalization. Rap Genius vandalized their rankings.
No industry needs to die. If it needs to, explain why.
Optimize your site for all users, not just search engines.
If you stay in this naive view of SEO you will never have a successful internet business in many niches. What is sad is that some developers and designers can not deliver a website that is SEO optimized, so a business owner has to pay double to get an SEO to fix all on-page accessibility issues.
This penalty is entirely the result of that blogpost and the PR it generated. I am willing to bet that Google already discovered many of these links, through spam reports, through manual spam fighters and through algorithmic detection. This PR, and the open letter, forced Google's hand. Of course they have to visibly act on it. Of course they are well aware that that niche has some scummy shady practices, and Rap Genius is not the only one. Of course they have a better internal tool than OSE and can check link profiles and detect networks in a manner of minutes.
Besides, you can be a great SEO and never do a single link-building campaign. There is tremendous value in simply finding new content and articles to rank for and drive traffic. I loath black hat SEO, not for the unfair temporary advantage it gives, but because it creates posts like yours. Because it burns companies that are now much more careful to hire good SEO's. Because I know that good SEO helps people access relevant content.
Like e-mail spam, search engine spam will die. SEO's will be retrained to create websites that are accessible, user friendly, give content that is relevant to the search query.
Just read: http://static.googleusercontent.com/media/www.google.com/en/...
And try to find a single practice in there that is either manipulative, entirely done to service search engines alone, or hampers accessibility and user friendliness.
Good SEO builds authority. Creates quality mark-up. Makes descriptive website titles. Implements breadcrumbs and a sane website hierarchy. Rids the index of duplicates. Drives more users to your sites. Makes sure that blind people can access your content, like images or video. Makes sites load faster. Makes content more trustworthy etc. Good SEO needs no manipulation. There is a lot to gain by simply aligning yourself with Google's vision.
Please reconsider your tone, or realize you do not add anything of value to the debate, or to the many online business owners on HN. Can you imagine raising funds for an online startup and then have your post be the slide for "online marketing"?
... which is a direct indictment of google and their (as you put it) ineffective search results. If the best content is not being served based on its merit, the search results are bad.
A very good explanation for this is that optimizing for good search results and optimizing for high ad clicks from searchers are two different things, and you can't do both. Therefore the price of optimizing for search-ad-revenue is less than optimal results and the unintended consequence is a side-game that parasites play called "SEO". I mean parasite in the nicest possible sense and I think it's an apt description.
"Please reconsider your tone, or realize you do not add anything of value to the debate, or to the many online business owners on HN."
I'm not here for the business owners - I came for the hackers.
I think Google has the best quality index of all search engines. There is a Kaggle challenge for Yandex right now where you can optimize the search results taking into account data like user sessions and dwell time. It's hard, but very cool. Google owns hard and cool stuff. Web spam is a multi-faceted and complex problem. But these guys are training neural networks on Youtube stills and make it detect cats. The last time I found a top 10 spam result that irked me was months ago. I filed a report and move on to the otherwise great index.
>If the best content is not being served based on its merit, the search results are bad.
If the best content is in an image without an alt attribute or longdesc or HTML fallback, on a page with no surrounding text, no sourcing, no pagetitle, no meta description. On a domain that accidentally blocks search engines with robots.txt, has no structural interlinking of pages, no backlinks, takes 50 seconds to load, redirects crawlers to a different page by IP, and only works with javascript on. If that happens to be the best content, then that is a shame. Google could probably still index it :). But the search results are better for not ranking that inaccessible, untrustworthy, undiscoverable piece of content very high. A store can sell the best goods, but if they put blinds in front of the window, do not promote or advertise, make customers crawl over obstacles to place their order, then such a store will simply not do very well.
>optimizing for good search results and optimizing for high ad clicks from searchers are two different things, and you can't do both.
Why not? Major sites both optimize organic results and their SEM. AirBnb could create content for each major city they host in and rank organically. They could advertise locally or very targeted to people interested in making use of their services and link to the actual property page or listings.
>I mean parasite in the nicest possible sense and I think it's an apt description. I'm not here for the business owners - I came for the hackers.
Google compresses nearly every information in the world to a single search query. That search box is the shortest program to expand to any related content that is currently found in the world. A giant information retrieval intelligence sends out its bots to crawl information every second and store it in global time consistent databases. From a black box algorithm with supposedly 200+ ranking factors, an SEO has to give each web page the optimal chance to rank for what its worth, by reading public documentation, experiment, analyze, track, predict, format HTML documents in an information retrieval friendly way, do split testing with contextual bandits or multivariate A/B, improve accessibility, improve the link graphs of the internet and semantic web with metadata. You come for the hackers, yet you treat SEO's and Google like script kiddies.
Just slightly curious about this, because I receive these 'link removal' emails all the time for a site I run and have never acted upon them (our comments pages are indexed by google but not linked publicly after we switched to Facebook comments). Is there any reason a website owner should act and remove the links? Surely it's not my/our problem?
Is there any reason a website owner should act and remove the links? Surely it's not my/our problem?
A website owner does not need to act and remove the links at all, you have the choice to not respond or refuse to do so. This is why Google allows people to disavow those links via the disavow tool[1] so they are no longer "counted" as providing the site with any link equity.
[1] https://www.google.com/webmasters/tools/disavow-links-main
What (if anything) happened between Google and Reddit, by the way? I remember Google results being stuffed with basically identical versions of the same Reddit comments pages (due to Reddit's language-localisation scheme: it.reddit.com , pl.reddit.com and so on), and then big chunks of Reddit (in all language versions) apparently becoming invisible in those Google results.
However, Google over the years have also worked on their international/localisation identification of sites. As a result they allow webmasters to geotarget them within WMTs[2] which seems to resolve the Reddit issue.
Having said that, there are other methods to fix the issue as well such as using the geotarget option within WMTs to using the rel alternate tag etc.
[1] http://googlewebmastercentral.blogspot.co.uk/2010/03/working...
If I remember correctly the site was completely deindexed.
How is that not fair? They were caught attempting to cheat google. Not only that, but cheat their competitors and us (the users). In most scenarios (criminal, sports, academic), being caught means it's game over. And that's exactly what happened here.
When you get caught stealing from a cash register or breaking into a house, they don't just make you put the stuff back and send you on your way. When you caught cheating in professional sports, you forfeit the entire game; in higher academia, kicked out of school.
If anything, I think google is too soft on people attempting to cheat them. When it's obvious and blatant, they need to lay down the law so hard that people won't even consider it next time. This will make the user experience better for everyone. A slap on the wrist tells people the risk is worth it and that means we will be served up potentially worse (less organic) search results.
How is that not fair?
If anything, I think google is too soft on people attempting to cheat them.
Since Google offers a public service and is owned by public shareholders, this poses somewhat of a problem....especially when you consider their marketshare and whether or not such sanctions offers them a competitive advantage.
Actually, in all of those cases, the judge and executioner (and, usually, the legislature making the rules) are all employees of the same organization. In some cases there is a separate jury as a finder of fact (e.g., in criminal cases in the US and countries similar legal systems), though in sports and academic cases there may well not be, depending on the particular rules of the particular organization.
but I kind of want to punch the CEO of Rap Genius in the face, for trying to shame the competition in an apology that they are responsible for.
Eh..
I absolutely second this because I just did a search for Rap Genius and it only showed twitter.. So I did a search for it on Bing instead.
Yes, that's exactly what spammy link creators should do: add insult to injury by ordering the sites they spammed to clean up their mess. Bonus points for adding a supposedly scary "or we'll have no choice but to ... disavow you" rider.
#42 (was #1) - justin bieber memphis lyrics
#43 (was #1) - justin bieber all bad lyrics
#44 (was #2) - justin bieber hold tight lyrics
#45 (was #1) - justin bieber one life lyrics
#46 (was #4) - justin bieber bad day lyrics
#48 (was #7) - justin bieber lyrics
#54 (was #3) - justin bieber change me lyrics
#59 (was #2) - justin bieber pyd lyrics
#60 (was #1) - justin bieber confident lyrics
#63 (was #1) - justin bieber heartbreaker lyrics
#64 (was #5) - justin bieber roller coaster lyrics
#68 (was #2) - justin bieber all that matters lyrics
#68 (was #3) - justin bieber recovery lyrics
#50 - rap genius
source: serpscan.com
edit: Followed links to past stories and got my answer here http://jmarbach.com/rapgenius-growth-hack-exposed
Here, they are bribing customers to link to their site? Would this not be illegal too?
Also, illegal isn't really the right term to use here.
If it is an ad, then it would definitely be safer to mark the link as nofollow as ads are not allowed to be links with SEO benefits.
I'm glad they have a proactive slap-down policy. Let's not go back to the late 90s and the likes of altavista, when you might have got one hit in ten pages of spam, SEO link farms.
At least https://duckduckgo.com/?q=it+was+a+good+day+rapgenius seems to work fine.
I've lost one Alexa top 2000 site to a google penalty, and several smaller sites, saw this coming miles away.
I know exactly what they are going through, never forget the day when my JV partner called me at 4AM and told me the apocalypse scenario has happened. Even worst than the financial blow of losing a $30k a day advertising property, was the brutal realization of googling your brands name and seeing it not show up. Going in to analytics real time and seeing a sub 100 number for the first time in 6 months was quite an eery feeling as well.
Google did this to set an example, and they will be reinstated worst case in 6 months.
1. Breaking rules
2. Not being liked and breaking rules
I performed the following searches in an incognito tab and could not find rapgenius in the first two pages of results (I did not look beyond the first two pages):
"rap god eminem lyrics"
"machine gun funk rap lyrics"
"lose yourself eminem lyrics"
I think you're right, that depending on how long this lasts, they could be in a world of hurt.
Since what they do have to offer is about a million times better than any other lyrics site I’m pretty sure they will weather this. I mean, they have to. Their service is pretty excellent, I’m not even sure why there way any need to fuck around with Google.
Clearly what RapGenius did was wrong, very stupid and extremely short sighted. However, you're still going to find a good percentage of your search results today from rankings achieved with equally as shady strategies.
Google is too big & too powerful to play these kind of games and arbitrarily decide who gets penalized and who doesn't. Dare I say it but part of me feels like there is not enough bureaucracy in large penalty decisions considering their impact. There are literally millions of other sites doing the same thing and happily continuing their day.
Source? Proof? RG was penalised because their method were made public and Google was made aware of it. If there is proof other sites are doing the same (I don't doubt they are but proof is needed) I'm sure Google will penalise them too.
Outch, this is going to hurt them.
I wonder if it's a worthwhile risk. While there are some high-profile busts, in general most sites probably don't get caught. Though it's probably harder to do effectively since Google improved their algorithms.
In the flower case [2], Google claimed the links hadn't helped the sites. Though Google couldn't have penalized all the Flower companies, since that would have made their results much worse.
[1] If you're paying someone to link to you, that needs to be disclosed, but that doesn't mean the link needs to be nofollow. Though I guess following that rule would make it very easy to be caught.
[2] http://searchengineland.com/ny-times-covers-paid-link-scheme...
Their overall link portfolio is not half as bad as their competition, however all this attention forced Google to take action being that they need to live up to their reputation as hard asses.
Let's face it, the algorithm might not pick up on these bartered links because they are a drop in the bucket of their overall link portfolio.
Really more than anything this is another PR stunt by Google's search quality team. They have been outing people on Twitter for the past month (backlinks.com I believe and another blackhat service) and want the general public to know that they mean business.
If you're not scared of google's double talk, blatant spying, abuse of monopolistic powers, then you either work for them or you are completely alienated.
I really like their n-gram viewer of the New York Times wedding announcements. http://blog.visual.ly/rap-genius-new-project-visualizes-30-y...
I would be interested in learning what they have to say about NLP (Natural Language Processing) since they deal with language which is quite vernacular.
Controversy aside - This seemed like the most interesting part of the write-up to me.
Seriously how hard could it be ... write a Node.js app that can handle 10k clients on one machine, and partition the data by the song being annotated. Don't even need $1 million dollars. YouTube hosts the videos, anyway.
Ah, but there is just one problem... song lyrics are copyrighted and a license to some database is like $20,000
Wish I knew a way around that.
RapGenius grew because their user experience was much better than their competition. This was the foundation for their branding & marketing. Their growth will be sustained by their content and (once these penalties are phased out) by users' ability to discover their unique content. The technology is merely an enabler for the community-generated content. New competitors must find ways around the community's cold-start problem.
Edit: typos
http://www.billboard.com/articles/news/5812335/how-rap-geniu...
As a result of this story, I now have heard of RG. So ironically, any press is good press, and despite the banning, and the public shaming in the media, it may very well be the equivalent of millions of dollars in free brand awareness.
http://cdn.elite-strategies.com/wp-content/uploads/2013/12/r...
The power of Google.
"Its not cheating unless you get caught"
for example TINT
tintup.com
once you create a free account they have the following:
Dear valued Tint customer, Would you like to write a review about Tint? In exchange for your opinions/blog post, we are giving away 50% off FOREVER promo code for our Plus Plan. To learn more, click the button below:
screen shot: http://imgur.com/wDnAJ4c
RG may want to consider a hedge bet on search engines.
[bieber heartbreaker lyrics rap genius] or even [bieber heartbreaker lyrics rapgenius.com]
I have mixed feelings about this. How long until we see a game of he-said-she-said? Of course, socially engineered link building aimed at gaming the SERP is against the interest of consumers and google. However, now that we can see a competitor get taken out from google SERP by pointing their attention to a malicious technique, we should see more stories like this.
The problem I have with SERP is that while it's good at finding the most relevant site (still returns spammy sites so not perfect) most of the time, it's built around the advertisement model. It doesn't give you a snapshot of the cross section of the data you are looking at across all the domains out there so that you can't choose on your own which lyric is the most to your liking (it really doesn't matter because lyrics are same across different websites). So the question of who ranks first even though they have identical content shows some underlying inefficiencies caused by the Advertisement revenue model. The more there is, the more advertisement gets seen and clicked.
I feel that the search engine space with selling our attention to advertisements only creates more waste by others trying to get a share of the revenue by jumping into a profitable niche by creating an identical or similar content (article spinning etc). In turn, more competition would create people to use more Adwords or other advertising platforms. The end result is that consumer ends up with untrustworthy websites with content in the back of their priorities. Content should be judged by comparing it with other websites that offer the same content with an overview listing all the different domains showing that content. While displaying such cross domain contents, it should not bring advertisement as an incentive model.
What is wrong with that anyway? Adwords and Adsense are just an affiliate program anyway.
I dont see much of a problem with it.