Rapgenius Uniques Fall 90%
quantcast.com
quantcast.com
Google has a domineering position in the market, and it's a bit ironic that people were quick to call for Microsoft to be split into multiple companies about a decade ago, yet nobody bats an eye when Google exercises their power. (I personally don't believe either company should be split; just pointing out an interesting parallel.)
So what can Rapgenius do? Are they screwed?
EDIT: Also, how long does the penalty last for? I think people are assuming 30 days, but is that really the case?
Even still, 30 days is a long time. They may not be okay afterwards. Whereas, say, a week-long ban would've had the same deterrent effect.
This is a penalty that they deserve. They took a risk and they lost. They knew what they were doing and the potential destruction of their traffic, and they did it anyway.
You don't get less time in the penalty box just because you're the favorite or best player on the team. In fact, you have a greater responsibility to play by the rules and I would make the argument that your penalty should be even greater.
Besides, if the refs played favorites, the refs would have no credibility.
Or kill their brand and reboot as a different one with a different URL on a different IP.
Google has indicated that domain names that change hands get reset to zero, so buying an aged domain wouldn't help.
It's possible they'll be bigger, what with all the free (though bad) press they've gotten.
Apparently they already anticipated something like this because several other companies were being manually penalized over the past year and half following an algorithm that seemed like: "Which big AdWords spenders cut down significantly on their budget? Are they getting it organically now? Let's cut that off."
Several of those companies are suing Google for violating EU regulations, so hopefully something will come out of that.
The first step will probably be that penalizations can be fought through the EU judicial system, but it will be a while before we get jurisprudence on that.
I don't know how this would work in the US legal system but, regardless of what the outcome would be in Rap Genius' case, it's probably time for similar jurisprudence to be established in the US.
http://searchengineland.com/google-research-even-if-you-rank...
A short penalty of a few days or weeks isn't much of a deterrent. It may suck for people searching lyrics now, but making the page rank penalty effectively toothless would suck for everyone.
And RG didn't just "screw up" by trying to game the system; they were blatantly cocky about it, so that's probably why Google felt they needed to step in and teach them a thing or two. Let's hope the lesson sticks, at least.
BS.
What's good for Google is showing people that if you engage in bad practices your shit will get nuked from orbit, even if your site is better than the competitors.
You'd better believe that companies are going to avoid blackhat SEO in the future after this, and that's absolutely in Google's interests.
I think that anyone who characterizes google as needing one site over needing people to not blackhat will find that google really doesn't need said site. (Maybe Wikipedia is an exception to this rule?)
Can you find another example of the other lyrics sites pushing a quid pro quo scheme on Twitter?
The other lyrics sites might be scummy and Google doesn't care -- or they might just cover their tracks enough to avoid giving Google enough proof to hang them.
Isn't that precisely what rapgenius had done (affiliate linkscam notwithstanding)? They were already the top result for many lyrics queries, they just got greedy
I applaud Google for this, and that's the first time I've applauded Google in a while!
I work at an online marketing firm and I've seen dozens of companies come to us for help with the process. RapGenius shouldn't get preferential treatment.
AZLyrics is a lot more appealing to me. Much lighter weight.
So, we have the PA.c Installer able to take directives like whether or not to preserve everything in App, whether to preserve or remove specific files, whether to preserver or remove specific directories, etc. In addition to overwriting the existing files with the new ones, of course. This made life a lot easier on everyone. And users didn't need to worry about manually deleting specific directories but keeping others when updating. You just point the new version to the same X:\PortableApps\AppNamePortable directory and it intelligently upgrades. It even supports custom code for things that fall outside those simple keep/preserve so we can specifically remove certain settings on an upgrade where a given app breaks its own settings and doesn't support the old ones, etc.
For example, I just wrote the package for FreeCommander XE Portable for the publisher to release. The new one doesn't support the old settings and will break if they remain, so I have the installer back them up within Data (just in case the user wants to revert manually later). I then had it read the preferred language from the old settings and write them into the new ones for the user for convenience. I could get more explicit and read/write other settings as well if I had the time.
To rectify this, they crippled my search results by penalizing Rapgenius and instead pushing some AZlyrics junk that not only lacks annotations but is inaccurate.
Seems like an easy fix, for the end user at least, add a keyword search for 'lyrics' to go directly to Rapgenius, right? No. Here's a search for smoke on the water
http://rapgenius.com/search?q=smoke%20on%20the%20water
First result? Pro Era – Like Water. Second? Childish Gambino – I. The Worst Guys. Third? Smoke DZA – Diamond.
Arguably the most popular song in the history of rock and roll? Sixth.
Those are some of the smartest people on the planet. I'm not being sarcastic. How depressing.
Why should Google give special dispensation to RapGenius? Why should putting in something like "lyrics" or related content to someone being penalized get them to show up higher in the results?
Why should Google be RapGenius' search engine? Why shouldn't RapGenius make their search better?
Which they don't after excluding RapGenius. They purposefully broke their search results to protect their broken ranking method.
How can you prove RapGenius is the "best result"? Especially considering that courts have ruled that ranking algorithms are just opinions. You can be of the opinion that RapGenius is better, but that's your opinion. Multiple people can also be of the same opinion, which is a huge factor involved in Google's ranking algorithm. Another huge factor involved in their ranking algorithm is dissuading blackhat SEO techniques, which can make it appear like lots of people are of the opinion something is good.
So, again, while you are of the opinion RapGenius is the best lyrics site, I'm not, even before all this brouhaha started I didn't really like their site, especially the annotations, which to me seemed like a bunch of bored English majors who didn't really have any insights on the annotations they were making, but wanted you to think they were really smart. I also don't listen to rap music, which might also affect my view on the annotations.
They changed the handling of search queries (searches are interpreted much less literally than a few years ago). They changed the way that pages are indexed, and changed the intervals to be smarter (faster for news sites, slower for sites that don't change). They now even "render" the pages partially to account for CSS and JS. They have smart stemming and synonym algorithms when building the indexes. But the actual algorithm - "PageRank" and the parts surrounding it - hasn't changed much I think, except for some parameter tuning.
Instead, they probably have a pile of semi-automatic "filters" to deal with spam and blackhat SEO. Engineers write a filter in code, test it against a test index, and then it goes into the new index version. I believe they are going more and more into manual intervention, and that they can now directly penalize/push sites without going down to the indexing level.
I can't prove it right now, but I'm pretty sure that the first SERP for popular search terms is actually manually curated, with the really organic links coming later. (I'm still trying to figure out a test methodology to analyse it :-).)
And how to find smoke on the water? Maybe put some quotes around it? http://rapgenius.com/search?q=%22smoke+on+the+water%22
What I'm trying to say is, Google doesn't have the monopoly because they're the only ones who can build a good search engine. The have the monopoly because they do have good technology, but also because of path dependence, and because there is no political effort to build up a competitor.
I already posted a couple of days ago, there are a bunch of ways governments could support competitors when one search engine controls a large fraction of the market, say more than 2/3. For example a special monopoly tax, subsidies for competitors, etc.
One thing I think would be nice is to make search engines offer a free/very cheap API (as long as they have > XX % market share). Anybody can come along and build a Google + Bing + Pokemon-Wiki meta search engine. If you tune it to be profitable, you'll have a lot of competing search engine frontends spring up. But you'll also have a lot of search engine backends (and not just Bing, Google, and Yandex), because its suddenly profitable to build verticals (e.g. indexing just tech news or recipies), which require less resources technically - because the vertical backends get traffic through the competing frontends.
And I could choose a frontend that gives me Google-like searches, or one that interprets terms more literal (like when you use +"term" in Google), which would be great for programming related stuff. There could be another one focussing on social media, or local stuff, etc..
Why doesn't this company get penalized? I have no idea. I guess nobody decided to write a blog post about it, it didn't get picked up on Hacker News, and Matt Cutts hasn't noticed it yet. Seems like if this is possible there should be a reverse black hat market springing up soon. Do some really bad black hat SEO for your competitors and then get them killed by Google.
Of course many spammy sites use this practice but they don't have the respect from the tech community that Rap Genius warrants as a Y-Combinator funded entity. They lost that, and it's going to be a steep climb for them to find talented folk that want to be part of a seedy operation.
To everyone here who's saying they brought this upon themselves, when rules are unenforced and all your competition breaks them, you've already hamstrung yourself if you don't break them too.
If everyone in the MLB is juicing and a rookie gets caught, you don't say "what an idiot", you look at either 1) enforcing the rules on everyone and broadly changing the culture or 2) question whether the rules make sense in the first place.
Either punish everyone or punish no one.
Find and punish all the murders or let them all go free!
First result for "jay-z holy grail lyrics" Google: http://www.azlyrics.com/lyrics/jayz/holygrail.html Bing: http://rapgenius.com/Jay-z-holy-grail-lyrics
The Bing result is better (though both are valid).
I would suspect that most sites pulling these sort of SEO scams are going to have lousy content, so the rule makes sense, but perhaps there should be some sort of human element to it as well? That's not scalable, though, so I'm not really sure what the best answer for Google is on this one.
It helps them in the long term, as it dissuades sites from using blackhat SEO if they know their company can suffer severely for it
Additionally, with AdBlock on, AZLyrics makes only 21 requests and takes 1.03 seconds to load completely on my machine. RapGenius (still with AdBlock on) takes 27 seconds to load, with 208 requests. Based solely on that I'd rather use AZLyrics than RapGenius.
Or taking it one step further, use these tactics to promote a controversial institution or politician (without them knowing). Then "leak" it to Google, so they get punished. Then claim Google is censoring them. Finally enjoy the outrage :-)
The link selling, not the meta data of their pages, was what caused this. It's likely that these other lyric sites do these sorts of arrangements in a less public way, whereas RapGenuis was using Twitter, of all places, to sell links.
Mind you, this is not the exact behavior RapGenius engaged in. They paid for backlinks, but they paid legitimate sites directly for links in blog posts. Either way, though, it's still blackhat.
It's a really interesting world if you feel like reading up on it at some point. I've always done basic SEO (read: real, legitimate, basic stuff that I'd never even advertise as 'whitehat SEO' since that nearly always means 'blackhat' but we'll pretend it isn't) for my personal sites and client sites over the years. But you're going up against these 'players' who routinely setup networks of spam link sites, backlink payment networks, etc. Granted, most of those are pump and dump. They build them up, make their money, get penalized, then burn it to the ground and move on to another domain. Still, it can be disheartening. And it may tempt some people to take the blackhat approach. But, one look at the penalties should be enough to change their minds.
Companies have survived much worse blows. I worked for a music company that was not only being sued by every major record company, they were also taken out of all app stores. They persisted, and today I still listen to music from their site, even almost a decade later.
Rapgenius got punished for playing the game better than anyone else.
Who's to say this couldn't happen to highly relevant sources of information? Wikipedia? Underground sites?
This is the equivalent of censorship.
The "game" RapGenius played was the act of being evil.
Let's get some perspective here. They were offering to promote people's pages in return for them linking back to them. Far from being evil, this isn't even illegal.
They broke the TOS and Google rightfully enforced it.
This was stupid, it wasn't evil.
Also, they could have submitted their site to Google.
Second, RapGenius cheated the system. They knew they were cheating the system. They violated the public guidelines that tell you not to cheat the system. And now they are paying the price.
Why is that so difficult to grasp?
Their emails were wanton risk, the wording had no stealth; they got what they deserved.
Exactly so. It seems they were doing it in the open because they didn't understand that anyone would object.
How is penalizing them for engaging in shady SEO tactics considered censorship?
When I google "rapgenius eminem" they don't appear in the results. That's pretty close to censorship.
I'm not necessarily against it, but I don't disagree with the usage of the term.
You can also go to rapgenius.com and search for eminem yourself. Or navigate a few pages in the search results until you find the link.
Think of it this way, if I want to be listed in a phonebook but I'm an obviously shady actor and the phonebook company refuses, is that censorship? You play by the rules if you want to be ranked.
I get that you play the rules. They're being censored for not playing by the rules.
Blackhat SEO out in the open is not a good game.
I'm sorry, what? Didn't they lose the game?
It isn't the equivalent of censorship, it's rewarding the good behavior of the competition.