Google Destroys Rap Genius’ Search Result Rankings As Punishment For SEO Spam | Hacker News Reader

Google Destroys Rap Genius’ Search Result Rankings As Punishment For SEO Spam

techcrunch.com

111 pointsacremades98 comments

We are working with Google right now to resolve this. They’ve been really great, helping us identify changes we need to make.

As if they don't know what they did. These guys are toxic. I'd ban them for a year.

[...] to create what’s often the best search result.

Poor them, they created the best lyrics site in the world, they deserve to be the first search result! /sarcasm

Give them the credit for trying to resolve a problem. Are you telling me if you run a business you would never this kind of mistake?

It is a good idea to explain why they did it and what they were thinking, instead of just say "damn I screwed up." I rather hear people explain to me why they did it and they now think they screw up. It is part of resolution.

If you don't care about explanation (or excuse if you think that's the right erm), well, fine.

What company leadership allowed to happen shows a real disrespect for one of the primary risks to the business. Without having the explanation that you described, the two most likely explanations in my mind are either this was standard practice all along (finally got caught), or somebody made one of those epic fail business decisions. If its #1, I doubt we'll see a clear explanation. If its #2, somebody may have to fall on the sword as the "poster child" of bad business decisions.

> Are you telling me if you run a business you would never this kind of mistake?

It's not that hard to, you know, not spam.

Wasn't the first thing they said was "damn we screwed up" AND "everyone else does it?"

Seems toxic to lay such claims without getting to know someone personally. Have they done you wrong?

Perhaps you are uninterested in rap music, perhaps you believe in staunch professionalism, but at least have respect for an individual's choice to live to their life as they please. Success isn't truly success unless you do it while being yourself.

There is a reason you don't have the power to ban them for a year. Rap Genius was built with love - a reflection of the founders - and is a vastly superior product. If you had ever used it, you would clearly see that. You are blinded by hate my friend.

I'm sorry, did they not, with "malice aforethought", blatantly disregard Google's terms. And then brag about it? And, after being called on it, issue a not-really apology that basically said "fuck you, we do what we want"?

Are Google's terms a moral imperative? Does observation of Google's terms define who you are as a person? Are rules created by a corporation means to determine whether doing something is right or wrong?

Google can do what they want, they own the rankings. But why would you personally attack someone? Is adherence to Google's terms a factor you use in judging the quality of someone's character?

Considering all the parent did was call them "toxic", something I personally agree with based on everything I've read and heard about them, and that wasn't even their main point, your questions are kinda moot.

No, Google's terms aren't a moral imperative, but Rap Genius' team bragging about breaking the terms and complaining when they didn't get their way doesn't make his statement of "Those guys are toxic" wrong.

You just called a group of human beings poisonous based on stuff you've "read and heard". Think about that. Poison kills people. You've never interacted with them personally, yet you are already calling them toxic.

So prior to actually analyzing the situation, you are already saying you don't care what actually happened, you don't like them anyway. Context is important, and I believe if you come into a conversation with a closed mind, it's not a conversation. That's why I don't believe those questions are moot.

Do you believe Edward Snowden is toxic for breaking rules and then complaining that things didn't go his way? I feel as though I could make the same argument about him using the logic you've just presented.

http://en.wikipedia.org/wiki/Hyperbole

So, stop whining.

>Poison kills people

It isn't a direct equivalence, dude. "Toxic" in everyday speech between real human beings means more like dislikeable, having a negative effect on the social environment, acts in socially undesirable ways, etc. Not 'kills you'.

You're accusing me of exactly what you're doing, speaking and making assumptions without analyzing. I have thought about the situation, and I had no predisposition against Rap Genius. Their actions, followed up by what I've read after this occurred, is what leads me to my conclusion.

And no, I don't like Snowden, not that it's relevant. But it has nothing to do with his leaking information, it has to do with how he came forward about being the leak.

Do you work for Rap Genius?

I'm sorry, I had thought you meant based on pieces and videos before you had even heard of this. That was how I read the last response and I suppose is why I am now defending them personally now, as opposed to their actions (which I still believe aren't a right or wrong issue - although it definitely will hurt them).

I don't work there but I use the site a lot, and when I used to live in New York they were one of the few teams of founders I met who were not complete bullshitters.

I started building an app and they had no reason to do anything for me, or provide me with any help whatsoever, but gave me detailed feedback on the product, gave me actual useful advice, and as my efforts became more and more successful have introduced me or championed me to numerous hard to get to people. Most people in their positions do not do this (at least in New York).

The reason why I responded to you in the first place was I saw you were still in university, and I bet when I was (I'm 25 so only a couple years ago) I might have thought the same thing of them. I might have burned a bridge for no reason, when actually these guys are extremely good people.

Although they put on a bit of persona when they deal with the tech press (mind you a lot of people I know wish they could do that), in real life they are sharp as hell, and are full of love. You don't meet a lot of people who give up jobs at Google, hedge funds and law firms to work full time on a lyrics website - but those are the types of people who put themselves on the line to try and work on stuff they are truly passionate about. Yeah sometimes outrageous, sometimes don't play by conventional rules, but those are the type of people that you want to work with if you want to try and build something successful.

PS. I don't believe you are toxic

Look at it this way: Whenever you use someone's services, following their wishes, which were expressed as ToS, is a moral imperative. You honesty to that agreement you made with them, is a moral imperative. And i can judge you based on that. Yes, that does tell what kind of a person you are.

> "These guys are toxic. I'd ban them for a year."

Wow, the hate is strong with this one.

Now RapGenius.com doesn’t appear on the first page of results for a search of “Rap Genius”

Readjusting their rankings is one thing, but that is just google breaking their own service. If people have typed "rap genius" into google, there is a reasonable chance that they might want to get to the thing named "rap genius", which is on page 5 in the search I just did.

And the thing is, that nearly all the rest of the results are not for rap geniuses, which presumably would be another reason for searching the term, but are just blogs and news sites reporting about the company Rap Genius.

edit -

Given that the public are fairly ambivalent about google and so are not likely to take google's side too much in stuff like this, and if you can get your rating fixed afterward by being nice, couldn't a public spat with google like this actually be good publicity, compared to google not making that much of a fuss and merely adjusting their rankings to adjust for the seo tactics used, rather than acting punitively.

If I was looking at this as a marketing droid, all I would see is that google have replaced the position my main site used to take, with loads of articles about my company that then link to my site, which makes it look as though loads of folk are interested in the company while Google are publicly burying the link, which sparks people's curiousity.

Is not that I think gaming the ranking is particularly good business practice, I just don't think this is particularly good tactics on google's part.

No, this is google enforcing a harsh penalty for one site to discourage other people from trying to game search which basically makes us all suffer through content farm links, etc.

I don't agree. I see the need for punishment of this kind of behavior, but removing the site from a Google search that seems to specifically be looking for that site is a bridge too far in my book.

It's pretty much the standard harsh google slapdown. Has happened to many other sites. I imagine they'll get out of jail sooner rather than later. It's meant as a punishment I think, not just a correction of the terms that they ranked for unethically.

Google isn't the only way to access the internet. All this complaining of not being able to type "rap genius" in google to go to the site sounds real juvenile, doesn't even rise to first world problem status. Try typing rapgenius.com and using their search mechanism. These actions may stop some of their growth, but should not be a hinderance to anyone complaining here.

It wasn't brand search terms that they where monetizing - they where obviously targeting general rap related terms by doing some very dark hat SEO

Google goes after anything it considers abuse with a bit of punitive damage, which seems fair to me. This is not a simple correction but punishment of past abuse in order to deter future abuse. The correction will come after the links are removed and their sentence is served.

They got greedy is all that happened. They were essentially at the top - but wanted to take the whole pie for themselves, instead of perhaps naturally getting there.

Still, the real story here is that almost all small businesses depend on the mercy of one giant corporation. This Google monstrosity / police man should be split up into smaller, competing search engines.

You realize the impossibility of this and this is a joke comment right?

The suggested solution may have been a joke (although it is not without precedent) but the analysis was spot-on IMHO.

Ever heard of Bing, DuckDuckGo, Yahoo, etc.? Google just does it better than anyone else at the moment.

Now I'm not saying Google doesn't have a de facto monopoly, but I'm wary of anyone who says a private company should be forced to split. Should they be punished for being the most popular?

Good thing google is public rather than private.

Do I need to explain how a publicly traded company is still private, or are you just being purposely obtuse?

Look at Googles multi class share structure and ask that Q again.

JHSheridan is using the term "private" to mean "not owned by the government."

Exactly. I understand there's a difference, I just thought the way I was using it was completely obvious.

I understand that Google is its own legal fiction. I don't see any inherent human rights issues in turning one piece of paper into two.

I think there could be some questions as to whether Google is engaging in anti-competitive behaviour in the way they use search revenues to undercut other competitors. Perhaps they are doing nothing wrong, perhaps they are doing something wrong, and perhaps an effective remedy to that is to split the company.

Frankly I think it's an inherently monopolistic industry.

In the current state of affairs (i.e. web browser interfaces and mobile phone integration), it's quite inconvenient for people to switch up their search engine regularly, or even use multiple ones regularly.

With say, supermarkets, it's pretty easy to shop at Acme one day, Trader Joes another, and Superfresh the third. Search engines aren't like this, or at least, we haven't figured out a way to use them in a way that is conducive to fast-paced, low-barrier-to-entry (or at least not astronomically expensive barrier to entry) competition. Maybe we should.

bang commands on DDG come pretty close. I use DDG as my primary search, but in reality I use about 20 different commands for specific searches on any given day, including quite a bit of google search and wolfram alpha. Not saying these commands will ever see wide use, but I find it a convenient platform for lots of different search engines.

That's a really good point. Every few months I try to adapt to using DDG as my primary search, but I always find myself throwing a !g into my searches, which kind of defeats the purpose. I'm curious, for what kind of searches do you find DDG's own search results to be good enough (or better) than Google's? What kind of searches make you bang to Google?

I generally use !g if I want local results (say, looking for a restaurant) or I use !m if I want a map (which is really the same thing as local for me). I also use Google if I know my particular search is going to be obscure and/or noisy. For instance, weird error messages get a !g. The other major exception is that Google Scholar is incomparable for finding academic papers on a particular topic. However, for everything else (general knowledge, non-weird errors, most programming-related topics) I find the DDG results entirely sufficient. In fact, I intentionally use DDG by itself if I expect to end up at Wikipedia since DDG adds possible Wikipedia matches at the top of the search results.

That being said, I end up using the DDG results for >50% of my searches. If you do more local searches, or if the things you search for tend to be noisy or obscure then, of course, YMMV.

About the same as me! I generally do DDG first unless I know I want a map, or I'm searching for error messages. Other commands I use fairly frequently are

!yt (youtube) !gmail !news !gg (google groups) !hn !boingboing !torrent !itunes !rubydoc !rubygems !codex !so (stack) !am (amazon) !jstor (and a few other specific academic journal searches, but you can also use !googlescholar) !flickr

and probably a few others! I encourage everyone to check out the list and get it into their workflow. With Alfred + DDG I can get to most things much quicker than I could through google

https://duckduckgo.com/bang.html

I don't see why its not just as easy to switch to other search engines. If Google is beating these other search engines purely because they are a better product then there is nothing monopolistic here.

The reality is that the barrier is very low. Blekko, DDG, bing, yahoo, and we can go back even farther and list more search engines. Nothing is stopping them other than their own engineering resources. Your entire argument is saying Google is better and out innovating the competitors so we should punish them.

But in this case, we would have been worse off. It's hard to imagine that a fragmented ecosystem of smaller search engines would all reacted this quickly to a case of someone engaging in blackhat SEO, so there'd be a lot more temptation for sites to try to get away with this sort of thing.

Online startups comprise a small portion of all small businesses. Most small businesses do not rely on Google.

Why do they depend on Google's mercy? What does Google provide them, that they must have and cannot be had elsewhere?

To phrase differently and hint at what I am suggesting- if Google provides something that companies cannot live without, what did companies do before there was Google?

Google provide them users.

> Why do they depend on Google's mercy?

http://readwrite.com/2010/02/11/how_google_failed_internet_m...

Google has been around since the beginning.

>what did companies do before there was Google?

They didn't.

They advertised.

Smaller companies seem to be replacing the "advertising dept." with "our rank on Google dept.", in which case it's not exactly Google's fault they are dependent on Google, any more than when a Craigslist scraper is dependent on Craigslist.

Before the search that we know of today, there were directories (see http://dmoz.org for example) where you submitted your site. Originally it was a list, then a hierarchal list, then there so many entries they let you search their database, then they crawl the web to find other sites, then they search the _content_ of other sites, so you can see how it evolved.

Web rings and pages of links to other sites were also common, basically what RG did, but more reciprocal.

Remember, Google is quite large, and hence also inhomogenous. Some time ago the Chrome browser team did something that could be construed as a service-for-links campaign, an the PageRank team penalized Chrome for two weeks (if I recall correctly - commentary on this [0]).

[0] https://news.ycombinator.com/item?id=3422386

"small businesses depend on the mercy of one giant corporation" - this can't really be blamed on google. It's hardly their fault they are the best at search. If someone comes along with something better people will use that. on the interwebs there's no such thing as brand loyalty. Going forward the internet's biggest problem is security. With hackers pulling millions of credit cards from Target and seldom a day goes by without reading about some big corp getting hacked, people will more and more look to big corporations like google that can hire the best security minds for their online protection. Using google gives you some degree of trust that the sites in the results are there for a reason, people they aren't scams or bogus companies. Google is the internet's filter, and in this case with RG I am not sure that i don't side with google. There are plenty of alternatives to RG. I don't see why anyone would care much about song lyrics, but it's not like RG are the only game in town.

How would you split it up? By category? Google A can only search for words that begin with A, or are about Apples and arch-rivals (but you need to go to Google R to search for rivals) Its not like no other search engines exist.

This is why your business shouldn't rely on it. I initially rely on it to get users, but then get info like email addresses, so if they ever decide to make changes in their algo and I drop rankings, I am still in business.

We all depend on one search engine and will quickly abandon it when it doesn't provide the results we want. Remember Altavista? Yahoo? All those other has-beens?

Also, you can pay for advertising rather than expecting to be entitled to organic results. Nobody forces anyone to build a Google-supported business and frankly, people really shouldn't be doing that for this very reason. It's not a good idea to live at Google's mercy.

Main discussion is over here: https://news.ycombinator.com/item?id=6963365

Adding to the discussion here will probably just duplicate what's being said over there.

NOTE: There is an update not in the original report, apparently RG is working with Google to resolve the issue:

We are working with Google right now to resolve this. They’ve been really great, helping us identify changes we need to make, even on Christmas. We’re working on it as fast as we can, and expect to be back on Google very soon.

It sucks to be off Google for us and for the thousands of our community members who have worked so hard to create what’s often the best search result.

We hope everyone who reads this will take a little time out from their Christmas and head to Rap Genius and sign up so you can contribute your knowledge on your favorite subjects – becoming a member of our community makes the site way more fun. Merry Christmas

By working with Google they mean they're going to work with them to identify the links they need to clean up, try and get those links "cleaned up" themselves, submit a disavow file and then get them to reconsider the site as outlined here: https://news.ycombinator.com/item?id=6963785

It happens all the time and no disrespect to RapGenius, to companies much bigger than them.

And for far less reason.

my first instinct is to hate these guys, probably because I feel they are just doing an act. But going beyond my initial kneejerk reaction, I do have to admit they are doing something different (as far as their "act" goes). Game changers almost never go with the flow, so whether you hate them or love them, at least they're not following the crowds.

I occasionally search for lyrics. If I know the name of the artist and song, I search for those with the word lyrics added. If I have a line from the song I'll type that and add lyrics.

I don't think I've ever noticed a Rap Genius result. To check this, I just searched for a few lyrics on Bing (where presumably Rap Genius is not penalized), and did not get any Rap Genius results. I even tried some Justin Bieber lyrics searches because I believe the links that got Rap Genius in trouble had to do with Bieber songs. Nothing on Bing.

Could someone give some examples of searches that used to include Rap Genius high in the results?

I laughed out loud when I realized this (I was searching for Tyler, the Creator lyrics) - instead of filing a page with their links, now they are nowhere to be found. Checked my watch and it was still December'13 so something is wrong.

The annotations have really degraded in quality - there used to be some very good ones that gave you context and explained complex puns and references and now you have 9gag style memes.

The RapGenius story and characters become more and more fit for a movie making fun of the whole internet industry.

Article content is interesting. But -1 for disabling pinchzoom, as it severely impacts usability on mobile devices.

It's probably a temporary ban that will last up to 90 days once appealed.

What grounds would they have for appealing? They haven't denied the original email posted was authentic. It seems to be black and white that they were engaging in blackhat SEO.

30 days.

Here generally is not a timeframe involved in this. Is google penalized them then they received a manual penalty for unnatural links. They will have to identify all the links that are unnatural, contact all those site owners and try to take get the links taken down. Or disavow them and show a huge effort was made... And tell google that they won't do it again through a reconsideration request. Unless google makes an exception, it is taking at least five days or more for google to respond to RRs. And even then, once a penalty is lifted, the site won't necessarily come back quickly.

RapGenius is not that good. It's heavy. AZ Lyrics has many ads, but it's pure text. Good for lyrics. I don't really want comments in the lyrics I search for.

I just watch the video in the article and I have to say are the founders really like this in real life or is it all an act?

Is there any hope for less softonic results now?

Rules are rules, but do they deserve a pardon?

No.

Yeah I guess you're right. Rap Genius' well-known brand aside, if they treat them differently to how they would penalize an unknown tiny site then that's a can of worms they won't be able to rid themselves of.

So, the best lyric site by a mile no longer shows up in ordinary Google lyric searches. As a music fanatic, searching lyrics is something I do all day long every day, by the way - and when I search lyrics in Google, I really want to go to rap genius. Anyway, supposedly this degradation of Google had to happen to make a public example, and most of us are ok with it because the founders of RG are known to be kind of jerky and unprofessional, and we loves professionalism cause the more we love it, the more it makes us look professional.

Just checking I'm still on the right planet. Yup, I definitely am.

> and when I search lyrics in Google, I really want to go to rap genius.

So, isn't it easier (and faster) to go directly to rapgenius or create custom search keyword in the browser, than ask google for the lyrics?

when I search lyrics in Google, I really want to go to rap genius

Sounds like a wasted step. Go to your site and search it. Or has it outsourced that (/expectation) to Google?

Even then, search {site: XYZ} in the g box and be done with it.

Better yet: add RapGenius as a %s-bookmark (or Chrome calls them search engines): "http://rapgenius.com/search?q=%s"

My favorite use for this is a Google IFL search restricted to site:wikipedia.org, so now "w something" in my URL bar takes me right to the Wikipedia page for Something.

So yeah, outsourcing to Google... but fast and handy and almost always what I'm looking for.

Their search engine isn't very good, as is the case with most sites. There's a reason why everyone uses Google to find things.

Actually, I've had a chance to use it quite a few times, and I was surprisingly happy with the results. Still I might have not tested all possible scenarios (like searching for a song knowing only a part of the lyrics with some errors), so your point obviously stands, thank for pointing this out.

It's not good at matching partial words, and sometimes a song's title only contains very common words, etc. It doesn't deal well with those

Hmm, so maybe they should, like, build a better internal one for themselves, instead of trying to scam the external one they rely on so thoroughly.

So, I guess the best site in any category should be above the rules then?

And come on - the idea that people only have a problem with the founders because they want to appear professional themselves is ridiculous. "Kind of jerky and unprofessional" is a staggering understatement.

If you want your results from rapgenius, that's why site:rapgenius.com exists.

Mmmm, we all love typing that.

Just add it as a custom search engine in the browser of your choice. In chrome you could just type:

rg justin bieber

You should try Bing for lyrics search. I've already made it the default engine on my phone.

Or, if you value privacy, you can use Duckduckgo.com which still ranks RapGenius plenty high.

Or even "!rg foo bar". (I typed that without looking it up, but I am sure it would work. That's how I use DDG all the time)

The hate seems to be coming because they're "unprofessional", "disrespectful", "juvenile" and "arrogant" - like people in the hip hop industry haven't heard that before.

Don't listen to the haters. Stay on the grind.

People build sites that are very good all the time, often 'the best site for X by a mile', but they don't make it to the top because of poor SEO or marketing. Just because they are a good product doesn't mean they get to the top. RapGenius should be no different.

And what about the majority of people who simply want their lyrics, instead of all those messy annotations? The other sites give me that and that's why they've been at the top for the past decade.

So why not just use Google to search rapgenious? I mean if you KNOW you want to go to rapgenius, and rapgenius's search isn't good. Just use google and tell google to search only rapgenius.

are they only the best lyric site until they have to monetize ?

Who needs google? Why don't you just go directly to rap genius and bypass search altogether?

> supposedly this degradation of Google had to happen to make a public example

How exactly did you come to the conclusion that this is some kind of public example?

Punishing websites that that are caught engaging in black hat SEO seems to be pretty consistent with what Google has done in the past.

And it's not like this is bad for search users because I doubt that we would prefer having link farms with useless content as the top result for every search query.

> because the founders of RG are known to be kind of jerky and unprofessional, and we loves professionalism cause the more we love it, the more it makes us look professional.

Who cares if the founders of RG are jerks or not? I'm incapable of understanding what you are trying to say here.

Best maybe for you. It doesn't have any of the stuff I'm listening to, so it's pretty much worthless to me.

Google is actually penalizing its users' as they'll have less useful results instead of the best possible results.

Google is not supposed to be the Internet police. It's supposed to analyze the best sites, and rank them. Rap Genius getting some spammy links doesn't make it a better site; but it doesn't make it worse either.

The next thing we are going to see is a Google Justice Department?

I'd like to hear from someone who downvoted you with an explanation about why. I genuinely don't understand - you've made a logical post but clearly 1-5 people have disagreed and you're now on the bottom of the page in light, light gray (downvoted). I'm struggling to grasp the "Why?"

Taking action which poses a stron disincentive to the kind of black hat SEO campaign RG was runnin benefits Google's users—tolerating such an action (especially as widely piblicized as RG's campaign had become) would have hurt Google's users far more than penalizing RG ever could.

Reply on news.ycombinator.com