Rapgenius.com Traffic and Demographic Statistics | Hacker News Reader

Rapgenius.com Traffic and Demographic Statistics

quantcast.com

120 pointsthemichael127 comments

Can someone explain why this is a topic worthy of discussion? I get that they had a SEO tactic, Google penalized them, they offered an empty apology for it, and now we're up to the present.

Stating the obvious of "traffic drops when Google penalizes you" seems like a "sun rose again today"-type story.

Why? Because the sheer magnitude is crazy. Rap Genius's traffic fell by nearly two thirds! This is a very prominent, YC-affiliated startup that is being nuked by Google. It should come as no surprise that the news is seen as significant by the HN crowd.

Moreover, I think many people are upset with the scope of Google's punishment. De-weighting RG is one thing, nuking it into the ground is entirely different, and is in my opinion unwarranted. Even searching directly for the name of the site does not yield any Rap Genius results.

If I type "Rap Genius" into Google, I am justified in expecting Rap Genius to be the first result. Period. When Google goes out of its way to effectively expunge a site from existence (to the extent that Google can expunge a site), they are doing a massive disservice to their users and are degrading the quality of their search results.

There is no reason that a response of this magnitude by Google is justified, and by issuing such a draconian response, Google has lessened the usability of their product and done a disservice to their users.

Penalties are blunt instruments.

The whole point of this penalty is that Google cannot trust the signals that are in its index when it comes to Rap Genius's site. And it sure isn't Google's fault that they can't.

Can Google subtract out all the "naughty stuff" and decide if Rap Genius deserves to even be on the front page for "rap genius" anymore? I doubt it.

If Google's search had a category for "obvious searches" then yes, maybe they could apply a penalty as delicately as possible. But delicateness is not really the point. Dropping them down 50 spots is blunt and simple. I think everything about this approach makes sense.

From I what I heard the penalty is only for a month long? Still, what other practices has Rap Genius been using all this time, without being caught, in order to gain so much traffic?

>De-weighting RG is one thing, nuking it into the ground is entirely different, and is in my opinion unwarranted. If I type "Rap Genius" into Google, I am justified in expecting Rap Genius to be the first result. Period. When Google goes out of its way to effectively expunge a site from existence (to the extent that Google can expunge a site), they are doing a massive disservice to their users and are degrading the quality of their search results.

Necessary reminder that Google does whatever the fuck they want with their service, and that no one is in a position to expect anything. Does it suck? Yes. Is it a predictable outcome of the free market on which Silicon Valley and the tech industry in General is built? You bet.

> Moreover, I think many people are upset with the scope of Google's punishment. De-weighting RG is one thing, nuking it into the ground is entirely different, and is in my opinion unwarranted.

If the impact wasn't significant, it wouldn't be a disincentive to engage in black hat SEO.

> There is no reason that a response of this magnitude by Google is justified

Yes, there are reasons. The reasons are "specific deterrence" and "general deterrence".

Cutts has said before their goal is to "break the black hatters spirit" (http://searchengineland.com/googles-matt-cutts-on-search-spa...). I assume that's what they're trying to do here.

The response was designed to send a message. And it seems to have accomplished that.

In one of the other threads, some stated that Google do not have monopoly status. The claim they used was that google only has 25% of the search market.

This topic is interesting because it shows how much in a monopoly status google really is in in controlling where traffic goes on the net.

Interesting point!

Google's search team has been famously resistant to any influence by Google's sales/marketing teams on ranking decisions. Search quality gave rise to the 'golden goose', after all, and the self-conception of the team, and its individual members, is based on independence from purely money-grubbing considerations.

But I wonder, what happens when Google's antitrust lawyers came to the ranking team, and say: "soften this penalty [or otherwise tweak these quality-rankings], because otherwise we'll be in more trouble with regulators or leave too much evidence of market-power."

Could the search team still say, "buzz off, we're sticking to what we know from the numbers is what's best for our users"? Or does keeping regulators happy, and avoiding smoking-gun fact-patterns, trump everything? I suspect it often will, because of the unique risks of state enforcement and appearing unlawful. It's also easier to rationalize "we're sacrificing our own idea of what's best because the law seems to require it" (even if the actual risk is fuzzy before losing in a formal legal process) than "we're doing this to make the guys over in ads/mobile/etc a bit more money this quarter".

you are not using that word correctly. If Google was the only company doing search and prevented others from entering the market you would be correct. Being the best through a superior product and innovating is not a monopoly.

Doesn't matter how you achieve a monopoly: it's defined by dominance, not bad-behavior. And once you have monopoly power, things that used to be legal for you to do – for example certain pricing or bundling strategies – can become illegal.

> Doesn't matter how you achieve a monopoly: it's defined by dominance, not bad-behavior.

Well when you describe it like that, makes me think antitrust laws are just silly. Isn't the job of a company to be the best and beat their competition?

the entire argument is questionable as to what Google's strategy is but they are only in that position because of a superior product in a market with a relatively low barrier.

Low barrier?! Giant companies like Yahoo, staffed with large teams of search technology pioneers, have been scared out of search competition. Google touts its massive custom infrastructure. The arms race between Google and SEO creates a massive pollution backwash of false quality signals on the web, that make things way harder for any upstart which lacks Google's decade-plus of proprietary know-how and billions in hardware.

And that's before considering the contractual defaults across software (Firefox/toolbars/bundleware) and subsidized hardware (Android), that few people change. (Many don't even know it can be changed, or don't even know the difference between "Google" and "the Internet"!)

Those are all big barriers to entry, even if they've been earned by excellent technology and business strategy.

while DDG started by 1 guy and entrants like blekko or qwiki can differentiate in other ways and enter and compete easily.

Again everything you are saying is because of superior product/employees and innovation. Your entire argument is based off no one else can come up with a better product so lets punish them

Where'd I say "let's punish them"? I'm just accurately labeling the situation.

Google's dominant position constitutes a 'monopoly' and 'monopoly market power', in a common understanding of those words, and in the senses often used by economists and regulators. Their search market has very high 'barriers to entry', in terms of costs-to-compete, ability-to-reach-customers, returns-to-scale, and network-effects. (Separate from all the specific things I mentioned which make it extremely hard for entrants to get a foothold, you can also just look at the profits. Whenever there are big profits, there are business 'moats', or else someone else would grab a bunch of those profits themselves.)

This makes them a likely target of government antitrust action – an observation which is true whether or not such antitrust laws, and related enforcement actions, are a good idea or not. Again, I'm interested in accurately describing what exists, not making moral or policy judgements.

Qwiki? Really? They never offered ranked search, abandoned their 'visual explainer' product long ago, and are now part of Yahoo with their multimedia storytelling app.

DDG, Blekko, and all search sites other than Google/Bing/Yahoo/Aol/Ask are together less than 0.3%-1% of the US market. And yet this market is wildly profitable for Google – and almost no one else. (Bing, the distant 2nd-place, has lost hundreds of millions every quarter since 2007.) That's not an indication of vibrant easy-entry competition: one company dominates market-share and profits, and the number of competitors and new entrants have been declining over time.

By the way, the founder of your 'compete easily' example Blekko also disagrees with your assessment, and has since the outset of Blekko. In this 2007 post he describes how it's impossible to compete for market share head-on with Google due to its "immense and amazing power" in a "winner-take-all market":

http://www.skrenta.com/2007/01/winnertakeall_google_and_the_...

>DDG, Blekko, and all search sites other than Google/Bing/Yahoo/Aol/Ask are together less than 0.3%-1% of the US market. And yet this market is wildly profitable for Google – and almost no one else.

I think this sums it up perfectly. Theres plenty of competition just no one can compete because google offers a superior product. As soon as google offers an inferior product or a competitor offers a better product people will abandon ship. They do nothing anti-competitive nor blocking entrants into the market.

From the perspective of web users, there is choice(even if heavily skewed by default search in browsers). But from the perspective of web sites/web companies/web devs/web advertisers, there is very less choice and bordering a monopoly, as this example illustrates. Even worse if you're someone like MapQuest, a shopping site, or Yahoo Finance, since most people tend to search locations on search engines/browser and are quickly steered to their equivalent competitors from the search engine company. Anyway, usually antitrust regulations are about market power and influence, for example see DoJ vs. Apple over eBooks, even though iOS never even had majority share, forget about a monopoly.

The U.S. accused Apple Inc. and five of the nation's largest publishers Wednesday of conspiring to raise e-book prices. I don't even see how thats relevant to how Google does business within their own ad products. Who exactly is Google colluding with to suppress or hurt competition ?

Companies will do business where the users are. The minute there is a better product or users abandon google, those same people you listed will leave as well. So maybe we should go after Facebook for destroying MySpace as well ? The entire argument is based on the fact that Google offers a superior product and should punish them for this.

> This topic is interesting because it shows how much in a monopoly status google really is in in controlling where traffic goes on the net.

Does it? I mean, is there any reason to believe that Rap Genius is typical in their degree of dependence on Google here?

I'd replace the word monopoly with popular.

"A monopoly (from Greek monos μόνος (alone or single) + polein πωλεῖν (to sell)) exists when a specific person or enterprise is the only supplier of a particular commodity. Monopolies are thus characterized by a lack of economic competition to produce the good or service and a lack of viable substitute goods. The verb "monopolize" refers to the process by which a company gains the ability to raise prices or exclude competitors." - Wikipedia

You can monopolize an market by popularity, and thus create a monopoly. However, monopoly and popularity is not the same word.

Google is not the only supplier to internet searches and is not excluding competitors. They are merely very popular and are thus not a monopoly.

You are interpreting "monopoly" literally. In the context of antitrust law, you only need to have a big enough market share to appear on regulators' radars. The actual market share threshold varies from country to country, but typically it's around 60%. Google certainly meets that criteria in many countries, not only for search but for online video as well.

As for "excluding competitors", Google is having a tough time with antitrust regulators in the EU for exactly that. For a more obvious example, see their shenanigans with YouTube and Windows Phone.

I totally don't buy that Google has a monopoly on search. Anyone here could start a search engine tomorrow. Sure, it might suck, but there is no barrier to entry. It isn't like Ma Bell telephone companies who had blocked other competitors from running copper wires to homes via tons of local ordinances... which reminds me, its pretty hard to have a true monopoly without some help from the government.

I think folks are interested to see how much the penalty hurts.

I think people - foreigners and valley outsiders for the most part - are often genuinely clueless about the workings of the valley.

They are quite apprehensive of the degree of even-handedness that is at play in the valley. They suspect an air of nuanced in-group favoritism, within startups here, that help propel some and sink others.[1]

Hopefully these kind of actions (on behalf of Google) help dispel such notions.

[1] http://www.slate.com/blogs/future_tense/2013/09/13/silicon_v...

I personally know of a number of legitimate sites that were invariably affected due to content and blog farms harvesting their content when Google's panda algorithm rolled out.

Those penalties hurt a lot of good people!

Anyways, this one is a bit more complex than how much the penalty hurts, rather it's because, a well known and well funded start-up adopted shoddy traffic building practices that were unacceptable years ago, got caught and then had swift and targeted judgement bestowed upon them.

However much people understand that sometimes business needs to be conducted in a grey area, there is a certain amount of satisfaction when the system catches up and behaves fairly.

What's a legitimate site? Plenty of sites most people would call "legitimate" like high profile blogs or ecommerce sites engage in activity that's frowned upon by Google. The fact that some unwanted behaviour was no longer rewarded isn't exactly a penalty.

What RapGenius (also a legitimate site in the sense that they're not scammers and providing an useful service) did wasn't just unwanted behaviour but specifically forbidden. And yes, they're far from alone in what they did. Which is exactly why Google is making an example out of them.

I know, it sucks, all your competitors doing similar things forces you to choose between taking this risk and never making it in the first place. But that's exactly why it's a good thing Google is making a stand and setting an example.

It's a move towards a level playing field where nobody is making the internet a worse place by engaging in tricks like link farming rather than a level playing field where everyone is doing it and the end user suffers the consequences.

Same here. I am in no way affiliated with Fluther other than being a user and Panda completely screwed them. I think they got screwed because Mahalo scraped their RSS feed to steal their content while providing links to the original content. So there was a thousand links going to Fluther from a shithole of a site.

Part of it might be Schadenfreude since HN does not seem to think much of RG.

I think this is exactly it.

Their affect is another example of cultural appropriation of "urban" culture by three very affluent guys (Stanford & Yale grads), which I think turns a lot of people off, including myself. As someone who genuinely grew up in that culture, it is painfully cringe worthy and just sad to see people who have no background in it adopt it. Their Disrupt interview was a genuine display of the generally awkward douchebaggery that comes from that appropriation.

So when I see folks like this do poorly because of arrogance, it is indeed schadenfreude.

Yeah, that interview is like the definition of the word 'tool'.

It's incredibly racist and classist for these privileged plutocrats to mimic the dialects of the underprivileged. The ironic and overplayed way in which they take on the pose also makes it clear that they intend it as a publicity stunt--not as genuine respect for the culture or because they fit into the culture or want to fit in. They just want money and they think this is a money making stunt.

It's one of the clearest examples of cultural appropriation and exploitation by a corporation. Really fucking disgusting.

Imagine if an American corporation wants to sell vacations to Mexico, and as a publicity stunt the CEO dresses up like a latino stereotype and adopts a Spanish accent.

This is how the Rap Genius douchebags-in-chief come off. Racist fuckheads.

Is it genuine rage or sarcasm? It seems to be genuine but I just can't believe that.

I don't find any of his points to be too far off. If they were a bit more modest and down-dressed I might think they had a genuine respect for hip-hop.

Beware trying to assess "what HN thinks" about a topic from informal, personal scans of items. Neither the top stories, nor the longest/loudest threads, are necessarily representative of 'most readers' or even 'most commenters' – just those that show up at those times/places.

Where the especially peeved choose to spend their attention (and comments and upvotes) can make lots of people or companies seem like community bete noires – even if most of the HN audience thinks the targets are fine or even admirable, but just has less energy for anti-carping.

why is that? i assumed HN would like then since i remember a bunch of articles about RG being big RoR proponents

I think that it's because they are not fans of their brogrammer antics. Read this thread for example (or any other Rap Genius thread on HN)

https://news.ycombinator.com/item?id=6904224

The original link that was deleted was this

http://aphyr.com/media/recruiters/rapgenius.txt

What did I just read? I don't understand that exchange at all. So Rap Genius sent a corny email to this guy trying to hire him. And...?

...and he said "no thanks" because he does not approve of RG's past behavior. They persisted. And then he said no again in verse form. That's pretty much the whole story.

I posted it more for the response on HN, not so much the email itself.

What you've just read was manufactured outrage for fun and profit.

For me, it's incredibly interesting because I did not find anything wrong with the tactic they used: in fact, it is far, far better than actual SPAM techniques they could have used instead. Or the bullshit tweaks to content that other sites use to improve SEO score, when those tweaks do not benefit the consumer in any way.

Think about the fact that there's a human in the loop, and that it's a plug or tit-for-tat. The ways that companies actually do SEO 'within the rules' is way, way worse than that.

Also as a consumer I have only been on their site < 10 times but I loved it each time. It is an actual benefit for them to gain ranking. They have a real service, and a good one.

It also seems quite genuine, from the tone of the email. Totally normal.

You might think I have some association with rapgenius, but I don't: my only association is that the handful of times I got to their site, I was absolutely thankful and totally surprised at the quality of the result.

By contrast, there is a good chance there are textual quotations I've searched for where I missed good commentary there that I would have appreciatied.

All of these aspects, especially above-all the fact that consumers are happy when they do get to the rapgenius site, that combine to make this story very interesting to me.

I see rapgenius as the good guys, and very honest, in their attempt to gain additional exposure on real content.

Not sure, but I'm sure it doesn't hurt that RG is a YC company.

Google just proved all that when someone breaks their laws they punish him, even if it's a YC company.

Who knows maybe because they are a well funded YC company Google decided to make an example out of them and won't forgive them so quickly.

Google even penalized its own product http://searchengineland.com/google-chrome-page-will-have-pag...

Fundamentally it is the magnitude. Web sites often manage themselves to a metric called "RPM" or revenue per thousand. It can be thousand 'views' or for people who are interested in accuracy per thousand 'visits'. Unique IPs often translate as a good metric for 'visits'.

So if your traffic is 1.5M uniques and drops to .5 uniques, then your revenue may also have just dropped 66%. If you were staffed to be cash flow positive at your previous traffic rate, you now find yourself burning cash at an alarming rate. Your response, dictated by your reserves, is either lay off a bunch of people to curb your burn rate, or delay your accounts payable, or both. If this happens at an 'inconvienient' time, it can sour acquisitions in process, or deals with initial outlays, to be blunt it can kill you.

So then here is the thing, Google has no good way of knowing how you got to your organic ranking. There is no 'what is the rank of this place without these links' sorts of computations. So there isn't a good way to judge. But judge they must, lest others abuse their algorithm, and so they do so.

Now lets speculate that they kill off Rap Genius. Overall what does that do? Who does it help? Is it justice?

Lots of thorny ethical questions, not many good answers. And no way to make restitution. A sad place to be.

eh i think this falls into the category of "if you don't care don't click the link". some people are interested in stats...

The funny thing about this whole controversy is that they're getting more organic links to rapgenius.

When the penalty is taken care of - they may be sitting in a better position than they were previously.

If you're right, and were this actually intentional too, well, that would be genius.

But given that their traffic is maybe off two-thirds now and that it is unknown how long google will leave the lights on (weeks, maybe months), during which time this will be long-flushed out of the news cycle and it becomes harder and harder to recover, and harder to secure things like more capital when you look this stupid, that's a high-risk bet. I'd say sabotage is more likely.

I don't think organic links matter much at this point when you can't even Google their name and find them on the front page.

That's why I included "When the penalty is taken care of" in my statement.

You call it a "SEO tactic" here but I'm sure you'd not bat an eyelid if it were referred to as "growth hacking" elsewhere.

The problem is how much power Google holds over most of the web. You game the results and they turn their attention on you? Congrats, you're wiped out.

Essentially, there's a very blurry line between 'growth hacking' and 'being annihilated by Google' and no-one's really sure where it is. That's what's interesting here.

>there's a very blurry line between 'growth hacking' and 'being annihilated by Google' and no-one's really sure where it is. That's what's interesting here.

Except in the cases where the line isn't blurry, i.e. the well-defined set of rules about which G has repeatedly said "we'll penalize you for ignoring these!" Unfortunately the tactic RG used wasn't a blurry-line grey-area tactic, it was one G clearly said don't use.

It is interesting to see just how much it hurts a site like RG.

>Can someone explain why this is a topic worthy of discussion?

I find it incredibly interesting to know just how hard they got hit. I find it interesting that maybe someone from rap genius will be in these comments providing some extra context and insight.

Can you explain why you care if other people discuss a topic you don't think is "worthy" of discussion? You know you can ignore the story and look at the entire rest of the front page, right?

>Can you explain why you care if other people discuss a topic you don't think is "worthy" of discussion?

I thought there might be a piece to this story that I was missing. I gained a lot of value from the replies to my question (esp. the conversation spawned from @snowwrestler's comment.)

>You know you can ignore the story and look at the entire rest of the front page, right?

You didn't really need this in your comment. It was a bit of snark that devalued your otherwise justified response.

> It was a bit of snark that devalued your otherwise justified response.

No it wasn't. Comments like yours are peppered throughout HN and Reddit, and the answer is always the same: the users found it interesting. It's a democratic voting system -- get over it.

How do you know it's a democratic voting system? Just because you assume it is, doesn't mean it is. Just ask around, plenty of stories about being mysteriously shunted off the front page.

RapGenius is a YC company and always seems to get attention on HN whenever there is some kind of perceived grievance against the company. Whether it's Google or Heroku, there's always a lot of screaming and shouting.

This is comparing pre-Christmas-day traffic to Christmas-day traffic. Many people spend December 25th doing very different things than they do other days of the year.

A good way to account for this is to look at year-over-year change. It looks like on Dec 25 last year, they took a 13% hit[1]. So it appears that their 60% hit today should be tempered a bit, to account for this. Even still, I think we can say that this is a very bad day for Rap Genius - especially when you consider that they have a lot of artificial publicity due to the currently unfolding scandal.

[1] https://www.dropbox.com/s/lidlugtdycdrrnx/Screenshot%202013-...

The next top result for lyrics thats 'Quantified' on that site: https://www.quantcast.com/lyricsfreak.com?qcLocale=en_US shows a slight drop.

Edit: Google Trends for 'lyrics' the past month http://www.google.com/trends/explore#q=lyrics&date=today%201...

They're on page 7 for their brand name. Christmas probably had little to do with that drop.

https://www.google.com/search?q=rap+genius&oq=rap+genius&aqs...

I'd say it probably has something to do with Google penalizing them.

A lot of people get new albums from the holidays and would want to look up lyrics.

either way, 1 data point isn't a trend. wait and see is the best course of action.

If you compare last years Quantcast traffic on christmas day to this years.. it wasn't that much of a drop compared to the few days surrounding it.

Like nobody posting to Facebook in the morning, combined with completely crashing Steam in the afternoon.

Why is Rap Genius afforded this coverage? There are a lot of SEO scammers out there and are punished as found. Reading through one of those TC articles it appears that they were interviewed on stage by TC during one of their events too. Why is a scammy link baity company given this importance?

Three reasons:

1) The site itself isn't actually half bad, especially when compared to other lyrics sites.

2) They're a YC company, and took money from other big investors. This automatically makes them high-profile on places like HN.

3) The founders often publically act like douchebags, write emails with bad spelling, and generally seem to think that being cool is the most important part of running a startup. I suspect that this is just for show, but still, many people want to see them fail because of this. This article is about them failing, so it gets upvoted by those people.

Also, nobody seems to mention the fact that this story was created out of an HN thread... I imagine that has a lot to do with everyone's interest in it.

Indeed. People are forgetting that the only reason RG came to G's attention is because Matt Cutts stumbled across the HN thread and took action.

#3 definitely is why there is so much enthusiasm running around regarding their punishment

It's actually kind of bullshit too. It's in a similar vein to the joy people have in many of the 'Facebook is no longer popular among 14-16 year old' posts. These sites provide a valuable service, just because you don't like them doesn't justify being (in the parlance of the youth) a hater.

I disagree somewhat. Yes, I think that if people are creating value then their own personal behaviors should have little to no bearing on how that work is translated.

I think their affects do set a bad standard for tech founders, and just how people behave in general and those attitudes spread into how they run their business.

I can envision other future founders looking at these guys as successful and thinking, "yea that is how I have to act to make it big." In a similar way that The Game and all that has influenced dating, I think these guys applied some of that to the startup/hip/rap thing which in my opinion is detrimental and smacks of trying too hard, especially when they are all very well placed, affluent people.

Add to that the fact that they genuinely did shady things (regardless of whatever anyone else did) and the bullshit wears off even more. So maybe someone who is coming up will think twice about behaving similarly in all aspects.

It is kind of funny how people think that only white rap-fans/rappers came from non-poverty backgrounds. Rap industry markets itself slickly.

Who said anything about race? The distinction is class and culture, not race.

> These sites provide a valuable service, just because you don't like them doesn't justify being (in the parlance of the youth) a hater.

The founders put on a public face to create reactions in audience members. Some of those positive, and that's no doubt what they are seeking. Some of those are negative, and while that may not be what they are seeking (although I suspect that, given their actual target market, the negative reactions of those outside of the targeted culture are just as intended as the positive reactions), those are no less legitimate than the positive ones.

On 3, I wonder if they think being cool is the most important part, or the reality is that they are executing on everything AND ALSO trying to be cool to differentiate themselves AMD get noticed, but the critics only see the PR front and ignore the rest.

I'll give you a hint: the other SEO scammers aren't YC companies...

Presumably because they are playing and pushing the story to get more inbounds links and thus higher SEO.

The real story is that Google is incompetent and incapable of identifying these sorts of things unless a company is called out publicly on it. No wonder their search results are increasingly a disaster zone.

Yeah, you're right, missing this makes them incompetent.

Does that word have any meaning at all anymore? It seems like everyone who isn't an amazing brilliant self-made billionaire is incompetent. Oh wait, I forgot, we were discussing the incompetence of a bunch of amazing brilliant self-made billionaires... So I guess it's everybody now.

Was about to write a similar response. In the end they will walk away with the penalty removed, and hundreds of links from bloggers talking about this story. While the links may not be the highest of quality in terms, they will still be links from sites like TechCrunch, The Verge, and many others with high PR scores. The story will die down over the next couple days and then I am sure we will see another influx of stories as their ranking increases again, providing more links to their domain again.

This is not at all surprising, they did a dumb thing because they got desperate and greedy.

In the words of DMX, stop being greedy: http://rapgenius.com/Dmx-stop-being-greedy-lyrics

Also, as of right now they aren't showing up on google for "stop being greedy rap genius" https://www.google.com/search?q=stop+being+greedy+rap+genius

Another one of DMX's classics: X Gonna Give It To You http://www.youtube.com/watch?v=5jkeRpQKFDg

    Arf arf
    Yeah, yeah, yeah (Grrrr)
    Uh, Yeah don't get it twisted
    This rap shit, is mine

What if X is actually Google? Could DMX have predicted this whole ordeal? 63% drop in traffic is absolutely insane. It's not like Google is going to backtrack their decision so recovery seems impossible now when typing rap genius doesn't even show up their main site.

As a total aside, how do sites like Quantcast and Alexa get traffic information from other websites that they don't have code on? (or do they?) I feel like getting accurate traffic statistics on one's own website is tough enough.

This is 'directedly measured data' from RapGenius (notice the icon / message in the upper right hand corner). RapGenius voluntarily added tracking code to their website to enable Quantcast to do this.

In this case, then, the numbers are likely to be quite accurate.

Ah, cool. I did not see that. That makes much more sense.

Many of them just make it up:

http://blog.reddit.com/2010/07/experts-misunderestimate-our-...

Actually no, in this case it's directly measured. Quantcast only shows exact stats for sites that include their JS tracker. Sites normally do this for advertising purposes (advertisers don't have to wonder about your traffic).

When I worked at reddit, the shakedown worked something like this:

1. ComScore (or one of their competitors) makes up a small number and tells advertisers that's your traffic

2. Advertisers believe them

3. You call up ComScore and say, "WTF?"

4. ComScore says, "Well, if you don't like the numbers we're reporting, you can just add our tracking bug to your site"

And then you're in a position where you either have to betray your users' privacy by putting a third-party tracker on every page you serve, or have your traffic stats dramatically underrepresented to the advertisers you rely on.

That sounds scummy as hell. It also sounds like a problem that needs fixing!

Depends on the measurement company, but it's normally a mix between ISPs, browser toolbars and panels.

you're looking at christmas day. it'll be more interesting in a week.

Interesting thing to note: the severity of penalty may be hurting Google too. It was already mentioned that Rap Genius lyrics can't be found even when the words "Rap Genius" are included in the query. So when a user wants to find a Rap Genius lyrics page via Google (as is the custom for most of them), Google fails to provide them with a relevant result.

It looks like this is because all the stats aren't in for December 25th yet. If you look at other sites, they also have a slant. It didn't look like last year Christmas traffic had much of an effect on numbers, so I will credit this to information not being complete yet.

It'd be interesting to see which website(s) benefited from the diversion of this traffic.

Worse happened to Quickmeme when they were banned from reddit for bot-voting.

Do you have any sources/further information on their numbers?

http://www.reddit.com/r/JusticePorn/comments/1hgceg/quickmem...

Thats well quite as expected. But I think out of those 493K, most is from the blogs posting their penalty news. This might go even lower !

The more interesting point is how this illustrates how many businesses are almost completely reliant on Google to make their business viable.

We hear all the time that it's not good business practice to rely on someone else's service to drive one's business (building a Facebook app as a business, etc.), but how many of us include Google search results in the mix?

Their site traffic growth over the past 6 months is impressive. And Google hit them where it hurts most given that 70% of their visits come via search. More details on Rapgenius.com analytics here http://www.similarweb.com/website/rapgenius.com

I'm sorry, do we really think that Christmas is an accurate sample day? L2Math.

Funnily, that gives idea that how much traffic bing contribute.

What exactly did they do that google didn't like?

They recruited bloggers and offered to tweet links to their posts in exchange for putting code linking to RG pages on their blog. It was on here a few days ago and Google quickly removed them from 1st page search results as punishment.

Its example where trick of growth hacking went wrong!

OT but how do lyrics sites handle copyright issues?

I believe they are meant to license the lyrics although accordingly to this article many don't: http://consequenceofsound.net/2013/11/rap-genius-and-other-l...

Thats a huge drop. How accurate is Quantcast?

When even the main domain is on page 6 or whatever, it will drop more. Quantcast requires site owners to implement a tracker, so I guess it's pretty accurate.

Here is some drastic example I found recently (thin/spammy content penalty I guess):

http://extremetracking.com/free?login=fm3u91 http://www.alexa.com/siteinfo/freemp3us.com

From 14,000,000 uniques a month down to ~80 a day :D But obviously Google still has work to do, since that site made a lot of money within that 1-2 months period.

in this case, very accurate. Rapgenius has Quantcast analytics codes on their website.

From my experience. Fairly accurate.

What if the remaining traffic is mainly due to HN people reading this article? Then they're in real trouble..

Error! We are unable to retrieve traffic data for this property at the moment.

Well, thats intersting...

What does 'online' mean? How can there be more people mobile than the total online?

still way way too early to tell, lets take a look on the 28th before we all go making friends

What do you mean exactly by 'making friends'?

its way too early to be friends as well, lets wait a few days.

lol its too early...friends = threads :)

This is the best PR stunt ever.

accidental one!

hahaha hope its not after my suggestion to google on there facebook page

Could they simply pay Google $$$ to fix this issue?

Haha!

Reply on news.ycombinator.com