Hacker News Reader

Comment by ryandrake | Hacker News Reader

The crux of the article was presented near the top:

"on social media it is easy to mistake popularity for credibility"

I wouldn't say this is limited to social media. This flaw is present in search engines as well, which is why link farms work (and search engines are in a constant, futile battle against them). In politics, it's why astroturfing works and why the more you spend on saturation advertising, the better your chances are. It's really a flaw in human nature--to confuse popularity for credibility.

I'm convinced that anything out there, online or off, with a ranking system that relies on popularity (counts of links, likes, eyeballs, votes, etc.) is ripe for disruption. What's the alternative way to separate genuine credibility from fake (purchased) credibility? Figure it out and you'll be a billionaire.

View on HN

Psychologists call it "social proof".

It's why brands say "we're #1 in X", "we're the most popular Y", "more people buy our product than any other".

It's why sitcoms have laugh tracks.

And I am not confident on "disrupting" human nature. Most "disruptions" follow the grooves laid out millions of years ago; what changes is the environment, not the subjects.

A slight disruption, or perhaps just coat tail riding. A Russian detergent company took advantage of Proctor and Gamble advertising. You know the type of ad: "when compared to an ordinary soap." So the company named their soap Ordinary and made it look like the soap in the advertisement. Free advertising thanks to PG.

http://articles.latimes.com/1999/jul/09/news/mn-54309

Never thought about that!

Oblig xkcd :

"https://xkcd.com/993/>https://xkcd.com/993/"

I think one of the fascinating things with social proof is that it usually works even when we know we're being manipulated, or dislike the method. E.g. people that dislike laugh tracks still report shows on average as more funny with them than without them.

It was already kinda figured out in the article. Examining the link 'neighborhood' is an effective way to determine spam within a network. The major players in search, social and email use this technique as one of their primary spam filters.

It's also important to understand that in any high stakes game sniffing out the bad actors is always going to be a never ending cat and mouse game, so no one solution will put an end to it.

but detecting spam is really just a way to filter out "legitimate" popularity from "illegitimate" popularity - it's still relying on popularity as a signal for quality.

Google is somewhat good a discerning credibility, IMHO. At least, whenever I search for something I tend to get exactly what I'm looking for without a bunch of spammy links. Of course, this could be because most of the stuff I search for is just on Stack Overflow or Wikipedia. :)

StackOverflow ranks answers based on how many upvotes it gets. In other words, it establishes a user's credibility based on his answers' popularity...

Yeah, but popularity on StackOverflow is related to "does it work for me," a metric that ultimately depends on evaluation by an impartial machine (or a human using their knowledge of the impartial machine to perform the same evaluation). You can't get popular on SO by answering with random snippets from TAOCP.

Or maybe you can and I'm missing a business opportunity. A TAOCP+OraclePropaganda bot sounds like something that might make bags and bags of money.

How do you know the up-votes on a StackOverflow answer were not fake/bots/paid for? How do you know a GitHub starred/watched repository is really useful and not some job applicant's astroturfing campaign to make it look like their code is widely-used?

>How do you know the up-votes on a StackOverflow answer were not fake/bots/paid for?

Because the quality of the top voted answers speaks for themselves in 99% of the ocassions.

I disagree, I find surprisingly often, the top one or two voted answers have low quality in many of the types of searches I end up doing on StackOverflow. Ultimately, it ends up being a resonable ranking when you have a smaller set of answers, and a good mod(?) community that's effective at filtering out actual spam.

Sometimes a longstanding answer has so many upvotes that it stays at the top even if it's no longer the best answer. In cases like that, there's usually a comment that explains why that method is no longer preferred and what you should do instead -- sometimes pointing to the better answer. And if no such comment exists, be kind and add it.

The more elementary the question, the more upvotes you get. So your credibility is also dependent on the superficiality of the questions you are answering, even more so I'd say.

This is the problem: SO ends up favoring more shallow knowledge as time marches on.

You can't use popularity metrics to mean anything substantive.

If you start your search term with the phrase "howto", chances are wikihow.com is among the top results.

There are many similar websites who focus on low quality content optimized for search engines.

wikihow is awful. Because it is burying useful content with their superfluous garbage.

OTOH StackOverflow employs similar tactics but it is legit and usually useful content.

Should Google discriminate between these two? If, yes based on what criteria? (Remember, wikihow pages look like they're user generated, legit content) Is there any objective criteria?

I personally would like wikihow to take its place in hell, ehm I mean obscurity. Right next to w3schools.

w3schools is at least useful: short API summary plus an example and the pages load a lot faster than Mozilla's MDN

An independent service that publicly rates a person's credibility and consistency (maybe other metrics such as honesty or accuracy) objectively into a reputation score. You would start out rating people for free, probably celebrities, prominent politicians, persons of note in scientific fields, journalists, etc. These would have to be thoroughly documented profiles with established public criteria for the scores in 100% transparency. Once you have established a reputation for accuracy you can start to charge other people to have their score calculated and added to the reputation index. Maybe have a public poll for who to profile next and do several for free each month. Basically a ratings agency or better business bureau type system but for individuals who want to be public figures and people of influence. Part of it would be validating the person's real life identity and the online personas that the person uses so that online exchanges (ratings, discussions, reviews, etc.) could potentially be filtered for only validated real individuals if desired.

This is very costly to do. Listen to a political candidate's speech, then try to validate what they are saying on your own. It's incredibly time consuming.

A ratings service that relies on the person being rated to pay results in conflicts of interest. You will also find it hard to have anyone want to pay for anything that might be critical of them.

A service that did this would require a strong guiding mission, because it's cheaper to shill blog spam than to actually dig in and do accurate research while citing sources.

It should be a crowd-sourced service like Yelp where people rate individuals and individuals get a star rating from 1 to 5 stars.

That would be a popularity contest, not a fact checking service.

What about a solution using a crowd like Mechanical Turk and just presenting a de-contexted quote? This could be presented in a couple of ways:

Is this fact correct? or Does this statement seem correct?

Then present a phrase: "There are less than 10,000 troops in Afghanistan."

This is somewhat related - tracks predictions by various sources to assess creditability:

http://www.pundittracker.com/

Trust metrics. See for example http://www.advogato.org/trust-metric.html

It's a definite flaw, encountered in so many places with varying degrees of danger, obviously some people are going to try and maximise their results by abusing it.

When I think about credibility though it quickly appears to be a bit of a mine-field, highly context dependent. If PG writes an article people are going to attribute a huge amount of weight to it. Obviously he is very credible when it comes to a lot of subjects, but context can tell all.

I'd say if he wrote about how paste engagement rings were going to be the next big thing there'd be one or two embarrassing proposals on Youtube within the month. But if he wrote on why paste jewelry is the next big thing in fashion businesses there is obvious weight (whether or not it's correct) to that opinion. So I'd say matching context with popularity, and figuring out transferability of credibility are where this can be disrupted.

While this sounds like a very plausible argument, and I agreed wholeheartedly at first, I suggest taking a closer look. I doubt many people, when asked, will say that popularity implies credibility. Credibility is not what's driving these people, it's just the popularity, plain and simple. There's a difference.

What people say when asked is not very interesting. How people actually act matters much more. People say a whole lot of things about their own behaviour when asked that has absolutely no relation to reality.

Atleast for twitter we can have a service which can list the fake followers a user has. This can predict fake followers based on some heuristics like following to follower ratio,tweet frequency, reply ratio etc.

Reply on news.ycombinator.com