Comment by schwap | Hacker News Reader

Comment by schwap | Hacker News Reader

I guess the reasoning behind the conclusion is here:

> When Hosoi compared the R values of FanDuel contests to actual professional sports teams, she found that the outcomes of real-life sports involved less skill and more randomness than

> their fantasy corollaries (with the exception of basketball, whose many opportunities to score reduce its randomness). In fantasy hockey, skilled contestants will succeed slightly more often than elite NHL teams will. In a real-world hockey game, a weird puck bounce can lead to a fluke goal.

> Because hockey games are relatively low-scoring, moments of randomness can have an outsized impact on the outcome of a matchup. To a lesser extent, the same is true in baseball and football, both games where scoring opportunities can be hard to come by.

But I think that's a flawed conclusion. The article notes that many unskilled players play daily fantasy as well. In the NHL (to continue the example), every single player and team is elite compared to all hockey players. If NHL teams played minor and rec league teams, their records would be stunningly lopsided.

From the article

>laws in the United States tend to have the opposite opinion, and favor skill over chance

>FanDuel approached Anette Hosoi... to see if their offerings were more games of skill or of chance

>she found that the outcomes of real-life sports involved less skill and more randomness than their fantasy corollaries

That should tell you exactly why the conclusion is flawed. As the old line about lies and statistics implies, the numbers could be massaged to show numerous different results. I imagine it is just a coincidence that in this instance the numbers show exactly what the sponsor of the research wanted to show. Both this research and this article are nothing more than PR puff pieces for the daily fantasy sports industry.

But you can't tell the conclusion is flawed just from that. You're saying that funded research is by definition only valid if it yields results that are neutral or harmful to the interests of whoever funded it. That's blantantly, obviously false.

You can take that as one piece of data that should tend to raise your level of suspicion, but you still have to look at the actual study to evaluate it.

That's a critical thinking error. When an interested party funds research, they don't publish results that are misaligned with their interests.

In technology, you'll see shill "analysts" publish white papers that the FooCorp Model Bar Computer delivers 46% more ROI. This is similar bullshittery.

Ben Goldacre - a British epidemiologist - has done a lot of work on this area, but focused on the Pharmaceutical industry and their practices in running/publishing the results of drug trials. He's got a great blog at http://www.badscience.net/ and a book called Bad Pharma.

That's a separate problem. It's true that the selection bias around a lack of negative results is a problem, all that means is that you cannot interpret a lack of support in the literature as evidence one way or the other. I can't assume that a drug has no side effects based on a lack of documented research pointing out those effects. Such a study would likely be kept quiet, I know. When a study is actually published though, you can hopefully interpret the study on its own merits. There's still potential for fraud and misconduct, but that's why we'd ideally devote more time to replication studies.

That said, I was assuming from the article that there was an actual peer-reviewed study resulting from this research, but the only thing I can find online is an affadavit Hosoi filed on behalf of Fanduel. If there's no published study anywhere, there's nothing to really counteract the suspicion you assume based on funding.

When I said why its flawed I was providing the likely reason and not evidence of the flaw. Rereading my comment, I can certainly see how one might interpret otherwise. Sorry I wasn't clear on that.

When it comes to the specific evidence of the flaw, I think schwap's comment that I was replying to has it covered.

>But I think that's a flawed conclusion. The article notes that many unskilled players play daily fantasy as well. In the NHL (to continue the example), every single player and team is elite compared to all hockey players. If NHL teams played minor and rec league teams, their records would be stunningly lopsided.

Came to say this - Hosoi should compare the top 0.01% of fantasy players against each other and report back

It's a completely ridiculous comparison. Imagine a high school team playing the Cleveland Cavaliers or Golden State Warriors.

Poker is probably a better example since it's far more likely for bad players to play in a game against a great player than it is a bad basketball player to play against Lebron James. In poker, the more skilled players there are at the table, the harder it is to tell who the bad players are. Even the highest cash games in the world rely on extremely wealthy gamblers who aren't skilled in order to remain lucrative to the professionals.

> Came to say this - Hosoi should compare the top 0.01% of fantasy players against each other and report back

Why?

If you really did that, then every single game would be considered a game of chance, even chess.

I don't necessarily agree with the article's conclusions, but only comparing the top players isn't a legitimate thing to do -- if you take any distribution and zoom in at the extremes, it will look just like noise.

It's not as ridiculous as you think. To continue to abuse the hockey analogy, there are ~979000 registered hockey players in the US and Canada[1]. There are 30 NHL teams, each with a contract limit of 50, for a total of 1500 players. That's 0.153% of all hockey players in North America.

EDIT: Need to read better, that's an order of magnitude off so it is a bit ridiculous.

[1] https://en.wikipedia.org/wiki/Ice_hockey#Number_of_registere...

I think you're missing what I'm saying.

The criticism is, 'I don't like the article's metric because it doesn't zoom in on the tail of the distribution'.

But if you do that with any game, the answer you will get is uniformly 0.5, even when comparing chess to tic tac toe.

No I understand your argument, I'm just trying to illustrate how small a part of the tail professional sports really is (turns out it's small but not that small).

Depends on the sport. For NBA and NFL athletes in the US, it is close to 1 in 10k high school athletes in those sports that make it to the pros

I think schwap's point was that you can't draw the conclusion "Fantasy games are more games of skill than their professional counterparts" when you're comparing people self-selected to have close to even skill levels (elite athletes) to a random sampling of the population.

Of course there's more skill advantage in a random sampling than a sample biased by skill.

They're basically saying that in RL sport, the skill level at that top level is so close that 'chance' has a significant impact on the outcomes. However, the skill gap between 2 random players of fantasy sport is so large that chance has very little impact.

If you read down the article it does say that games of skill between similarly skilled players will result in 'chance' having a bigger affect on the outcome. But lets not act like DFS are not a game of skill.

Reply on news.ycombinator.com