Hacker News Reader

Comment by acatton | Hacker News Reader

> Many researchers are privately terrified of being falsely accused by the "data cops," one scientist said. But no one wants to criticize them because no one wants a target on their own back — including this individual, who was granted anonymity for this reason. "Batman is a vigilante," the scientist said. "So is the Joker."

> For collaborators, it's a stressful and infuriating time. When a paper is retracted, the research is erased, and all related citations are lost.

> Others who attempted to build on Gino's studies are grappling with having wasted time, money, and energy.

It feels, to me, that they're making Data Colada (the data vigilante group) the bad guys here. It's like blaming the Police for stopping a criminal because they took away the criminal's source of income... I'm kinda uneasy with this line of editing :/

View on HN

Everyone, please read the whole article, not just this comment. The second two quotes here appear very out of context in this comment - they are about anger and frustration towards Gino, not towards Data Colada.

The first quote here is not a defense of Gino, it's a nuanced opinion of self-proclaimed vigilantes who are also human and might not always make perfect decisions. It is followed by this:

> Discussing the allegations that Gino falsified data is seen almost unanimously as fair game, especially in light of Harvard putting her on leave. People expressed more mixed feelings toward disgraced researchers such as Cuddy, who failed to replicate her findings but did not fake her data.

I have no special insight here, but this at least seems like a reasonable thing to discuss.

> It feels, to me, that they're making Data Colada (the data vigilante group) the bad guys here.

I strongly disagree. The light questioning of Data Colada was a very small part of the article, and even that section did not give me the impression that they are "bad guys".

If the findings can’t be replicated, were they worth publishing to begin with? I know there is pressure to publish, but the effect on society is significantly greater than a single researcher’s personal job security. If nothing else, papers with expensive or one-off experiments should be clearly marked as observations that should not be relied on as foundational for other work until replicated. When they’re cited peer reviewers could make sure they are referenced as such without requiring the peer reviewer to intimately know the details of the reference.

I'm in a different field (computer security and privacy) and I want to point out that the notion of "replication" is trickier than what you might expect. Let's say I do a simple empircal measurement study, say on network traffic on some piece of internet infrastructure day-by-day in the year 2020. I find there's a huge sustained increase starting in mid-March, sustaining through the end of the year. I submit my results for publication. How can someone replicate those results, when the measurements were subject to conditions unique to that observation period?

The best anyone can do is to replicate my statistics/analysis if I gave them my data set. That's different notion of replication. Would you consider that sufficient?

That narrative is insane. It is one thing to produce shoddy research. Even good senior researchers sometimes just have a paper that... doesn't work out well, with a late found flaw in the methods that can't be fully mitigated or uninteresting findings, and then they'd rather still try to publish it than shelve months of work. That's not good, but almost unavoidable in the current academic environment.

The accusations here are very different: Producing fake data is fraudulent, of course the papers need to be retracted, and people involved and in the know need to be punished. The quote from the article describes well something that seems to be a growing group of people within academia:

> It's possible "these people don't care and never cared about the science," Michael Sanders, a professor of public policy at King's College London, said. "They talk about science, and they talk about experiments, and they sort of wrap themselves in the fabric of the scientific method — just as a way of selling stuff."

Is there a market for publishing the retrospective of the process which resulted in a "late found flaw"? I would expect uninteresting data would still be published if only to prevent someone from going down that rabbit hole in the future.

Indeed. That's nasty. They throw in a detracting bit of racist accusations as well.

> all related citations are lost

What an extreme BS argument. It's really grasping at straws.

The citations aren’t lost though, are they. They still exist in the paper, they just point to a retraction. You can still find the papers that cited the retracted paper.

And I suspect you’ll still get new citations since it’s not unheard of that people cite a paper based on seeing it in some other paper, rather than reading the paper directly. I don’t think people should do that, but they do.

The citations won't be counted by the big databases like scopus and web of science, which are often a prerequisite when being measured for grants etc.

Next to nothing related to scientific research is tracked in scopus and web of science anyway, so your point is moot.

I don't follow you. They track scientific publications and their citation networks. I feel like you are reading something in my post I didn't claim. My point was that the citations can be lost in the sense than they wont be counted by the bibliographic databases that are usually used by administrators to get citation numbers for evaluating researchers and institutions. Of course the citations still exist between the papers. But for a researcher their bibliometric indicators will come out lower after a redaction, in most cases when they are evaluated for grants or career progression.

But that measurement isn’t done through Scopus or Web of Science.

It’s done by evaluating biosketches of grant applicants in their application. It’s not like the grant reviewer checks Scopus for the applicants citation count or something.

I recently reviewed some grant applications and I reviewed the applicants bio sketch including their experience and publications. I didnt count the citations for each paper. But if I did, I certainly wouldn’t count citations for any papers that were retracted.

Of course they shouldn’t be measured. They are recalled.

Of course, I don’t think people with recalled papers due to their fraud need to ever concern themselves with grants. (Kind of like felons can never be an officer of a publicly traded company)

Of course the citations should be lost (not counted).

But the paragraph is about collaboraters being angry and frustrated. And they seem to be angry at those who faked the data. Which is understandable.

You would hope that this would also mean that collaborators getting their name on a paper would put a little more effort into scutinizing the data provided by co-authors.

Maybe it is just a little to easy to work on paper and just accept the underlying data without asking any questions.

Guilt is a very effective way of keeping corrupt status quos going. Even if you change your ways, there is still dirt on you. The only way to break the cycle is if someone from the outside exposes the filth, or someone guilty on the inside decides to pick up his cross and accept the consequences of admission and exposure. On top of that, there are many who won't even entertain the possibility that the accused could be guilty even with evidence, because they have an emotional interest in preserving a false image of decency. Or they're terrified of the consequences of reporting the problem.

This is what happens with sex abuse. When there's enough guilt to go around in the right places, the guilty form a kind of hellish camaraderie, a Mexican standoff of pointing fingers. Once such people assume positions of power, they also have a tendency to concentrate corruption in the institution by favoring the corrupt. Doing so entrenches the standoff and eliminates those who would oppose or expose them. The so-called "lavender mafia" is a well-known example, because the media hate nothing more than the RCC, but this occurs everywhere and at greater rates.

What's important is what we distinguish between institutions and the people that temporarily occupy them. Good institutions should generally be preserved. It's the corruption that should be dealt with. Of course, that's not always an option. Sometimes, corruption destroys the institution, or perhaps the institution is beyond repair. But generally, there is a presumption in favor of cleaning house rather than abolition.

> it's like blaming the Police for stopping a criminal because they took away the criminal's source of income

Coming soon to a San Francisco near you!

It's obviously already there in parts of California/Oregon.

I wonder if it's like a steroid doping situation:

What kinds of steroids and how much of them can I get away with using so I'm in the perfect spot where I am ahead my fellow doping athletes, but still under the bar of detection...

There's a pretty clear solution for that in athletics which is sometimes implemented. They take blood samples and store them for decades. New methods can come along and analyze the data and detect fraud long after the fact.

The same can be done in science, it just more disclosure of data.

This doesn't really help, though? My understanding is that in the sports where doping was a thing, if you wanted to be competitive at top level, you basically had to find a way to dope. This isn't something 'just the winners' did. Near everyone was doing it.

So, yeah, you can look for it with later testing, but all that will do is claw back a winner. It doesn't help anyone that wasn't doing it. Is why none of us have any clue who the best cyclist during the Armstrong era was. With the scary view that the cheating was so widespread that the current winners were still the best if everyone stopped cheating.

Of course you can't fix the past before they started retaining samples.

The point is to prevent people from doping moving forward because they know they can be caught.

If we had better data on cyclists from when Armstrong was racing, you could absolutely claw back the Awards and give them to the best cyclist that wasn't doping.

I think that misses just how entrenched doping was. As mind blowing as it is, it is fairly safe to say that everyone that was making the leaderboards was likely cheating.

Probably the entire peloton was -- if so, can it still be considered cheating?

Of course, not everybody used EPO and not everybody who did used as much as Riis, Ullrich, Pantani, Virenque, Armstrong, Zabel, Hamburger... Blood doping existed long before EPO. Amphetamines (and strychnine!) were used long before blood doping. Induráin almost certainly used either EPO or blood doping.

EPO and blood doping can still be used (and are likely still used) but noone can push it to the dangerous extremes of the 90's. That will be caught (because such hematocrit values aren't allowed anymore -- and it's easy to test for).

You are still trying to fix the past which is impossible and not addressing the possibility of fixing things moving forward.

For a lot of people, they are most interested in the recognition and achievement. You are basically saying "you may be a footnote as someone that should have won, if the others weren't caught cheating years later." Right? I can see how that is less than comforting. (Especially so with no way of knowing who all is actively cheating at the time.)

I can't say it is a bad idea to try, mind you. It could very well work. I just have my doubts.

They still all are. Just not as brazenly as the past. More nuanced, more systematic, but overall less of it is happening.

No one has ever been unfairly accused.

The bar for being accused is so incredibly high. People are given so many possible outs and are allowed to produce the most implausible explanations. And then, even having one or two faked papers basically has no impact.

Here's an example.

Ken Rogoff and Carmen Reinhart. Professors at Harvard. Despite having faked the paper that introduced austerity to the world, had an immense impact on the economy, and caused massive and needless suffering for hundreds of millions of people. That paper still continues to haunt countries today. To make their paper work they had to lie 3 times: to exclude some countries, exclude data from others selectively, and reweight all of the data. None of this is reported in their paper! Each of these took deliberate actions on their part, they cannot be mistakes. Every single "mistake" they made is carefully calculated to produce the result they want. Every one of them is needed otherwise their paper would be junk.

What happened to them? Well, Reinhart has been promoted to the Chief Economist of the World Bank and is still a professor at Harvard. Rogoff is still a professor at Harvard too. Their faked paper literally resulted in deaths, toppled governments, and the rise of the alt right in places like Greece.

But they got a promotion for it.

The only people who are afraid are the people who are faking their data. No real scientist is getting accused of anything here.

What do you make of Reinhart and Rogoff’s rebuttal to this criticism? They acknowledge the excel error but dispute the rest. https://www.nytimes.com/2013/04/26/opinion/reinhart-and-rogo...

Look at it from the point of view a scientist.

They published a paper that claimed they ran analysis X. Reviewers look at it, and believed them, because we usually don't assume that people are lying. And it's hard to read over every line of code and compare it to the paper to make sure they did run the analysis they claimed.

But in reality, they did a totally different analysis. They hid data, they fudged data, they inserted weighting factors into the analysis. And they never mentioned in this in the paper.

In the article, they claim this was justified.

But hiding the true nature of the analysis they did was not. That hiding, of making the paper read one way but the code another, is plan and simple scientific fraud.

If what they believed in reality is that the code is correct, and that's the better analysis, then they should have just reported in the paper. The reason they didn't, is because reviewers would have looked at it and said that this is unacceptable and doesn't meet basic scientific standards. So they lied.

Even if you are caught red-handed in fraud it's not the end. Dan Ariely, mentioned in the article, continues to be a professor at Duke.

> the paper that introduced austerity to the world

This is complete nonsense. The paper was published in 2010.

The principle of fiscal austerity for EU member states was enshrined in the Maastricht Treaty of 1992 (Article 104) and the accompanying Protocol No 20 on the excessive deficit procedure. [1]

> The reference values referred to in Article 104(2) of this Treaty are:

> — 3 % for the ratio of the planned or actual government deficit to gross domestic product at market prices;

> — 60 % for the ratio of government debt to gross domestic product at market prices.

In fact R&R cite the Maastricht Treaty as the source of the 60% magic number in their paper!

These numbers were basically pulled out of thin air by a French civil servant in 1981. [2]

That is how government policy is actually decided. Not based on an academic paper published 20–30 years after the fact.

[1] https://eur-lex.europa.eu/eli/treaty/tec_1997/oj

[2] https://openeuropeblog.blogspot.com/2014/01/from-archives-st...

You have just enough knowledge to completely misread and misunderstand EU treaties. And, to be rude about it.

These are not hard ceilings. God does not descent from the heavens and smite your country when you go over them.

Virtually every EU country has gone over these ceilings routinely. Germany is above the 3% mark right now, and has routinely exceeded it!

Heck, the average deficit to GDP ratio in the EU is above 3! https://ec.europa.eu/eurostat/web/products-euro-indicators/w...

These are aspirations to be applied as needed, not a proscription for austerity. The behavior of the ECB, the European Central Bank, is what matters. And that's a political and economic decision made with essentially a free hand (given that it has only one legal goal).

Austerity is an economic and political choice. No mechanism in the EU forces it.

Neutral party here, after reviewing the thread:

I read it as an argument for a causal chain of 2 professors => paper => Greece elects far right. A few problems with this:

1. Paper came far after the idea of cutting government spending, so causation is wrong off the bat, as OP gently pointed out.

2. The aspect of the treaty doesn't have a clear enforcement mechanism, yes. However, it was a major part of many countries actions at the time.[1]

3. "What are they going to do, kick me out?" doesn't obviate rules without a clear enforcement mechanism.

4. It's unclear why Greece electing a far right party at some point in the 2010s is enough to motivate the argument, it has a liberal technocratic government and has turned things around substantially, in fact, recent elections delivered an unexpectedly wide reelection victory.

[1] To wit, that's why the Greece stuff took O(years) to resolve. To wit, recently, it was shared that Merkle _broke down crying_ when she felt she was being bullied into enabling violations of it.

n.b. On a personal note, I found the tone you're adopting surprising and not the conduct I'm familiar with after 14 years on HN. What stood out was making personal attacks in response to innocuous comments by interluctors, and the aggressive insistence that they don't understand -- dang has a wonderful shibboleth, "come with curiosity", and I don't see that here. I see someone berating other people for engaging at all because it feels like they're questioning you. If it's the case you are 100% wedded to the idea two Harvard professors created "austerity", it's not worth saying anything at all, it's ill-defined and there's a 1000 ways to read "austerity", 999 of them may conflict with however you are internally defining it.

Just because I didn't point out every detail that was wrong with that post doesn't mean anything else is right. Even the basic facts are wrong.

> 1. Paper came far after the idea of cutting government spending, so causation is wrong off the bat, as OP gently pointed out.

This just isn't true.

The Greek crisis started in 2009, the paper came out in Jan 2010, harsh austerity measures were imposed on Greece in May 2010 by the EC/ECB and then they just kept coming for years on end. The timeline works perfectly and the EC/ECB had not imposed such measures before.

https://en.wikipedia.org/wiki/European_debt_crisis

> 2. The aspect of the treaty doesn't have a clear enforcement mechanism, yes. However, it was a major part of many countries actions at the time.[1] > > 3. "What are they going to do, kick me out?" doesn't obviate rules without a clear enforcement mechanism.

Let's talk about 2 and 3 at the same time.

This just isn't how the EU works. And it isn't how laws work. Laws don't rely on a strict interpretation of a text. Laws are not computer programs.

When someone says something is a "rule", they mean that it is a principle that must be followed "one of a set of explicit or understood regulations or principles governing conduct within a particular activity or sphere."

The 3% is not a rule. The EC does not see it as such. The member governments who routinely go over the 3% level don't see it as such. It's misleading to call it a rule.

If you really want to know how it works, you could start with https://en.wikipedia.org/wiki/Sixpack_(EU_law)

The Stability and Growth Pact (SPG) is the official mechanism for dealing with this. At the moment the SPG is suspended wholesale.

Never in its history has the SPG ever imposed sanctions on any country for violating the 3% deficit to GDP ratio. It would be unthinkable for it to do so. Germany and a few other countries have been trying to reform the system for a long time to give the SPG teeth and turn that 3% level into an actual rule, but without any success.

These are not rules. If you want to call them that, you're misleading yourself and any reader.

> 4. It's unclear why Greece electing a far right party at some point in the 2010s is enough to motivate the argument, it has a liberal technocratic government and has turned things around substantially, in fact, recent elections delivered an unexpectedly wide reelection victory.

I said that austerity led to the far right getting elected into parliament. Which Golden Dawn did, and far right parties continue to win seats even today at lower levels. I'm not sure what this argues? That Greece is not a fascist state today? I never said that.

https://en.wikipedia.org/wiki/Golden_Dawn_(Greece)

> n.b. On a personal note, I found the tone you're adopting surprising and not the conduct I'm familiar with after 14 years on HN. What stood out was making personal attacks in response to innocuous comments by interluctors, and the aggressive insistence that they don't understand -- dang has a wonderful shibboleth, "come with curiosity", and I don't see that here. I see someone berating other people for engaging at all because it feels like they're questioning you. If it's the case you are 100% wedded to the idea two Harvard professors created "austerity", it's not worth saying anything at all, it's ill-defined and there's a 1000 ways to read "austerity", 999 of them may conflict with however you are internally defining it.

I never said they created austerity.

On a personal note, I find the contempt for facts and scholarship on HN quite amazing. All of this information is available on Wikipedia. From the timeline, to the fact that the EU does not consider this a binding rule, to Golden Dawn getting elected as a consequence of austerity. To the fact that I'm not "internally defining" austerity, it's an actual field of study in economics.

"come with curiosity", and I don't see that here ~> On this we agree. Curiosity would involve a quest to find out what the truth is.

> the EC/ECB had not imposed such measures before

Greece had not asked them for a bailout before.

Those measures were a condition of the Troika (IMF/EC/ECB) bailout. And the IMF has a long history of imposing similar conditions, dating back to at least the 1970s. The conditions where straight out of their playbook.

There was an IMF intervention in Portugal in the 1970s, and here [1] is what they had to say about conditionality. Sounds familiar?

> it was accepted that the stabilization program would need to include quantitative limits on domestic credit expansion and on increases in credit to the Government, if it was to be supported by a stand-by arrangement. These limits would, it was hoped, make certain that total expenditure would be kept within the overall resource constraint. Policy changes strong enough to give reasonable assurance that the public sector deficit would be reduced had to be put in place.

[1] https://www.elibrary.imf.org/view/journals/022/0018/003/arti...

> The Greek crisis started in 2009, the paper came out in Jan 2010, harsh austerity measures were imposed on Greece in May 2010 by the EC/ECB and then they just kept coming for years on end. The timeline works perfectly and the EC/ECB had not imposed such measures before.

Greece was free to not borrow money and go bankrupt instead. That would also have led to austerity but in a much more chaotic way.

I have to say your final paragraph seems very off-base. Tone issues were started by diogocp with "complete nonsense" and were only slightly increased by light_hue in their response by calling that "rude".

You're the one calling out only one side for tone and in very dismissive manner. Ironically you are doing the exact thing you accuse light_hue of and should probably take your own good advice.

This is about like saying "associativity is a choice when doing mathematics."

Austerity is simple math. When you borrow too much, you get squeezed later.

Unless you subscribe to Magical Monetary Thinking.

It's fairly common for people at prestigious institutions to write papers about stuff that already existed for years to try and snatch some points, and then to get all the credit, because they are from a prestigious institution with many press related megaphones. The most famous example is probably Von Neumann writing about ENIAC a project he wasn't involved with at all.

> Despite having faked the paper that introduced austerity to the world, had an immense impact on the economy, and caused massive and needless suffering for hundreds of millions of people. [..] But they got a promotion for it.

There is a book from Clara Mattei, The Capital Order (https://press.uchicago.edu/ucp/books/book/chicago/C/bo181707...), which explains they were not the first, and probably are not the last.

This reads like an accusation in disguise ...

Completely and totally different case. You are wrong on the facts.

There is no one in the economics profession who thinks Reinhardt or Rogoff deliberately made that error bc they had something personal at stake in convincing policymakers that austerity policies were the right outcome. Quite unlike the cases of ariely or gino or various others.

They made an embarrassing error but ultimately it was a coding error.

This is an appeal to authority and regardless, if you pass off something as proven fact you should have to be damned sure. Being wrong and claiming later that it’s a coding error should not cut it. This should result in a total loss of credibility

I don’t mind appealing to authority when I know the economics profession personally.

To be clear: it is an embarrassing error. Everyone also thinks this. They have been dragged for it quite a bit. But it was not deliberate.

In terms of the “total loss of credibility” … the fraction of empirical papers with some coding error approaches 1. The question is how bad it is.

I'm right about the facts, you're misinformed about what is wrong about the paper.

They did 3 things in that paper. One of them they have called a coding error. Ok.

Two of the 3 were intentional. They intentionally omitted date ranges for some countries. And they weighted countries differently. This they admit to. And they defend.

Yet, nowhere in the manuscript does it say that in order to get these results you must apply weights to the data and you must leave out some inconvenient data points.

I can make up any argument for anything that I feel like if I start to apply weights to the data designed to make my argument work. They failed the most basic of scientific ethical standards.

Reply on news.ycombinator.com