Scientific integrity in a climate of perverse incentives and competition (2017) | Hacker News Reader

Scientific integrity in a climate of perverse incentives and competition (2017)

ncbi.nlm.nih.gov

100 pointszoid18 comments

This perfectly describes the current state of academia. The belief in treating academia like a business to promote 'efficiency' and 'accountability' leads to all the ills you see in business. The bottom line is that you should be extremely distrustful of anything aggressively promoted as 'scientific consensus'. If an idea is successful, whether correct or not, then as in business people will blindly support it to 'get a piece of the pie'. As an example, machine learning is replete with incremental, incorrect, and false papers all riding on the success of a few genuine results. The most successful academics are not the smartest or best researchers, they are those best at marketing their results.

> If an idea is successful, whether correct or not, then as in business people will blindly support it to 'get a piece of the pie'.

I wonder whether articles like this are driven in part by the same copycat effect described above. People love to complain about their workplace. Although the article identifies real problems, it is lean on evidence for the magnitude of their impacts. Many of the citations are to other opinion pieces. It's certainly worth improving things where we can, but the sky is not falling either.

For example, the abstract leads with "If a critical mass of scientists become untrustworthy, a tipping point is possible in which the scientific enterprise itself becomes inherently corrupt and public trust is lost, risking a new dark age with devastating consequences to humanity." Which sounds serious. But based on articles posted to Hacker News previously, 1–10% of scientists (excluding countries with direct cash payments for accepted articles) fabricate or falsify data, which is a typical percentage of bad actors in most groups. The other 90% still appear able to serve as quality control in the usual way. People can tell if an article is likely to be b.s. with about 70% accuracy based only on its description, not even reading it. I think any defects in science feel more severe than they really are because science has been so successful that our expectations are now sky-high.

I do emphatically agree that promoting altruistic and ethical norms should be a focus of scientific institutions. Major discoveries will arise from curiosity and serendipity, as they always have, they cannot be predicted in a proposal's Gantt chart.

In reality, there is no such thing as "consensus" in any expert field, not even in maths or physics, not even when it comes to fundamental theorums, let alone in leading edge or new fields of study. So, if someone claims there is "consensus" among "9 out of 10" or "97%" or similar, it's a tell that there is something else in play.

General question about incentives..

How do incentives get fixed? Do we have any social systems engineers that focus entirely on designing or adjusting incentives to encourage social systems to organically produce the desired outcomes?

Is there any incentives-first systems thinking going on?

This quote from the article sums it up:

> "When you rely on incentives, you undermine virtues. Then when you discover that you actually need people who want to do the right thing, those people don't exist." —Barry Schwartz

I'd say you need to reinforce psychological traits in the work environment, like following your curiosity for the sake of it (watch others do it), or feeling guilty about publishing something that you know may mislead the readers (watch your supervisors correct such failures, instead of teaching you how to trick your paper into a high-impact journal). De-emphasize gate-keepers, the quantitative measurements (like citation count), financial pressure to perform. All of those will be gamed if they are too important.

Then build a reputation (as an institution) to work that way, I guess. This will give publications from that institution a higher credibility (long-term). The cost of being mislead by the claims of every paper you read is simply too high.

>"When you rely on incentives, you undermine virtues. Then when you discover that you actually need people who want to do the right thing, those people don't exist." —Barry Schwartz

Quite so. And things may be worse than that. It may be that both incentives and virtues are unreliable. For example, the famous day care study mentioned in Freakonomics casts doubt on the utility of incentives when it comes to social problem-solving:

https://sites.google.com/site/cvhsbahm/economics/econ_calend...

Moreover, my guess is that virtues themselves may also be unreliable because they're are about outward behaviour, which is often inherited and not explicitly understood, and which may not be passed on.

Jacob Bronowski, scientist and author of The Ascent of Man, identified the primary scientific virtue as what he called the habit of truth. This is to rigidly tell the truth about all things, both in private and in print, including about the minutest details, in one's scientific work.

It is a matter of opinion, but it seems that the habit of truth no longer pervades the scientific enterprise, now fully professionalized and bureaucratised.

Perhaps it was lost because it was only a habit. Whereas the love of truth, beauty, and so on, are spiritual values: modes of being rather than habitual outward behaviours. Which may explain why (according to Ed Dutton and Bruce Charlton), many 20th century scientific geniuses were first generation atheists. They inherited a reverence for truth and reality; they were able to make important scientific progress, but their ardour could not be sustained beyond a generation or two.

https://geniusfamine.blogspot.com/

I started a PhD believing that scientists still held on to the habit of truth. The gradual realization that modern science rewards bullshit artists, and that great research happens in spite, not because of today's science culture, is one of the most bitter disappointments of my life. I feel robbed of one of the main reasons to be proud of being human.

I think your account is a common one, unfortunately. I’m at the age where many of my peers are in grad school in a variety of disciplines, and while I’m not proud of my disillusionment, it scares me on behalf of them when the conversation reveals that their expectations aren’t matching the reality. I wish they knew what they were getting in to.

This is also why we can’t let science evolve into a religion, for what it’s worth. It can’t be the core of human spirituality: it’s too human of an institution.

> For example, the famous day care study mentioned in Freakonomics casts doubt on the utility of incentives when it comes to social problem-solving

Technically it mainly casts doubt on monetary forms of incentives. But it also mentions there are other kinds of incentives too.

No one wants to talk about this (or at least few), but in my experience, so much is being fed from the federal government, at least in the US, that changes at that level force massive changes system-wide. Without meaning to come across as reactionary, most of US academics and its problems are being driven by money flows from the federal government to universities via grant mechanisms.

I think like a lot of major problems, it will require a lot of changes coming from a lot of places.

One massive one, though, in my opinion is serious, significant changes in grant funding processes. There need to be big changes to how grants are awarded and funding decisions are made, and how those grants are actually funded. Moving to more of a lottery-style system (where everything above, say the 50%tile are entered into a lottery), and funding individuals are examples.

I also seriously believe indirect funds need to be cut. In the very least least indirect funds need to be line-item justified, rather than being based on some arbitrary percent cut or however they're negotiated now, with independent audits of the indirect funds to determine whether universities are asking for more than they're actually cost.

Some universities get around the indirect fund incentive structures in creative ways. For example, one place I know of takes all the indirect funds and redirects it toward university-grant support services at the university, so none of it goes into other activities, and it doesn't go to specific departments. But even that seems insufficient.

A lot of this is going to require other things, like trimming administrative bloat, democratizing university decision-making, punishing superficial promotion and tenure criteria, and increasing transparency about why universities are making decisions that they are. This is just the tip of the iceberg.

I have no idea what's going to happen. Former federal agency heads have compared it to the pre-2008 financial bubble, and that's what it feels like to me. But how that bubble pops I have no idea. I could see the government coming in and cutting indirect funds; I could see something like indisputable proof of COVID being the result of a lab leak maybe causing a public outcry over what's going on; or I could see people just not going to grad school anymore and things drying up. Or maybe it will just get exponentially worse for awhile.

I agree (academic research scientist, geosciences, now semi-retired). The current system is utterly unsustainable. Many outside academia are unaware how much universities rely on federal funding of indirect costs, and although the overhead rate continues to creep, the university itself supplies less and less real support. I also don't know how it will end or how the bubble will burst, but the entire enterprise is suffering from unsustainable costs and corruption of ideals.

It may be slightly out-of-context here (...because it refers to ethical incentives instead of academic incentives) but Transparency International UK has published a paper about management and selection of incentives that encourage good behaviour: https://www.transparency.org.uk/publications/incentivising-e...

There may be technical remedies if not cures for this channel noise issue. A community which grew out of BOINC got involved with distributed ledgers and independently invented many concepts which are hyped today. - They get no attention because they are non-profit, but they stand poised to reinvent the basic infrastructure underlying peer review and to with it upend conventional systems of governance because the basic problem they are facing (as per the topic) is the same. I think it's a matter of time before that project branches or merges with more visible distributed grids.

lots of solid research work on distributed consensus, layered voting, and "deep democracy" existed 20 years ago, on paper. Yet we got a decade of Facebook and spy-oriented DNS tricks, for the masses. I am not certain that that technical excellence you describe means very much at all, by itself. To further adoption in the field, there are multiple and well-known steps to implementation. Do these BOINC academics want that? not clear at all ..

ref: Diffusion of Innovation studies https://en.wikipedia.org/wiki/Diffusion_of_innovations

This is more of an evolution than a design so I'm not sure how much intent will factor into what they produce. The software tools, the head-turners and the ear grabbers that are available to them constrain where they may go from here.

The beauty of the distributed platform that is emerging is that each service which makes up a project like Gridcoin can be produced independently within a larger ecosystem. Systems of governance are also such services, meaning the system does not have to be set in stone from the start.

How do I square this with the knowledge that some groups benefit by discrediting legitimate scientific consensus, as has happened numerous times in the last century? I'll share my own FUD [1]:

> The statement reading “Indeed, delving deeper, 34% of researchers self-reported that they have engaged in “questionable research practices,” including “dropping data points on a gut feeling” and “changing the design, methodology, and results of a study in response to pressures from a funding source,” whereas 72% of those surveyed knew of colleagues who had done so (Fanelli, 2009).” should have been written as, “Indeed, delving deeper, up to 34% of researchers self-reported that they have engaged in “questionable research practices,” including “dropping data points on a gut feeling” and “changing the design, methodology, and results of a study in response to pressures from a funding source,” whereas up to 72% of those surveyed knew of colleagues who had done so (Fanelli, 2009).”