'Tiny Shortcuts' Are Poisoning Science | Hacker News Reader

'Tiny Shortcuts' Are Poisoning Science

nautil.us

16 pointsBrajeshwar14 comments

I think part of the problem is that what is often referred to as "science" these days is very different from the hard sciences of yesteryear.

There are a lot of "soft" sciences that get increasingly softer every year. Social sciences, gender and women's studies, political science, some of the fast and loose use of "economics" these days.

There are a lot of "studies" these days that are little more than slanted questionnaires or selective correlational studies with wild unsupported theories as to the results.

> There are a lot of "soft" sciences that get increasingly softer every year. Social sciences, gender and women's studies, political science, some of the fast and loose use of "economics" these days.

I don’t think anyone is claiming these are sciences, except perhaps economics. I think you’re fighting a straw man

People get doctorates in these fields and post studies in journals that get picked up by thinktanks and media outlets. It's "science" for all intents and purposes; they're used as a source of authority based on data and analysis and formal papers.

I think your objection is that these fields inform policy and other decisions, and you feel that only science should do that. I think I disagree that only science should inform decisions. Non-science things can inform decisions, we’re generally opposed to murder/racism/bigotry even though there’s no double blind study we can run to determine the correct morality. These fields can impact decisions, and yet not be “science”

"science for all intents and purposes" is the most accidentally flattering anyone has said, and is hilarious in context. Wasn't expecting women's studies to get glazed on HN but I'm here for it :)

Need that on a tee

you keep fighting a strawman that appears to be 'wokism'.

Just so we know you're talking about a real thing, point us to a data and analysis formal paper that you believe exemplary of the "science"

The technical term for these tiny shortcuts is questionable research practices. They are difficult to study, because scientists are publicly shaming their peers when they use such practices, but in private they continue to use them.

It all depends on the integrity of the researcher, which in turn depends on their upbringing (the example set by their academic advisor).

Relevant links to two podcast episodes about p-hacking:

- https://nulliusinverba.podbean.com/e/p-hacking-i/

- https://nulliusinverba.podbean.com/e/p-hacking-ii/

https://archive.ph/2026.03.25-072244/https://nautil.us/how-t...

At a certain point, the distinction between "fraud" and "not fraud" is a red herring. The downstream effects start to become similar enough.

If a scientist is doing more work to secure grants than doing science (my understanding is that this is very common), trying to justify their own existence, then I wouldn't be surprised that results get skewed towards that end.

If every software engineer and developer had to do more work justifying their own existence than actually coding and developing, I suspect overall software quality would be worse than it is today.

Heh, in support it would be kind of like metrics on numbers of tickets.

The first thing you game would be splitting any big work tickets in to smaller and smaller tickets. In some senses you should ensure any 'different' work on the same call is already split up, but typically you do this within reason.

After that you get into the bullshit number generating phase. Tickets for work not done (fraud imposed by unrealistic number requirements), tickets that have zero relevance or meaning (ticket stuffing). Tickets covering conversations or other things that shouldn't be a ticket.

You're not measuring what you actually want, you're measuring what you can measure, so people start producing what you can measure and not what you really want. Aka, Goodhart's Law.

Isn't the entire VC funding of modern software the exact thing you're pointing at? And the grifts that have occured because VC funders grivate towards justification of existence rather than anything else.

Isn't the rise of the AI companies that claim they can improve programming yet present such poorly debugged front ends, websites, etc, exactly the "software quality" problem you dont think exists?

I think introspection is required here.

> It would go on to become Halsman’s most iconic image.

Indeed!

https://arthur.io/img/art/jpg/00017344b99caa17a/philippe-hal...

The article misattributes the cause of the public's loss of trust in science.

The public has lost confidence in science because commercial and political entities have worked systematically to undermine scientific authority that threatens their business models and narratives.

Fossil fuels, tobacco, and refined sugar single-handedly manufactured decades of scientific "controversy" around climate change, tobacco's health impact, and the role of sugars and fats in obesity and heart disease. Religious fundamentalists fund pseudo-scientific books and articles attempting to muddy the waters concerning geological and biological evidence about any time period before the invention of agriculture. Grifters fabricate arguments and data to delegitimize Western medicine to sell placebos at high markup. Politicians attribute all unflattering research to partisan skullduggery.

It is true that hard science, social science, and medicine all have published flawed work; sometimes even in bad faith. These occasional failures -- which should be corrected and never tolerated -- are then used by the same people to indict the entire enterprise.

It's true that there's research misconduct, and it must be weeded out.

What's also true is that funding for research is declining in the US, especially relative to our need for it.

Note how many "tweaks" center on things like sample-size or statistical significance. Human-oriented research is always intractably complex, and we get the scientific outcomes we do because studies are perennially understaffed and under-resourced relative to the questions they seek to answer. Trying to answer fundamental questions about health and wellness with 30 people tracked over 6 weeks is fundamentally flawed, but it's the best we can do with the resources we have.

In a world where funding was plentiful, and career paths not cut-throat and perilous, there'd be far fewer examples of these kinds of "tweaks." People respond to incentives, and if the options are "add a few more participants, because after all the initial sample size was somewhat arbitrary" or "fail to publish, fail to graduate, fail to get a permanent position after a decade of post-secondary education because applicants must be perfect," the only surprise is that they aren't more common.

Reply on news.ycombinator.com