A modern metaphor with a somewhat similar meaning to the original is: "A fish rots from the head down." Pointing out that organizational failures are usually the result of bad leadership. A rotten leadership will quickly result in a rotten organization. Therefore, it is important to make sure the leadership is not rotten in a organization. It also points out that low-level failures indicate there are deeper high-level failures. If the line-level is screwed up, the leadership is almost certainly just as screwed up. The fix being replacing the rotten leadership with a new one as lower-level fixes will not fix the rotten head.
Another, more direct equivalent metaphor is a Chinese saying translated as: "One piece of rat poop spoils the pot of soup." That is hopefully self-explanatory. We should probably use it instead of "a few bad apples" as nobody will reverse the meaning of that one.
In this case, the "a few bad apples are not representative of a group" meaning have grown above the "One bad apple spoils the barrel" meaning, and so the phrase as changed, for better or worse.
Maybe it would be best if everyone used the long version instead of the short one. When you say/write "A few bad apples", the meaning is ambiguous, but if you use the long version, it's not. Problem solved :)
Most of the time when I hear the "only a few bad apples, the rest of us are fine" meaning it's coming right from the mouths of badly spoiled apples twisting the meaning of those words and popularizing that usage to suit their agendas.
Generally, I think that there's nothing wrong with pushing back against words and phrases used incorrectly. We get to decide how words are used, and a large part of that decision making process involves social pressure and education. I think it's particularly useful to defend the meaning of words and phrases when they're being deceptively misused and promoted.
I think it's right to correct it because when people misuse this phrase, it isn't gaining a new meaning--it's making it meaningless. Why apples? The comparison to apples adds no information or nuance.
Like when there's a story about police corruption, and someone says "they're just a few bad apples, not all cops are bad." Again, why compare them to apples? Why not just say a few bad cops?
This isn't words/phrases changing meaning, it's losing meaning.
It is literally not, it still means something, just not the same as it originally meant. This happens all the time, with "literally" being one of the best examples of something that literally means the opposite of what it used to mean.
I have never heard that phrase, it has always been that the few spoils the whole.
I have heard people say "the proof is in the pudding", which means nothing at all, when the real phrase is "The proof of the pudding is in the tasting".
I'm from England and I speak English, so maybe it hasn't translated well to Americlish.
"the proof is in the pudding" is a much more benign change. It's literally just a shortening, but no meaning is lost... if you want the proof, you'll find it in the pudding (implying you should try the pudding to verify your assumptions)
Too few people have enough apple trees in their lives to preserve the meaning.
To be fair, the "real" phrase you give here doesn't make much more sense to me. Even assuming the use of the term "pudding" across the pond to be more than just a fairly niche dessert like it is in America, what does it mean for pudding to have "proof"? Is is some sort of philosophical thing where you don't accept that the pudding exists unless you taste it (which I feel isn't super convincing, since if we're going to have a discussion, we kind of have to accept that each other exists without having similar first-hand "proof", so we might as well accept that pudding exists as well)? I know there's a concept of something called "proofing" in baking, but I'm pretty sure that happens long before people taste the final product.
In general, I don't find most cliches to be particularly profound. "It is what it is" is just a weird way to state an obvious tautology, but somehow it's supposed to convince me that I should just passively accept whatever bad thing is happening? "You can’t teach an old dog new tricks" isn't universally true, but it apparently also is supposed to be a convincing argument in favor of inaction. "You can’t have your cake and eat it too" is probably the most annoying to me, because the only way anyone ever wants to "have" cake is by eating it; no one actually struggles to decide between eating their cake or keeping it around as a decoration or whatever.
There's something about stating something vaguely or ambiguously that seems to make it resonate with people as profound, and I've never been able to understand it. In my experience, thought-terminating cliches are by far the most common kind.
A vaguely similar sentiment to when people say “eating your own dog food” (or words to that effect) to mean testing something by using it themselves. Albeit the pudding proverb doesn’t necessitate the prover to be one’s self like “dog fooding” does.
The original saying, of course, is all about why you have to make a big deal out of this, for reasons that apply to both apples and cops.
> The phrase originally meant that one bad thing corrupts and destroys all associated.
It's saying if you don't remove the bad apple, you will get a lot of bad apples in the future. The presence of a bad apple doesn't imply all of them are already spoiled right now. If you're seeing other apples that still haven't spoiled, it suggests you still have time to do damage control. That seems consistent with how the metaphor is used nowadays.
But the larger issue here is that all public institutions are, by that definition, broken. For example, businesses also won't hesitate to spread falsehoods to sell their stuff, governments will try to convince their people that they are needed through propaganda and policing, and so on.
How do we solve these problems? We have laws to regulate what businesses can't do (nevermind lobbying), and we split governments' responsibility so that no single branch becomes too powerful. In general, we have several independent institutions that keep an eye on each other.
In case of science, we trust other scientists to replicate and confirm previous findings. It is a self-correcting mechanism, whereby sloppy or fraudulent research is eventually singled-out, as it happened in this and many other cases.
So I guess the gist of what I want to say is that you're right in not trusting a piece of research just because it was made by a reputable institute, but look for solid results that were replicated by independent researchers (and the gold standard here is replication, not peer review)
They do hesitate. It's quite hard to catch businesses openly lying about their own products because, as you observe, there are so many systems and institutions out there trying to get them. Regulators but also lawyers (class action + ambulance-chasers), politicians, journalists, activists, consumer research people. Also you can criticize companies all day and not get banned from social media.
A good example of what happens when someone forgets this is Elizabeth Holmes. Exposed by a journalist, prosecuted, jailed.
Public institutions are quite well insulated in comparison. Journalists virtually never investigate them, preferring to take their word as gospel. There are few crimes on the book that can jail them regardless of what they say or do, they are often allowed to investigate themselves, criticism is often branded misinformation and then banned, and many people automatically discard any accusation of malfeasance on the assumption that as the institutions claim to be non-profit, corruption is nearly impossible.
> It is a self-correcting mechanism, whereby sloppy or fraudulent research is eventually singled-out, as it happened in this and many other cases.
It's not self correcting sadly, far from it. If it were self-correcting then the Stanford President's fraud would have been exposed by other scientists years ago, it wouldn't be so easy to find examples of it and we wouldn't see editors of famous journals estimate that half or more of their research is bad. In practice cases where there are consequences are the exception rather than the norm, it's usually found by highly patient outsiders and it almost always takes years of effort by them to get anywhere. Even then the default expected outcome is nothing. Bear in mind that there had been many attempts to flag fraud at the MTL labs before and he had simply ignored them without consequence.
Another way to look at it: perhaps Tessier-Lavigne only got this scrutiny because he was president of the university. And the fact that they didn't guarantee anonymity when "not guaranteeing anonymity in an investigation of this importance is an 'extremely unusual move'" might be a sign that the scrutiny was politically diminished.
So it could be that most of the equally dubious researchers don't get caught because not enough attention is paid to patterns like this except when it's somebody especially prominent. Or it could be that this one was not as well covered up, perhaps because of the sheer number of issues. Or that the cross-institution issues made Stanford more willing to note the wrongdoing. Or that Stanford is less likely to sweep things under the rug because of its prominence. Or just that there was some ongoing tension between the trustees and the president and that this was an opportunity to win a political fight.
Grad students, Supreme Court clerks, 19-year-old soldiers.
> A 2016 survey by Nature on 1,576 researchers who took a brief online questionnaire on reproducibility found that more than 70% of researchers have tried and failed to reproduce another scientist's experiment results (including 87% of chemists, 77% of biologists, 69% of physicists and engineers, 67% of medical researchers, 64% of earth and environmental scientists, and 62% of all others), and more than half have failed to reproduce their own experiments. But fewer than 20% had been contacted by another researcher unable to reproduce their work. The survey found that fewer than 31% of researchers believe that failure to reproduce results means that the original result is probably wrong, although 52% agree that a significant replication crisis exists. Most researchers said they still trust the published literature
Not sure if the results of that online study have (or can) themselves be reproduced, however. It's turtles all the way down.
A success story would be this is exposed at large... we work out some kind of effective peer-reproduced tests... and the hundreds/thousands of cheating professors are fired.
FWIW, most top-ranked CS conferences have an artifact evaluation track, and it doesn't look good if you submit an experimental paper and don't go through the artifact evaluation process. Things are certainly changing in CS, at least on the experimental side.
It's also possible that theorems are incorrect, but subsequent work that figures this out will comment on it and fix it.
The scientific record is self-correcting, and fraud / bullshit does get caught out.
I might agree they're self-correcting in the (very) long-term, but we're seeing fictitious results fund entire careers. We don't know the damage that having 20+ years of incorrect results being built upon will have... And that's not to speak of those who were overlooked, and left academia, because their opportunities were taken by these cheaters (who knows what cost that has for society).
Study claims 1 in 4 cancer research papers contains faked data https://arstechnica.com/science/2015/06/study-claims-1-in-4-...
As an aside, the article cites that there have been a total of 2000 retracted papers in the NIH database. Considering that there are 9 Million papers in the database overall, that is a tiny percentage.
So you deflect from the entire content of the article with that distraction? And then an additional misdirection regarding retraction? Why?
> So you deflect from the entire content of the article with that distraction? And then an additional misdirection regarding retraction? Why?
What do you mean? I take issue with the headlines and reporting. And I believe if one claims lack of evidence, sloppy evidence or fraudulent evidence one should be pretty diligent about ones one evidence.
Regarding the claims in the article. If you look at the 1 in 4 article you find that the reality is actually much more nuanced, which is exactly my point. The reporting does not necessarily reflect the reality.
If you call that deflection...
One nuance is that his approach only focused on one easily identifiable form of fraud: Western blot images that can be shown to be fraudulent because they were copies of images use in different papers. Of all the potential opportunities for fraud, one must think that this must represent just a small portion.
If there are other nuances you care to mention, I'm all ears.
Instead, you refer to an entirely different article, as if the article I cited has no relevant content, which misleads casual readers of this comment stream. To paraphrase your comment in a less misleading way: "Inside this article you can find a link to an entirely different article whose content does not support the headline of the original article."
Instead, you're repeatedly citing this 25% number as if it was something the paper established. Even the author of the paper disagrees with you.
Yes, the ArsTechnica headline is poorly written, and not supported by the content of the article, because not all instances of duplication are fraud, but we can clarify that issue by quoting the article itself: "... the fact that it's closer to one in eight should still be troubling."
Think of Tour de France. Famously doping-riddled. There are a lot of clean cyclists, but they are much less likely to be able to compete in the tour.
You can fight cheating with policing: doping controls, etc. But as the competition gets more extreme, the more resources you need to spend on policing. There's a breaking point, where what you need to spend on policing exceeds what you get from competition.
This is why almost no municipalities have a free-for-all policy for taxis. There are too many people technically able to drive people for money. All that competition drives prices lower, sure, but asymptotically. You get less and less lower prices the more competition you pile on - but the incentives for taxi drivers to cheat (by evading taxes, doing money laundering as a side gig etc.) keep growing. London did an interesting thing - with their gruelling geography knowledge exam, they tried to use all that competitive energy to buy something other than marginally lower prices. Still incentive to cheat, of course, but catching cheaters on an exam is probably cheaper and easier than catching cheaters in the economy.
(Municipalities that auction taxi permits get to keep most of the bad incentives, without the advantage of competition on price.)
I don't expect 1-5% fraud in airline pilots, bank tellers, grocery store clerks, judges, structural engineers, restaurant chefs, or even cops (they can be assholes but you don't have to bribe them in functional countries).
I think academics can do better than 1-5% fraudulent.
Store clerks, theft is about 1-2% of sales typically. It has been said for years that the majority of that theft is from employees. Airline pilots have been known to drink during their flights (or go away from there seat for other reasons that are not in the rules).
Cops, I mean don't get me started, just the protection of a cop who has done something wrong by the other cops would count as fraudulent, but I don't see many cops going after their own black sheep.
Judges, in Germany deals (i.e. the accused pleads guilty to lesser charges so the bigger ones get dropped) are only legal under very limited circumstances (almost never and need to be properly documented). Nevertheless, in studies >80% of lawyers reported that they had encountered these deals).
I think you seriously underestimate the amount of fraudulent behaviour.
Perhaps "nothing" is too harsh. Various people in the field raised concerns on several occasions. But the journals did nothing. The "field" still honoured him. And _Stanford_ did nothing (except enable him and pay him well) until public embarrassment revealed the ugliness.
E.g., in math / theory, if someone has a history of making big blunders that invalidate their results, you will be very hesitant to accept results from a new paper they put on arXiv until your community has vetted the result.
So yes, I do trumpet science as a model of a rational, self-correcting social enterprise, at least in CS.
Other sciences like biology and psychology have some way to go.
Forcing authors to share raw data and code in all papers would already be a start. I don't know why top impact factor papers don't do this already.
Forgive my ignorance but I thought that was SOP for all papers. Is it not?
But really we shouldn't be talking about math errors and falsification in the same category.
The result is that research in many fields tends to stagnate and reinforce old ideas, regardless of whether they are right or wrong.
Giving up on science is such a vast overgeneralization. You could take your statement and replace "manipulated research", "scientific institutions" and "established scientific truth" with just about any negative article in any domain. You could just as easily make this statement about startups (Theranos, Juicero), or government, or religion, or suburbs, or cities...
Yes.
> Giving up on science is such a vast overgeneralization. You could take your statement and replace "manipulated research", "scientific institutions" and "established scientific truth" with just about any negative article in any domain. You could just as easily make this statement about startups (Theranos, Juicero), or government, or religion, or suburbs, or cities...
Institutions go through similar cycles of breaking and systemic reform. Not surprised that you can see patterns in other domains.
Look at papers that have real impact they get cited. Look at ones that don’t …
IME some of the more effective engineers I've worked with have gravitated towards politics, not "raw technical skill". It's not because they prefer it. They use their "smartness" to win.
The problem is that being good at social engineering >> anything else. Intelligent people often look at the system and say: "What's the point in naively following this when no one successful unless they game the system?"
What's the point of putting one of your great, well considered idea into the fold. It's far more effective the be a mediocre huckster. You don't have to deal with the uncertainty, giving your idea to someone else, etc. Better to work the social game and phone in the rest.
It works better and you don't have to deal with the crushing disappointment that goes with fighting for an idea in a horrifying bureaucracy.
https://www.technologyreview.com/2020/11/12/1011944/artifici...
https://reproducible.cs.princeton.edu/
https://ui.adsabs.harvard.edu/abs/2018Sci...359..725H/abstra...
And what happens when it is caught, it is just quietly ignored by the field, right? How often are there retractions?
It's frankly not worth the effort to debunk a shitty piece of researchin a low-profile journal that's never been cited in a decade.
> ...26% of the papers had problems that were so widespread that the trial was impossible to trust, he judged — either because the authors were incompetent, or because they had faked the data.
This is not a one off.
Assuming that they did a proper sample of said papers, that implies that for whatever domain they sampled, 26% is likely a decent estimate of actual issues. Increasing the scale doesn't make a proportional estimate any better.
Of course misconduct is unavoidable, that doesn't mean you should become president. The politics aren't working.
It's clear by now this isn't the case of a few bad apples - our scientific institutions are systemically broken in ways that promote spreading fraudulent results as established scientific truth
This is a concern about the corrupted institutions, with the downstream concern that science itself may be under threat. The primary concern is the systemically broken institutions who promote the fraudulent to the top of their hierarchies. Not sure why you insist on straw manning this thing, but clearly you have some person reason for doing so, and I wish you luck in that endeavor.
https://news.ycombinator.com/newsguidelines.html
If you want to argue this further, you should probably snipe the swipes off the end of your comment.
Considering that we get fraud every time we have humans and prestige money, I would really like to see some statistics against other things human activities. I suspect science still has some of the lowest fraud rates and the strongest mechanisms to detect and deal with it.
People know about police abuse. We don't talk about doctor abuse. I'm honestly not confident that there's any police/doctors that don't engage in abusive practices (or it's a tiny percentage of the population).
The decay has been super fast though. Maybe some academics will find the courage to do a longitudinal study of this decay. Now that'll be an interesting paper to read.
The fact that a Stanford president can be pushed out for bad research conducted before he was even there? It tells me there’s still some integrity left.
I personally believe this is an underestimate.
We do get unbelievable findings such as CRISPR, but beware these will be very few and far between.
Scientific consensus is still very reliable and if 95% of accredited scientists in a field say something is true it is in society's best interest to consider that to be the truth.
The point here is to save the good apples - not throw out the whole barrel for zero gain.
See this recently published article https://www.nature.com/articles/d41586-023-02299-w.
One would think that clinical trials would be documented and scrutinized out the yin-yang but they are not.
Bad things are gonna happen in every single institution ever created. A better measure is how long those things persist.
Science is about getting closer to "the truth". Sometimes science goes further away from the truth, sometimes it gets closer. Sometimes bad actors get us further away from the truth. It gets reconciled eventually.
I completely agree. Seeing slack chats and emails regarding Proximal Origin and how researchers were disagreeing with it all the way until they published a paper that served what purpose are really disheartening. Instead of guiding future research toward preventing similar outcomes, countless scientists spent untold years of combined effort on a theory the authors didn’t even believe.
Just the fact that Stanford managed to conduct an independent investigation against its OWN PRESIDENT, tells very positive things about the University.
After this episode, I might trust Stanford research even a bit more than any University that never caught fraudsters.
Every time a plane crashes it's international news. But just because you regularly hear about plane crashes doesn't mean flying is unsafe.