Hacker News Reader

Comment by rjmunro | Hacker News Reader

Reading this, I don't get how you can take all "nudging" and declare "No evidence". Surely "nudging" encompasses a whole range of different actual actions. Some nudges work, some don't. You can't just average across all of them.

I'm probably totally misunderstanding, but it sounds similar to saying "there is no evidence for medicine" because you've averaged all the papers describing medical interventions that work and those that don't.

I thought the point of "nudges" is that they are so cheap to implement you can easily afford to try many. Most won't work, some will.

View on HN

This is my interpretation as well. Also weird to think about publication bias in this way: "these studies about the effectiveness of snake oil as a drug weren't published, so we must be overestimating how effective drugs are".

The authors do mention that there is likely to be heterogeneity in (real) effect sizes, but somehow still go with this title/abstract.

Maybe there is a valid conclusion that some of the many nudge studies are probably claiming effects that don't exist. That could be interesting in itself. But rejecting the whole field based on this kind of argument seems wrong.

> but somehow still go with this title/abstract.

While it might be to catch the reader, I don't think it is wrong. The large problem is that we had policies implemented to nudge people in certain directions. Apart from the ethical question there needs to be hard evidence before we employ authoritarianism like that. So the headline should be pardoned, but not those that employ nudging for the time being.

I have the same concern. Nudging is an umbrella term for a vast number of very different activities. For example, nudging is a term used for motivating more carefully designed road markings. I find it hard to believe that some of these newer designs don't "work" better than the old ones, some of them are quite ingenious. At the same time "nudging" is also used for all sorts of public policy framing issues that are more questionable and have probably harder to measure effects. As you say, each "nudge" needs to be evaluated individually.

Agree with you. Nudging is a type of user experience design. UX designers nudge with every design decision they make, and the effectiveness of those decisions is quantifiable. So it's hard to argue that all nudges are ineffective, just like one can't argue that all UX design is ineffective.

There are a number of posts that address this issue, as well as issues raised by responses to this post. I posted somewhere "no true Scotsman"--I'm not claiming my post was enlightening in itself, but the post I was responded to (and the entire thread) was, IMO, enlightening.

As for averaging, yes, you can: if a nudge is ineffective, then its result will be random, and a bunch of ineffective nudges will average zero effectiveness. The effective nudges will then push the overall average above zero. We don't see that. (The same would be true for medical interventions, unless some cause harm.)

As for being able to try lots of them: in some circumstances, maybe. But when a government is trying to nudge people towards some desired behavior (vaccination, say, to take a random example), they don't try sending out a bunch to different groups of people, then polling each of those groups to see which groups--and therefore which nudges--moved. And it's not always practical, anyway (and the vaccination example is a case where it's almost certainly not practical).

See also threads that mention A/B testing.

> if a nudge is ineffective, then its result will be random

Ineffective and random are different. Ineffective means that the effect size is smaller than required.

For example, if you read "Paracetamol was ineffective for pain relief after surgery" it doesn't necessarily mean that the effect of paracetamol is unpredictable, inconsistent, immeasurable, or that it had no effect or negative effect. You would most likely interpret it to mean that the paracetamol did have an effect but it was insufficient - the patient was still in too much pain.

Similarly, if a nudge intervention was ineffective, it doesn't tell you how it failed, only that it didn't reach the threshold for success. And it certainly doesn't tell you anything about how well an aggregate of some effective and some ineffective results would perform.

Reply on news.ycombinator.com