Hacker News Reader

"The Atheist post seems to imply that they only sent 1 package to Michigan."

"In my experience, more people lie or are mistaken about their package delivery than packages actually being lost."

Read more carefully: they sent exactly two packages to each person, with the only difference being the tape. This isn't affected by some random customer complaining about a package being late or lost; these are people who are each reporting the arrival times of two packages.

"So 178 items sounds like a ridiculously limited amount of shipments to consider."

If you agree that each of the 89 pairs of packages are more or less independent, then we can use p-values to determine whether the results are significant based on the sample size. They are.

There are still plenty of systematic effects that could have affected the study by invalidating the assumption of independence between the 89 "trials." For example, if the customers all knew the purpose of the study, some of them may have intentionally mis-reported their result. So I agree that you can't reach a firm conclusion based on this one study. I just wanted to point out that many of the objections you raised don't seem to apply, and that this is a pretty interesting result not to be dismissed so lightly.

I see what you're saying, for the most part, and I definitely missed the fact that each customer was sent one of each. (I'm going to claim that I was distracted by the pictures :)

I'm still having a hard time viewing it as proof of a religious bias. There's still a number of conclusions that could be drawn that aren't that... The decorative tape might be harder for automated postage-processing machines to deal with. The decorative tape might be less sticky than the non-decorative tape (they make no mention of that). I'm not sure how they delivered this stuff to the Post Office, but if the packages were presorted into a group of decorative tape boxes and a group of plain tape boxes, that could be the _entire_ reason one type was lost/delayed more than the other (if a whole bin or a pallet or whatever had an issue).

It's only 9 packages that went missing, too. So if 5 of those 9 went to one city, 3 of the 9 to another, and 1 of the 9 to a third city, then the conclusion to draw is that those 3 cities had issue with the decorative tape for some reason. Not that the entire USPS is religiously biased. If each loss were stated to be in a separate (and significantly different) city, that would be closer to the stated conclusion.

Finally, at the bottom in the small print (I read it a bit more carefully this time), you see that they were shipped via Deutsch Post then handed over to USPS when they reached the US. (That's pretty standard for how international shipping works.) Without tracking, how can they lay blame for lost/delayed packages solely on the USPS? And even if they had tracking only up until the packages left the country (as is how some international shipments work), there are still factors outside of the USPS that could affect this, for instance maybe someone at the Deutsche Post decided for fun (or maybe even some sort of processing reason) to sort all/some of the 178 packages. Then imagine that a single bin of packages went missing and it just happened to be one of the bins of decoratively-taped packages... That could mean one act of incompetence (or even maliciousness) is deciding the entire outcome of the study.

Really though, the fact that they're concluding this based on 10 lost packages still strikes me as odd. An issue with just 10 packages is so well below the levels of extreme incompetence that I've encountered with the Post Office in my years that claiming religious bias feels like giving the system far too much credit.

Stats below:

If there is no bias, the lost packages reports would have equal probability of having come from the athiest group or the non-athiest group.

The (frequentist) probability that the ratio would be 9:1 or worse (given 10 lost packages) is 10/(2^10) which is approximately 0.98%. If you're the type to care about p-values, here we have p < 0.01.

Suppose there indeed exists this bias as it appears (9 times more likely to lose athiest package), we might be curious to ask: how likely is it that we would have detected a statistically significant (p < 0.05) bias with just 10 packages lost total?

Doing a similar calculation as above can check that we only see a statistical significance bias if the loss discrepancy is (10:0, 9:1, 1:9, or 0:10). Now if we suppose the bias is 9:1, how likely are we to have seen one of those events? If we add up the probability of these events (10:0, 9:1, 1:9, or 0:10) occurring under a biased scenario, we get about a 74% chance of us detecting the bias.

A standard requirement is for this number to be at least 80%, our result is close but would like to see more data before publishing in the Journal of Atheist Footwear.

The latter metric is called statistical power or the probability of not getting a false negative, whereas p-value is the probability of a false positive.

I'm glad you ran this analysis. I was about to do something similar!

I appreciate the statistics even though I don't really understand them all that well. Since you seem fairly knowledgable of statistics (though I know a lot of people here are too, so other chime in too if you want), could a case be made that we shouldn't be comparing 9 lost decorative tape packages out of 178 with 1 lost non-decorative tape package out of 178, but instead be comparing 9 directly with 1? When 10 packages

And if you're curious, please see my other reply as to why I think, even including statistical significance of one type being lost more than the other, we can't really draw a "religious bias" conclusion.

Yep, the numbers I ran are already doing exactly what you suggest. Looking exclusively at the 10 lost packages and considering how likely it should be that 9 of them would belong to one category, compared to our expectation (which is that they would be equally likely to have come from either category, coming from the fact that the same number of packages were shipped with Atheist labeling as without).

The statistics is entirely identical to flipping a coin 10 times and asking the following questions: * Given that I saw 9 heads and 1 tail, how likely is it that the coin is a fair coin. This is the p-value which was about 0.0098 (less than 1% chance) * Given that we are only tossing 10 coins, how likely are we to have seen a statistically significant bias with the predicted bias in place. This is the statistical power, which was 74%.

As you suggest in your other comment, the stats only point to a bias, the origin of the bias is left for people to decide. That said, the packages were sent in pairs, so subdividing the country into states and offices is irrelevant since a given states saw the same number of packages from each category as did each given office, the statistics are saying that on average (across the whole country) there is a bias.

To ease your worries a little, this is not saying the USPS hates Atheist packages 9x more than non-Atheist packages, most of the packages were delivered successfully. All this is saying is that, for the ones that were lost, it is very likely that they were lost because of their labeling. This is the effect you would expect to see if 97.3% of USPS employees treated the packages identically and the other 2.7% have a strong bias.

If we had more specifics about the delayed packages as well, we could get a more nuanced picture, I suspect we would find that 80+% of USPS employees are unbiased, some 10-15% might have some bias against Atheist labeling (would delay an Atheist labeled package), and 2-3% have a strong bias against Atheist labeling (would lose an Atheist labeled package).

If I was at USPS, I would push to conduct an experiment to try to identify which offices were most likely to have some bias one way or another (not necessarily specifically about Atheism, one could test whatever they wanted presumably).

I'm sure we're quite a few years away from the possibility of a POTUS admitting to being an atheist.

My thoughts exactly. We've almost certainly had closet atheist presidents before, and I highly doubt Trump believes in God. I have my suspicions about Obama and the Clintons as well.

True. This is really what I meant.

Why would the packaging tape make recipient lie (or err) more about not recieving a package? And if there's no conceivable reason then why have you brought it up?

I know your point is that USPS has high variance but I think statistical tests should account for this.

I definitely missed that each customer was sent one of each. Sorry. Really embarrassing considering it was first paragraph of the picture-y part.

This was a randomized experiment with results that were significant with p<0.001. What more do you want? 178 is more than large enough a sample size for an effect size this strong.

Please see my response here, if you're curious: https://news.ycombinator.com/item?id=12610428

I am addressing your original objection, that the sample size isn't large enough. The sample size is more than large enough. That comment you link raises entirely new and separate issues.

It's true they didn't say whether or not they properly randomized the boxes. But I assume they were competent enough to do that, and they certainly don't say that they didn't. They did say they had no issue in Europe and that the problems were entirely from the US. I really doubt it's with the quality of the tape or the automated systems, using a different kind of type shouldn't make such a huge difference.

It really seems like you are just looking for any reason you can think of to disregard the results. Religious bias is a huge problem in the US. Their hypothesis is totally reasonable. And their experiment, while not perfect, provides evidence for that hypothesis.

I'm not "looking for any reason [I] can think of to disregard the results". I'm just saying that based on the fact we were given it's naive to jump to the "religious bias" conclusion that they (and other people) have jumped to.

Additionally, based on my experience with the USPS, it's far more likely to be attributed to negligence/incompetence than religious bias.

Finally, I'm not denying there isn't or can't be a religious bias in the USPS, and such a bias may very well be what's causing this company's problems. But this "study" is far from conclusive, and fails to even acknowledge the possibility of a different conclusion.

Yo would love to talk shot re: shipping. Hit me up at alex@monthlyboxer.com