Comment by magic5227 | Hacker News Reader

Comment by magic5227 | Hacker News Reader

Notable comment in the post:

Bill Harris Sioux Falls, SD 56 minutes ago As a long-time researcher in omega-3s, I've watched the fish oil rollercoaster since the mid 1980s. This most recent posting by O'Connor continues the trend. The best meta-analysis (grand summary of many studies) published to date was from Rizos et al. in JAMA 2012;308:1024-1033). They concluded that fish oil capsules offer “no benefit” for heart patients. Unfortunately, Rizos used a highly controversial statistical maneuver. In their actual data (Fig 2) there was a highly statistically significant reduction in cardiac death associated with fish oil use (p<0.01 for the stat-saavy). So fish oils DID reduce risk for cardiac death. Why the "no effect" conclusion? Rizos et al. decided to set the statistical bar higher than I’ve ever seen it in meta-analyses. They defined a significant p-value as <0.006, instead of the universally accepted p<0.05. This trick changed a positive finding into a negative one and generated a media storm of "fish oils don't work." More recent meta-analyses (Chowdhury et al. Ann Intern Med 2014;160:398-406) reported that higher dietary intakes AND higher blood levels of omega-3 fatty acids were both significantly linked to reduced risk for heart disease. The problems with the recent fish oil studies are legion, and include using a low dose for a short period of time in older, already-ill patients who are also being treated with up to 5 heart medicines. In this setting it’s nearly impossible to show a benefit. With 0 risk, I still recommend fish oil.

Just a couple of notes. The "Rizos et al. in JAMA 2012;308:1024-1033" refers to the article:

Rizos E. C., Ntzani E. E., Bika E., Kostapanos M. S., Elisaf M. S. (2012). Association between omega-3 fatty acid supplementation and risk of major cardiovascular disease events. JAMA 308, 1024–1033 10.1001/2012.jama.11374

The article can be read for free here: http://jama.jamanetwork.com/article.aspx?articleid=1357266

The reason for using .006 is stated in the article:

"Within each assessed outcome, we adopted a level of statistical significance adjusted for multiple comparisons testing by a factor of 8 equaling the number of overall and subgroup analyses performed using the 2 measures of effect (RR and RD); thus, statistical significance was assumed at a P value threshold of .0063."

P-value adjustments are fairly common [1], but I'm not smart enough to say if it was applicable in this case.

Also, in the interest of full disclosure, it might be worth mentioning that Bill Harris is the President of OmegaQuant Analytics [2], whose stated purpose is to "advanc[e] the science and use of omega-3 fatty acids to improve health."

[1] http://stat.ethz.ch/R-manual/R-patched/library/stats/html/p....

[2] http://www.omegaquant.com/. He also seems to have a pre-canned response, which he pastes as comments to various sites. The pre-canned response is here: http://www.omega-3centre.com/images/stories/pdfs/harris_jama...

I checked the Rizos et al study (http://jama.jamanetwork.com/article.aspx?articleid=1357266) and the statistical issue is a bit subtle. They applied a multiple-hypothesis correction, to account for the fact that they were looking at multiple subgroups and endpoints. The problem is that in their data, most of the subgroups and endpoints show an effect, and these aren't being combined together. So in that paper, no one subgroup alone contains enough evidence to show an effect, but the groups put together, do.

The effects seen in the various sub-groups and endpoints are all non-significant at the p = 0.05 level. The lowest I can see is p = 0.07 (cardiac death prevention, as reported in their Table 3).

The multiple-hypothesis correction they apply is reasonably appropriate in the case of simply looking at all results, which is what they do.

Furthermore, because the outcomes are disjoint (cardiac survival vs stroke vs sudden death vs all-cause mortality, etc) there is no simple way to combine the results.

To take a silly example, studying the effect of seatbelts on cancer and cardiac death might well show a bit of an effect on both (because people who wear seatbelts are generally healthier, say) but it would be illegitimate to combine those two studies because the endpoints are (so far as we know) unrelated to each other. Without some kind of causal account the issues become very deep and difficult to say anything very definitive about.

So I'd say their statistical treatment is fair and appropriate. If fish-oil is supposed to have such a large effect as to be worth taking the risk that it increases the risk of prostate cancer, its effect should be unequivocally measureable in population studies. That is not the case.

For what it's worth, I think the prostate cancer studies are at least as flawed, at least the one I've seen, which is a case-control study that shows an extremely modest increase in relative risk of the kind it is very easy to produce from statistically identical populations: http://www.tjradcliffe.com/?p=1745

And also, the lack of Omega-3 fatty acids may have some important effects. Example: http://www.ergo-log.com/margarine.html

The most interesting part of this text is that 5% significance is "universally accepted" for medical meta-analyses.

The vast majority of medical researchers don't have a deep understanding of statistics. The ones that do understand statistics comply with the "universal expectations" of their editors and reviewers.

Reply on news.ycombinator.com