Comment by cwyers | Hacker News Reader

Comment by cwyers | Hacker News Reader

Parent Full thread

> Jeff Settleman, who left academia for industry 5 years ago and is now at Calico Life Sciences in South San Francisco, California, agrees. “You can't give me and Julia Child the same recipe and expect an equally good meal,” he says. Settleman has two papers being replicated.

Uh, Julia Child WROTE FREAKING COOKBOOKS. The entire point of Julia Child was that she tried to develop recipes in such a way that another cook could produce an equally good meal. Now, yes, if I went into a boiler room at Goldman Sachs and picked 10 guys at random, I doubt that most would be able to duplicate the recipe. If I picked 10 professional sous chefs at random and none of them were able to make a dish as good as Julia Child's from her recipe, I would start to have my doubts about the recipe.

By the same token, I don't expect rank amateurs to be able to duplicate state of the art cancer research. But if labs run by pharma companies and academic institutions are having the failure rate at reproducing research that the article claims, I think it's more than reasonable to start questioning the paper that documented that research, if not the research itself.

Analogy doesn't hold up. The complexity of cancer research is vastly greater than cooking. But anyway, chefs claim that there is special, non-reproducible elements about their environment and creations, e.g., oven, ingredients, etc. all the time.

> none of them were able to make a dish as good

Defining "good" is the problem for both cancer research and cooking. You can have two experts saying this is "good" or this is "bad" and not really be able to prove who is right. Fine, maybe for cancer research you can ultimately prove who is right but its not realistic in most cases given the resource constraints of even top-flight labs.

Well, it's not my analogy. But I think it's a good one, it just undermines the point of the person who made it.

If I go into Momofuko, I'm looking for a good meal. A good meal is one that tastes good, in this case. If I'm looking at a Julia Child cookbook, I'm looking for a good recipe. One of the criteria for a good recipe may be that it tastes good, or that it produces a healthy meal -- there's several different criteria you can use here. But one criteria for a good recipe is reproducibility -- in order for a recipe to be good it must contain enough information and be accurate enough for me to create the dish that the recipe is for. A recipe for a tasty meal that does not contain the right ingredients or enough detail in the steps to prepare it is a bad recipe.

By the same token, an experiment that cannot be repeated is a bad experiment. It may not be false. But its explanatory value is limited -- if a reaction can only take place in water that's treated a certain way or has/lacks certain minerals, then a paper that doesn't tell me that is leaving out important information. Regardless of whether or not you define the point of cancer research in purely scientific terms -- that is, to learn more about cancer -- or in more pragmatic terms -- that is, to allow us to create better cancer treatments -- omitted information about the circumstances surrounding the test that has a significant effect on the test result gives us less information and is less likely to lead to better cancer treatments.

I've been told altitude and atmospheric pressure has a measurable effect on recipes.

Reply on news.ycombinator.com