bmahmood

Hi this is Bilal from Science Exchange, and I help with the Reproducibility Initiative.

You are right that the per study costs are cheaper than normal as the original protocol is already developed, and there is less trial and error involved. In certain cases, we may not need to validate every aspect of a study as well, only those specific experiments that warrant replication. And depending on the study, those experiments may vary in cost (i.e. a mouse model component would be expensive, but RNA extraction and western blot testing relatively inexpensive).

Additionally, we will leverage expert labs on Science Exchange which perform experiments regularly on a contract basis, allowing for bulk savings on reagents and supplies.

This is Bilal from Science Exchange, and I help with the Reproducibility Intiative as well.

I think that in general you're right, with the assumption that p=0.05. However, each of the studies being replicated is composed of many intricate experiments. We will be reproducing most of those experiments, so while the odds of a single validation experiment failing are certainly not zero, I think the chances of an entire paper's worth of reproduction experiments being false negatives is much less likely. All of the protocols and reagents used, and the data from the replication studies, will be publicly available when they are complete, so that everyone can see what we did and how we did it.

"The training is nothing! The will is everything!" - Batman Begins

While not a whole solution, there were aspects of the 2012 JOBS Act that sought to relieve some of the provisions of SOX on small-cap companies.

http://en.wikipedia.org/wiki/Jumpstart_Our_Business_Startups...

Part of why there are fewer small-cap IPOs since 2000 has been due to the required external audits on companies - on average costing $2M. This had forced many companies to delay their IPO until they had the revenue to afford the audits, at which point often they no longer needed an IPO (due to folding, or selling).

The JOBS Act offered a series of exemptions for small-cap companies to be exempt from such audits, though I'm not sure what stage of implementation the SEC has taken it.

Neat ideas. I recall Dan Ariely of 'Predictably Irrational' noting that an effective method of exerting willpower was to mentally associate difficult / cumbersome tasks with positive rewarding ones. i.e. Watching a movie right after taking a medicine w/ many side-effects (so the medicine becomes associated with 'movie time' rather than 'side-effect' time.)

Ethan Perlstein, as Princeton research fellow is crowdfunding his research on amphetamines. So far, he's raised raised $18,246 from 258 donors, and needs $6,754 to complete the fundraising.

This is one of the first hard science crowdfunding campaigns I've seen, and I believe if successful, can be a testament to the power of crowdfunding to support scientific research in the future.

The researcher has promised to provide regular updates on his research in an open-notebook fashion, and the preliminary metrics of the crowdfunding campaign may represent the diligence with which he is already assessing his efforts.

We can help support the cause here: http://www.rockethub.com/projects/11106-crowdsourcing-discov...

Sorry, not sure why the title changed. Was originally 'Crowd Fund a Meth Lab', as noted in other media outlets.

For more information on the researcher's discipline, you can check out this neat video on 'Evolutionary Pharmacology': http://vimeo.com/44687770

Apologies, but that's the general headline the researcher is using to raise awareness for the initiative.

Can read more here: http://mashable.com/2012/10/25/rockethub-meth-lab/

It's true the issue of reproducibility in research is a complex one, with many underlying factors. Culture, misaligned incentives, and lack of resources play strong roles.

We believe however our Initiative can help in laying an initial framework for how one can possibly address aspects of the problem. We've tried to build incentives (ease of outsourcing, rewarding publications) that factor into this issue, and can assist in improving outcomes. But we definitely agree that a holistic solution will require further changes to the academic research infrastructure.

You're welcome! We completely acknowledge the difficulty in reproducing scientific research, acros disciplines. We are focused on preclinical biological research for now, but hopefully in the future we can expand to other areas like CS and Statistics.

Thank you for the support!

To clarify, the validation studies will be matched to core resource facilities and commercial research organizations, who specialize in conducting certain experiments on a fee for service basis. As they are paid upon completion of a service, regardless of outcome, we feel they are the solution to many of the misaligned incentives in academic research.

With respect to communication between the original authors and those conducting the validation, we definitely agree there needs to be some degree of communication, given the complexities of research. We will originally match a researcher to their provider in a blind fashion, so they have no choice in who conducts their study. But once a provider is selected, they can communicate with one another in explaining the methodology, experiments, etc.

Bilal from Science Exchange here. The validations are blind in the sense that we match an investigator who has submitted a study to a provider who can validate it. The investigator doesn't have a choice of who validates their study though.

The fee-for-service actually helps to guard against biased results in two ways. Firstly, providers who validate studies are paid on completion, regardless of a reproducible or irreproducible outcome. Secondly, operating on a fee-for-service basis allows providers to operate outside academic incentives for publications, ensuring no incentive for biased-positive results.

Bilal from Science Exchange here. Definitely agree that Plasmyd is a valuable service. We feel this Initiative operates in complement, providing a mechanism for independent validation.

This is Bilal from Science Exchange, and we greatly appreciate your support for our initiative!

We will provide the methodology of the original study to those reproducing the results, while the results we believe will be helpful to check against.

For conflicts, it's true we can't force an investigator to publish or note the lack of reproduced outcomes. We do hope they will through the PLOS Collection, for transparency. We do feel though it can provide a valuable check for 'failing fast', for those investigators who want robust results.

In this initial stage, we also agree that funding will be difficult. That's why we hope to focus on small biotechs and research labs that are interested in commercializing their research, and need to show robustness of results for licensing opportunities.

Hopefully then, it can serve as a proof-of-principle for funding agencies to provide a requirement or increase support for reproducibility.

Hi, this is Bilal from Science Exchange. We are in discussions with funding agencies and foundations to possibly gather increased financial support. We feel this initial launch will provide a proof-of-principle to gather support.

Currently though, we hope investigators will pay for the validation themselves. We believe it can provide a valuable service to those in small biotechs or those who want to commercialize their research, to improve the robustness of outcomes for potential licensing opportunities.

Hi, this is Bilal from Science Exchange. Appreciate your concern, but that is why we feel Science Exchange is uniquely positioned to make a difference. We have developed a network of 1000 core facilities and CROs on our platform, who operate on a fee-for-service basis to conduct experimental services. The Reproducibility Initiative will leverage this network to outsource specific experiments within a study to these facilities.

I think its possible to contend that our physical body is sufficient in providing a framework for our human consciousness to emerge, but it may not be necessary in maintaining it.

Think of a stroke patient. Often stroke patients lose their memories / personality temporarily, but then regain it some time later. The injured portion of the brain heals in some respect, but the memories/neural functions of the injured area often move / reorganize to a different location in the brain (which is why sometimes a stroke patient can feel sensation in say their hand when someone touches their face).

I think this comes to show that our consciousness/memories/etc are not necessarily tied to a specific source of physical matter, as even the brain can reorganize the "coded" functions to different areas inside it.

Ray Kurzweil has a nice point on this, calling human consciousness an "evolving pattern of information that persists through time."

Increasingly, pre-clinical studies to identify potential drug targets are conducted in academic settings.

Looks awesome. Really easy to use, and visually intuitive.

Agreed. I find it visually frustrating to read two columns simultaneously (going back/forth from left/right sides of the timeline). It's so cluttered I don't ever find myself visiting my own profile anymore.

About the only thing I use FB now for is the feed, but then Twitter serves the same function w/ better control over who I want to follow.

My family is of Pakistani origin, and I have lived in Pakistan for several years. I think discounting her talent, simply because she's from a "very rich Pakistani family" is unfair.

Yes, elitist families have more opportunities than the rest, but counter-intuitively given the feudal elite culture of Pakistan, the rich are often devoid of ambition and simply join family businesses. Girls, especially, are raised with no other ambition than to get married.

I think for this girl to have such greater ambitions, despite the rampant chauvinistic expectations of her society, despite a country where everyone (including the rich) have 5-10 hrs of electricity/gas/water a day, despite living in a country in political upheaval, is pretty exemplary. And her talent, as such, is impressive in context.

I think that's the problem he's pointing out though with medicine: that we're unable to account for all the side-effects, that our hypotheses are often wrong (in drug development, its practically like a lottery if you start at pre-clinical), and that these gambles on drug development are getting way to costly for the healthcare system.

I think there needs to be revision of the experimental method overall, as the models of drug development in the 20th century simply aren't sustainable or efficient for the diseases of the 21st century.

Agree that the author's extrapolations of the problems of medicine to science are cringe-worthy. The focus on medicine though makes me think it was just a bad editorial decision for the headline.

That said, I think he is correct in his critique of medicine/pharma. The cost of drug development has gone astronomically high these past decades, with billion dollar pipelines to account for the cost of failure.

The pharma drug development model has not really evolved beyond a lottery system of testing random compounds to treat diseases, and going back/forth until the right permutation of a compound is found. This might have worked before for initial "easy" diseases (that had easy drug targets, or single gene mutations), but the problems we face now (Alzheimers, Cancer) are too complex for our lottery-based drug development system.

I found Darksiders to be very similar to Zelda, based on all the points mentioned by the author.

Separately though, I agree Zelda holds a unique charm that's very difficult to evoke or copy (kind of like Pixar movies).

The article certainly identifies an important issue, but seems simplistic in it's assessment of the root causes. As a a bio major, who wavered between pre-med and business aspirations, my main gripe with science education was the theoretical framework it was taught in. It's mostly complete rote memorization of thousands of inane details, without any practical context to understand the relevance or application (no, labs do not fulfill this purpose).

If the implications of bio/chem theory were taught in a more practical, problem-solving approach, (a la computer science), I feel people would develop a better passion for science (as they did in High School with science fairs as the article describes), and stick with it longer.

Very true: "You have to stand up for yourself and sometimes that means being a bit more aggressive than you normally are. This is the way of the world."

I think people are blowing his words out of proportion. I found his comments were delivered casually, with a pretty straightforward message: he wasn't discounting what SV has to offer, he was simply stating it has more to offer to those who know don't know what they're doing.

He didn't know what he was doing when he founded FB (by his own admission), and so moving to SV was immensely helpful for him. He knows how to run a company now, and was simply stating that for those who live outside SV and know what they're doing, there's nothing intrinsically holding you back from success by locale.

I'm teaching myself to code right now, and have tried 3 different introductory courses/guides, including CodeAcademy. The problem I've faced with each one is I'd learn the same thing (methods, arrays, variables, conditional statements, etc) but was never able to piece together the practical use. I'd get what if/then clauses mean, and what arrays do, but never could grasp their broader functionality or use case in an actual program.

I recently started taking Stanford's CS106A class online, and its the only thing that's really clicked for me so far. The first few weeks teach you the same thing as other guides, but they create an introductory program for you to test out the basics, and help you learn the methodology of programming.

From there is where I realized what's lacking in most "learning-to-code" intro courses: they'll give you the tools to code, but don't help you learn the methodology of coding. I realize now its a very different mindset, and if you don't actually solve problems with those tools, its difficult to learn conceptually.

Hopefully CodeAcademy can crack this issue of helping people learn how to think like a programmer, and not just show the tools that programmers use essentially.