A Letter I Will Probably Send To The FDA on 23andMe
slatestarcodex.com
slatestarcodex.com
As they should. If they can't show that the associations they're reporting are valid, what purpose does it serve to report them? That's the whole point. It's not like drug regulation at all, because most of this stuff isn't settled science (and hey, not for nothing...we do test the effectiveness of the antibiotics that get sold at your local pharmacy. A doctor should know that.)
Tell you what: send me your saliva and $80, and I'll write you up a really nice genetic report. It'll tell you all sorts of great stuff, and it'll be $20 cheaper than 23andMe. But don't try to force me to certify that I'm not making everything up -- that would be stifling my innovation! After all, I'm providing a cheap service. Disruption!
For a community that goes absolutely crazy with skepticism about every scientific study (i.e. "correlation! not cause!"), there sure seems to be a lot of credulity to go around on this issue.
First, 23andMe isn't working in the realm of established science -- most of this stuff is highly speculative, at best. Blood tests are not speculative precisely because they're regulated by the process that you're arguing 23andMe should be able to ignore.
Second, the "your doctor shouldn't have to run the study" argument is the reddest of herrings. Your doctor doesn't have to do that (anymore...) because the regulatory system has provided a network of providers who do it more reliably.
Third, your doctor really does have to know how to interpret the results of your blood test. She can do that because the science is well-established, and the results are directly correlated to certain clinical outcomes. For most of this genetic testing, there's no equivalent interpretation. That's why you need genetic counselors, who know enough to know when an association study is total garbage. Or if a "50% change in risk" is meaningless in a larger context.
In short, it isn't especially onerous to want 23andMe to report only those genetic variations that are based on clinically relevant data, for which their tests are accurate and precise. That's just the basic standard for medical testing.
[1]http://www.cms.gov/Regulations-and-Guidance/Legislation/CLIA...
Again, most of this stuff is sketchy science. You can do the microarray experiments in a proper (e.g. "CLIA certified") lab, and still have a useless, dangerously misleading test.
Showing correlations and study information related to markers found in your genes doesn't serve a purpose?
Shouldn't that be up to people to decide with their $99?
Tell me that this information and their method of finding correlations isn't doesn't have huge potential benefits: https://www.23andme.com/about/factoids/
The point is, like all other medical tests before them, they need to go through the proper regulatory channels to ensure their safety and validity. It decidedly should not be "up to the people to decide" because we know that society at-large is not informed enough to make these decisions.
What makes 23andme different from all of the other medical tests and technology out there?!
So we're smart enough to vote for our overlords and trust that they'll put in FDA bureaucrats who have our best interests at heart and will watch out for us -- but we're not smart enough to receive information on the genes in our bodies. Besides the lack of logic in such a system of decisionmaking... besides the lack of freedom of such a system of information control... the system you seem to want is a self-fulfilling prophecy in that the more people are shielded from the black art of medicine, the more that ignorance will grow as information sources are kept from the masses.
What makes 23andme different from all of the other medical tests and technology out there
The fact that they're not marketed as a clinical tool for diagnoses? The fact that they are not marketing themselves as a substitute for seeing your doctor and in fact encourage you to seek out your doctor if you have real health concerns?
From the 23andme homepage: "A simple DNA test led to an unexpected diagnosis for a mother and daughter, changing their lives forever.". Obviously the key is the "led to" phrasing, but you can't deny how that wording is trying to position the product.
The fact that they are not marketing themselves as a substitute for seeing your doctor and in fact encourage you to seek out your doctor if you have real health concerns?
As I mentioned in another 23andme thread, basic take-home pregnancy tests are regulated by the FDA. Certainly, when one of those tells you that you are pregnant, your actions don't stop there and you will eventually follow up with a medical professional. What differentiates 23andme's genetic testing from a pregnancy test?
(on a side-note, I would appreciate an explanation for the down-vote on my earlier post in this thread. Are we supposed to down-vote people we disagree with, or was there something wrong with my wording?)
I'd say that it's the specificity of the purpose of the test and the scientific understanding of what is being confirmed. Women go to the drugstore to buy a pregnancy test because they have a REASONABLE EXPECTATION of learning whether or not they're pregnant. Personally I wish the FDA would get out of the business of validating each pregnancy test product on the market. I'd prefer that something like the UL be used instead.
23AM is doing something much different. They're allowing you to have markers identified in your DNA for a very low cost and then they link those markers to labels and information sources as well as tying them in with their own correlations to other 23AM users.
It's all very unspecific and bleeding edge. It's not about proving every little correlation and its interpretation - it's about providing that information to users and letting them make their own decisions.
What amazes me is that here on HN where we want the source to every driver, the schematic to every chip, and the right to know every secret of the government related to our personal information -- there are those who are arguing that we should not be allowed to have data pulled straight out of our genome and correlated with the data from other users unless fully sanctioned and sanitized by a government agency that is woefully incapable of operating in a fast-moving environment where uncertainty is okay and a REASONABLE EXPECTATION.
I would appreciate an explanation for the down-vote on my earlier post in this thread
Meh, I got downvoted earlier too... I try not to worry about it since Karma means nothing. I'm more interested in learning new things from others and clarifying my own personal philosophy while practicing expressing myself.
First, you can't substitute UL for the FDA, they have very different mandates. However, with deep enough pockets, UL will hand-hold you through the FDA process. With the kind of funding 23andme has had, following FDA guidelines would not have been prohibitive.
Second, pregnancy tests have been around for a relatively long time and are well-understood and established. The point is, if these are still under regulatory control, how can one argue that something much less established, understood, and developed, in terms of a take-home medical test should not be subject to the same rigor.
What amazes me is that here on HN where we want the source to every driver, the schematic to every chip, and the right to know every secret of the government related to our personal information -- there are those who are arguing that we should not be allowed to have data pulled straight out of our genome and correlated with the data from other users unless fully sanctioned and sanitized by a government agency that is woefully incapable of operating in a fast-moving environment where uncertainty is okay and a REASONABLE EXPECTATION.
The whole point of regulation is that having medical tests, drugs, and devices coming out of a "fast-moving environment" where "uncertainty is okay" is not safe. If you don't like this regulation, either lobby effectively to change it or get your medical services in a non-regulated jurisdiction. To not obey the regulations out of disagreement is like saying "I don't believe that stealing is a crime, so I'm immune to those laws".
Take the liberation therapy for multiple sclerosis [1] as an example. Some claim it to be a revolutionary treatment that drastically reduces the symptoms and onset of MS. However, this has yet to be proven in a rigorous clinical trial. You would argue that as a consenting adult, paying for the procedure yourself, you should be allowed to do what you want. And you can, just not in the United States (or Canada, or many other countries). The FDA is not just going to let doctors start implementing this risky and unproven treatment [2]. There are too many people out there who assume that because a food, drug, medical test, device is being sold or used, that it is safe, that we can't just let them find out on their own. Just because you feel fully informed and consenting about these things, doesn't make that the norm.
In the end, I'm still not clear on what your stance is.
Is it for FDA reform and the reduction of its scope? If so, that needs to be backed up with a rigorous definition of how things stay safe in the new regime. Regulation is all about not letting stuff slip through the cracks.
Is it that 23andme is not a medical device and should be exempt from regulation? If so, then when the very first pregnancy test was developed, it should have been regulation-free as well.
[1] http://en.wikipedia.org/wiki/Chronic_cerebrospinal_venous_in... [2] http://www.fda.gov/NewsEvents/Newsroom/PressAnnouncements/uc...
No, and that's not what he claimed. Society at-large is not sufficiently educated or informed enough to know whether 23andMe's genetic profiling regimen follows best practices, actually correctly interprets the genome, and provides information that is both open, accurate, and appropriately disclaimed, as needed.
23andMe has chosen to pay lip service to the FDA, and steadfastly refuse to respond. They know what is required, say "yes, yes, we'll do that", and then don't.
"The fact that they're not marketed as a clinical tool for diagnoses?"
Only by very careful wording, and even then only subjectively so - sufficiently so that the FDA is concerned.
"Arm your doctor with information on how you might respond to certain medications."
"A simple DNA test led to an unexpected diagnosis for a mother and daughter, changing their lives forever."
"Find out things like if your body metabolizes caffeine quickly, or if you're at a higher risk for diabetes."
Statements like these may not /literally/ be "clinical tools for diagnosis", but the reasonable layperson can easily interpret them as such. For comparison, I'm an EMS provider. We are taught all through school that we don't diagnose. We "assess" and "rule out" and "treat for", and only a physician diagnoses. Do you think we don't say "this person has appendicitis/other given condition", and act that way, and do you think it makes a great deal of difference to the patient I speak to? "I'm not diagnosing you with appendicitis or a kidney stone or a XXX, but that's what's going on."
Seems to me that they've been going back and forth with the FDA for some time.
My interpretation is that the FDA wants a strict legal interpretation that the AMA would be happy with. Probably the AMA would only be happy if doctors could get a cut of every genetic "test" being performed by 23AM.
Only by very careful wording, and even then only subjectively so
Subjectively, any interpretation is possible of even short sentences. I've learned that from years of arguing the inerrancy of the Bible and the Koran with fundamentalists for decades.
The takeaway for me is that you admit that they've carefully worded their claims so as to not give the overt impression that they provide tools for clinical medical diagnoses.
All the claims you quoted are well within the realm of possibility based upon the tagging of genetic markers.
For comparison, I'm an EMS provider. We are taught all through school that we don't diagnose
Yeah, but that's actually not because you couldn't be of a huge help with early diagnoses. It's because you have strict liability concerns thanks to a tort system that consistently reinforces suboptimal emergency medical outcomes in the name of risk reduction. I hardly think that your industry's lowest-common-denominator liability concerns are a good basis for how we treat cutting edge genetic tagging and correlation services.
At the end of the day, I didn't pay 23andMe to become a member so that I could learn of the handful of markers and correlations that could pass muster with bureaucrats at the FDA. And if I weren't willing to pay for that handful of markers for $99, I certainly wouldn't want to pay a lot more for a service that couldn't achieve economies of scale at $99 because they had to CYA to such a high degree.
I'm an adult who consents to 23andMe to have my saliva, pull out markers and do their best to show me correlations that might interest me. Why is the desire to have that liberty so difficult to understand?
1) Stop making health claims about their service 2) Produce the evidence required to substantiate their health claims
The FDA has very high standards when it comes to health claims, see the Warning Letter for Cheerios: http://www.fda.gov/iceci/enforcementactions/warningletters/u...
Fiber can help lower cholesterol. Cheerios have fiber, thus why not make the claim that Cheerios can help lower your cholesterol?
The FDA is not big on "extrapolation". If B causes C and your product A causes B, you can't claim it also causes C.
How did General Mills respond? The simply modified their claims:
3 grams of soluble fiber daily from whole grain oat foods, like Cheerios and Honey Nut Cheerios, in a diet low in saturated fat and cholesterol, may reduce the risk of heart disease.
http://www.fda.gov/ICECI/EnforcementActions/WarningLetters/2...
Medicine is NOT, I repeat, NOT 'hard' science. Most of the 'normal' lab values we use when reporting things like elevated white blood cell count, anemia, etc. are based on population studies with the average considered normal and setting extremes as 'abnormal.' You will find that different hospitals even have different values of abnormal. Not to mention, for cutting edge therapies (Oncology, I'm looking at you) there is no 'standard' of care and there is a high variability between physicians. My point is that, I could go on and on about how much the field of Medicine is subject to the vagaries of stuff like regional practice, academic pedigree ("Oh, we at Hopkins always do it this way, but Mass Gen does it that way."), as well as bias like anecdotal evidence (the plural of anecdote is NOT data!) or just plain mis- or dated information. If you are out of med school/training for only a couple years, the amount of new therapies, lab tests, etc. can be staggering and if physicians aren't learning about the new, then they quickly become dated.
So, what am I saying? The FDA is holding 23andMe to an unreasonable standard. If your doctor saw the same test results, they might tell you the exact same correlation (oh, you are predisposed to Alzheimer's or Parkinson's). Tests are incredibly complex... I mean we still don't have a perfect pregnancy test! Again, there is sensitivity, specificity, with false positives and false negatives. If you are unfamiliar with how inaccurate medical tests can be, please read up on some of those items. This is the best we've got for now. If people can order pregnancy and HIV tests over the internet, there's no reason they shouldn't be able to find out they 'might' have a predisposition to high blood pressure. I would argue 23andMe is not doing something so radical... as far as I'm concerned, pregnancy and HIV tests should NOT be over the counter if we are going to limit genetic screens because there is a LOT of work to be done in positives of either.
Unconvinced? Well consider this. Over the counter Tylenol products are responsible for about 500 deaths per year in the US. Do you believe that Tylenol should be prescription only just because a minority of consumers make mistakes? If not, then you must conclude that over the counter sales of 23andMe should be allowed unless the FDA can show it has ACTUALLY harmed people.
http://www.mercurynews.com/business/ci_23974801/tylenol-warn...
Yes, 23andMe makes claims, and uses phrases like “likely”, “most often”, and so on. The FDA is asking to be provided the evidence that such statements are accurate, or how accurate they are.
https://www.23andme.com/health/all/
Under each item, there is an extensive explanation including the technical report regarding the exact details of what is being tested with citations.
This is the list of FDA approved 'genetic' tests:
http://www.fda.gov/MedicalDevices/ProductsandMedicalProcedur...
It's a sad mismatch between practice and regulation. Industry is WAY ahead of regulation. Very common tests performed by Clinical Geneticists are not included in this list. Yes, they will give you the disclaimer, but the data is there.
Sincerely etc. etc, Snidely, Sr.
But, there is a lack of quality tools available to query public genomic databases based upon raw data received from services like 23andMe. If anyone is a 23andMe customer, try out Promethease[1]. There is probably room for selling software to DNA service customers, but it probably isn't that big of a market (yet?).
An example, well-designed tool for viewing raw data files is 4Peaks[2].
Or maybe I'm totally wrong.
If true, that is quite embarrassing to the medical community.
Looks like a case of the FDA doing what it does best--ensuring the task of medical innovation remains with big pharma, and leaving Americans stunned at the results.
http://www.sciencebasedmedicine.org/new-concerns-about-the-s...
But the alternative is banning all information that anyone could possibly use to evaluate their own health.
stopped reading there.