Twins get different results when they put 5 ancestry DNA kits to the test
cbc.ca
cbc.ca
It's not like they're fully sequencing every DNA sample they get.
> AncestryDNA found the twins have predominantly Eastern European ancestry (38 per cent for Carly and 39 per cent for Charlsie). But the results from MyHeritage trace the majority of their ancestry to the Balkans (60.6 per cent for Carly and 60.7 per cent for Charlsie).
This part of the article especially seems like hair splitting considering the Balkans and Eastern Europe tend to have a lot of overlap. In fact, "Balkans" in particular is an extremely ambiguous linguistic term and can mean so many different things to so many people.
"She also has French and German ancestry (2.6 per cent) that her sister doesn't share."
I assume from this that 23andMe is reporting the results to this level of accuracy (a tenth of a percent). For the sister who got 2.6% they are strongly implying a certainty that she has a small amount of French and German ancestry rather than what actually seems to be the case which is a small chance of some French and German ancestry.
Since it seems their results are quite lacking in both precision and accuracy they should do a better job of reporting them and advertising them in a way that makes that clear.
> I assume from this that 23andMe is reporting the results to this level of accuracy (a tenth of a percent). For the sister who got 2.6% they are strongly implying a certainty that she has a small amount of French and German ancestry rather than what actually seems to be the case which is a small chance of some French and German ancestry.
You picked out the single biggest discrepancy they report in this article, and yet even that doesn't seem to indicate an actual inconsistency to me. I would tend to interpret those results to mean that for one twin, they felt sufficiently confident to call a portion of their DNA "French and German", while in the other it fell below that confidence threshold so it got thrown into the "Broadly European" share.
This process already pretty much precludes objective perfection just based on what they're reporting. What does it mean for your ancestry to come 2.6% come from France? During what time period? What if it's on the border in one of those regions that switched hands several times? These are subjective decisions they have to make but it doesn't invalidate the whole test because occasionally you have enough evidence to call portion of your DNA "French and German" and sometimes you have to fallback to just calling it "Broadly European".
I think that’s the whole point of the complaint here. These companies pretend to have found overly precise results (which I’m sure are explained away to nothing in the fine print) and yet are subjective and lean heavily on guesswork.
You can't say something is 2.1374cm long when your error is +/-1cm. You can say it is 2cm long though.
In the majority of technical fields the standard is to round the result so that the significant digits do correspond to error rate.
https://www.ruf.rice.edu/~bioslabs/tools/data_analysis/error...
But even then, rounding to the error rate is a standard practice, but it certainly does mean it's wrong or even unethical, it's just pointless not to. Unless of course your motivation is driven by marketing/sales.
One might call that deception
But if you estimate your first moment is 2.1374 and your second moment is 1, I want to know just that, because it is the numbers I'll plug in my equations. I will decide to round if and when necessary
The only time this wouldn't be true is if you mistrust the significant figure calculation of the source, and believe you have sufficient insight to do better.
If the precision of your estimation is not a direct function of the standard deviation, but is a "hidden property of the process that obtained" it, we have much bigger problems that losing "valuable information"
In fact, this is part of why you'll see physicists do all their reductions with variables and plug in numbers at the very end. This ensures that you doing get (what we could call) floating point errors. You don't have extra numbers hanging around (from real numbers like 1/9 or pi). There are also other benefits to doing this.
I think you're confusing different types of error. There is error between measurements and an inherent error to the device you use to measure. There's also a difference between precision and accuracy.
Standard deviation is the difference in multiple measurements. For example if you measure something 10 times to be 51mm, then your standard deviation is 0.
But that doesn't mean you have no error.
The "property of the process that obtained it" is not hidden. A simple case is a ruler. You have lines on the ruler that tell you certain intervals. If the smallest interval on your ruler is 1mm, then all your calculations can be made to +/- 1mm (that is, up to 30.5cm on a standard 12in ruler). There is nothing hidden about this. All that is being said here is that your measuring device is not perfect.
So using the two errors, we have a measurement of 51mm +/- 1mm (or frequently in a short hand you'd just say 51mm). It would in fact be deceptive to say that your measurement was 51.0mm, because that implies that you have more precision than you actually have (implying that you have on the order of +/- 0.1mm precision).
Even if we go to the example you give, the measurement should be done n times, each reporting the exact result found like 51.0 51.9 51.95 etc. Even if the decimals are outside the smallest interval of your ruler: take enough of them and you can get closer to the actual length which may be 51.55345 and that you would never have been able to measure anyway without a caliper
The best thing is you can even do that by resampling old measurements (a process called bootstrapping)
So yes, if you remove the tenth of millimeters, you lose information.
What's wrong is not the number, but that custom makes people think 51.0 means 51.0 +- 0.01 or anything else while it was never said like this.
Repeat enough time and you can interpolate 51.9, which you wouldn't have been if you had thrown away the precise measurement even if the precision is within your measurement errors (of 1 mm here)
Why you do multiple measurements is because as humans we are imprecise. Any engineer, wood worker, whatever knows the saying "measure twice, cut once". In your example, the thing you are measuring could well be exactly 52.0000...cm.
If you don't believe me I seriously want you to ask ANY physicist. They can even be an undergraduate (assuming they aren't a freshman) and they should know this. We even use this to figure out what we should spend money on. We can process these errors and determine which measuring device needs more certainty and buy that new device.
This is WHY we have very precise devices. With your method, we could theoretically get measurements to nanometer levels. I can tell you, I would much rather spend 15 minutes making a hundred measurements than spending thousands of dollars on a laser and equipment needed to make precise measurements down to the nm level.
To sum up:
> take enough of them and you can get closer to the actual length which may be 51.55345
NO! This is just dead wrong. It'd be right if you said 51 or 52.
> that you would never have been able to measure anyway without a caliper
No! This is why we have calipers!
> if you remove the tenth of millimeters, you lose information.
Not if you didn't have that information in the first place.
>What's wrong is not the number
Yes it is.
I can only assume: 1) You are trolling, 2) You are really thick headed, and/or 3) You've never taken a physics class. I'm not saying anything here that isn't easily verifiable. I have others backing me up. So if you have no interest in learning, then there is no point for me to continue.
I remember watching PBS DNA documentaries[1] where it's mentioned in one of the episodes that DNA-wise there's only 1% difference between any two humans.
I looked for the same details today on genomenewsnetwork[2], it has following line
> we are all 99.9 percent the same, DNA-wise. (By contrast, we are only about 99 percent the same as our closest relatives, chimpanzees.)
It seems like even an error of 1% would be actually be same as missing the whole difference between two people.
[1] https://www.youtube.com/watch?v=apOP8MMedqE
[2] http://www.genomenewsnetwork.org/resources/whats_a_genome/Chp4_1.shtmlNot really, Balkan countries are well-defined and have been so for over a century now. It's basically European countries that were under the Ottoman Empire:
Greece, Albania, North Macedonia, Bulgaria, Romania, Serbia, Kosovo, Montenegro, Croatia, Bosnia. One could also throw Slovenia in there but historically speaking, they have always been pretty different from other Balkan-folk.
"It can be difficult to define exactly which countries are included in the Balkan States. It is a name that has both geographic and political definitions, with some of the countries crossing what scholars consider the 'boundaries' of the Balkans." [2]
Almost the entire wikipedia article is dedicated to explaining the various definitions of it and how it is a problematic term. [3]
[1] https://www.britannica.com/place/Balkans
[2] https://www.thoughtco.com/where-are-the-balkan-states-407024...
According to certain definitions, only the coast of Croatia is in the Balkans (without Istria peninsula). Vojvodina (the northern region of Serbia) should definitely not be in the Balkans. Slovenia definitely not. Romania? I can't tell without looking up the definitions.
See the problem?
Also not sure what you mean by Slovenia being historically different - slavs, similar language, part of austro-hungarian empire, part of Yugoslavia, etc etc
We don't have a good way to name the origins of populations. We use approximates based on nationalities which is probably a really, really bad proxy. Take France, where there is probably not a single part with the same kind of gene pool (germanic invasions from the North, gallic roots in places that were untouched, italian tribes from the south and I could go on) so "French & German" as a split means absolutely NOTHING.
If somebody says they're a "quarter French" -- putting aside whether that's interesting or not -- do you believe that's meaningless? Do you demand them to specify whether it was one of they have Gallic roots or if it was one of the Germanic tribes that invaded from the north 1500 years ago or something else altogether?
I think the point of this test is to try to capture the spirit of what Americans have done for a long time, which is to try to describe their heritage in fractions using nationality of a distant (but not too distant past) as a rough guide.
And in the same way that your genome might prove more complicated than naming fractions by nation, somebody might say "My grandfather came to the USA from France... but he was part of a second generation immigrant community from a Greek-speaking part of Sicily who emigrated en masse to Alsace, which was located in Germany, not France at the time".
So maybe the DNA test will call that French or German or Italian or Sicilian or Greek or whatever and that might oversimplify it and the test doesn't work as well as somebody whose ancestors stayed in one location and was part of a largely homogeneous population, but that doesn't render the test meaningless. Part of the excitement people derive is that it can sometimes illuminate your ancestral history beyond what is well known. Using the previous example, somebody might not be aware of the complexity of their grandfather's heritage, and might be interested to understand that part of the family has roots in Sicily and not just France.
I thought it was pretty well known that these services are estimates only to a large degree? These services only take samples, they don't sequence the entire genome. Of course there will be errors when you have to extrapolate from a sample.
Even for me personally, my results changed when my parents did it and linked up to me, because their results were able to flow into mine.
I think part of why they used twins is to say its two different people so they'd have to treat it different.
I'm actually curious how repeatable the results are. If they are using sampling of portions of the DNA (are they?) how much variability is there in a single dna sampling enun through the algorithm 10 times? Similarly for how much variability in 10 dna samples from the same person?
Not if you sign-up with someone else's identity and pay with a different card. I very much doubt they go to any effort to verify the human the sample came from is the same human who paid and who the results are set to be sent to.
And there's a difference between "not perfect" and "doesn't provide the same result twice".
"doesn't provide the same results twice" is the honest acknowledgement that the concept of perfection is itself what is flawed here because it creates false expectations of what science can give us in this arena
The accuracy of the broad conclusions drawn from population wide samples about any single individual’s ethno-historical origins is a falsehood that science cannot ethically or logically support
Not getting the same result twice is pretty much 100% failure :-)
Frankly I am skeptical that they could even ascertain any semblance of your heritage but I'm definitely a layperson.
Reading other replies there seems to be a cheaper less accurate tech used for these home testing kits.
Basically all forensic science has turned out to be crap or misapplication of good science except for DNA which a) exonerates a heck of a lot of people and b) was a concept that matured in biology before being adopted by forensic scientists.
So are other forensic methods, like fingerprint[1] and hair[2] analysis. Turns out that forensic evidence isn't very scientific nor accurate.
[1] http://theconversation.com/fingerprinting-to-solve-crimes-no...
[2] https://rss.onlinelibrary.wiley.com/doi/pdf/10.1111/j.1740-9...
DNA fingerprinting involves:
* Sequence amplification of polymorphic sites by a panel of well-characterized primers, many of which cross-check each other by amplifying over the same locus.
* These amplicons are then characterized by size via restriction digest and simple electrophoresis. No sequencing is performed, only the length of the DNA fragments is characterized.
* Due to population-level variation of short-tandem repeats, analysis of ~20 loci can uniquely identify an individual with a desired level of statistical confidence. The kind of data used to confirm a match in e.g. a murder trial will be absurdly robust.
DNA fingerprinting is limited by the quality of sample collection and handling (i.e. don't mix up or contaminate samples), but not the accuracy of the method.
Even sequencing a genome is not perfect - the current most common method works by cutting the genome into pieces, sequencing those and then reassembling them. A pretty obvious limitation is that you can't detect repeated sections.
I'm not sure what method the police use but it definitely isn't the same as what 23andme do.
So were the results of FBI hair and fiber analysis, and look how that turned out.
https://www.washingtonpost.com/local/crime/fbi-overstated-fo...
So I picked up a mid-range automatic bp cuff, and started to measure it more regularly. The numbers were all over the place and completely unrepeatable unless I maintained exactly the same position/breathing/etc. I can literally swing my BP by 30+ mmHg, by re-positioning my body, drinking a cup of coffee, etc, all while officially still maintaining proper technique for measuring it. And yes, moving up a cuff size also tends to remove a ~5mmHg from my measurement.
I could write a book about how to get any BP you want out of the machine. Its the same with weight, I can shift my weight +-10% by eating a bowl of salty popcorn drinking a few 64oz cups of ice tea, then eating a large meal. Then I can go for a 3 hour bike ride, use the toilet and drop it all back off.
So, its sort of surprises me the fine grained aspect of DR's always measuring your BP/Weight, neither of which really mean anything until they get into the extreem ranges, and in the weight case you can get a better feel for it just by looking at someone...
We would schedule people for Nurse only BP checks to help establish the pattern.
This statistical anbiguity is why the medical community as a whole is often sceptical of the Direct to consumer DNA testing and SNP analysis. They report data in a convincing way which is statistically misleading and in no way plainly reveals what is within measurement error.
>An entire DNA sample is made up of about three billion parts, but companies that provide ancestry tests look at about 700,000 of those to spot genetic differences.
>According to the raw data from 23andMe, 99.6 per cent of those parts were the same, which is why Gerstein and his team were so confused by the results. They concluded the raw data used by the other four companies was also statistically identical.
>Still, none of the five companies provided the same ancestry breakdown for the twins.
So these people getting results and finding out they aren't related to their parents, are the results suspect?
Ditto the results about genes associated with dementia or heart disease?
I'm aware of the companies having disclaimers, I thought that was on the basis of them not really know what the genes did, not that the test itself was rubbish.
Is it a case of them adding too many decimal places to the percentages? Making statistics out of noise?
These SNP arrays, however, are exceedingly accurate and reproducible in their calls. A common medium-density SNP array used for human genetics is the Illumina CytoSNP-850K [1], which routinely gets >99.5% accuracy on calls. This is at least as good, or better than, whole genome sequencing.
The challenge in ancestry assignment is not determining what is in the sample, but rather in the interpretation and assignment of ancestral haplotypes.
[1] https://www.illumina.com/content/dam/illumina-marketing/docu...
>99.5% accuracy is great for a scientific measurement device that's only been developed very recently; it's absolutely rubbish if you want to sell the results to consumers.
Or potentially put people in jail based on these results...
You can get considerably better accuracy, you just have to be willing to fork over $1500 rather than $60. How much are you willing to pay to know if your 31% German vs 30%?
There was a good 5-10 years where microarrays were the go-to method for genomic investigation. Since RNA-seq has supplanted it, there has been no revolution in overturning those old results. At best, RNA-seq reveals what microarrays cannot: novel sequences, specific sequences that have difficulty hybridizing, etc.
The variation between these services is due primarily to how the data is interpreted and presented to customers, not the reads themselves.
In this case "SNP microarray" is a "digital" (or binary) readout of the genomic DNA sequence; specifically the array has attached to it sequences that match both the "reference allele" (e.g., a "G") and the "alternate allele" (e.g., "A" or some other base). Note that across the world's population, most sites in the human genome are "bi-allelic" meaning that there's really only two variants at each site, though of course it is possible to have tri-allelic or even fully polymorphic (A, T, C, or G) sites. For the purposes of ancestry, bi-allelic identification of commonly polymorphic sites is adequate, though the more sites and the rarer those sites are in world populations, the more specific you can be re: ancestry. This is why the gold standard for ancestry and haplotyping is whole genome sequencing-- you simply get far more informative sites to map to other populations.
When my doctor asks how tall I am, if I'm 6'1", then 6'1" is a good answer. 6' (or even better: about 6') can also be an acceptable answer, as it's an approximation. I wouldn't say I'm 6'8", and then try to rationalize it by saying my tape measure is cheap, and anyway it's still right to about 10%.
Reporting what you don't know can be just as important as reporting what you do know. It doesn't matter how good your laboratory method is if you can't report results in a way that doesn't mislead customers.
You're not going to get a very high degree of accuracy from a ~$100 test. They're not sequencing your entire genome for that amount of money. It is still good enough to determine the broad strokes of someone's ancestry, to within a few percentage points as you see in the linked article.
If you're willing to pay for full sequencing then you're going to get a lot more accuracy and repeatability. You're also going to pay a lot more for it than would be warranted just to satisfy your curiosity as to where your ancestors came from. These kinds of tests would really only be warranted if you have some rare, difficult-to-treat genetic disease. That might not even help you directly, but rather, be done to contribute to the body of scientific knowledge of the disease.
If you do it 10 times, you will always get the same buckets, but the percentages will differ back and forth for a bit. If you're goal it so find out your heritage, which these services are offering, then you are getting the right heritage repeatedly.
If your goal is to find out how many 0.1% of you is of an exact descendance, then you're going in with the wrong expectations.
When you sample a genome, you're checking for one specific string. Traditionally this is done by mixing a sample of the DNA with an enzyme that breaks apart a specific DNA sequence and then measuring whether the size of the DNA molecules have changed. Take the string that I mentioned above (FOO). Suppose that I mix it with an enzyme that cleaves along "GATTACA" such that "GATTACA" becomes "GAT| |TACA". If I mix a DNA sample containing the string FOO with the enzyme I just mentioned, then the molecule would be split into: "....GAAAAGATACCCACAGAGAT| |TACAAAAC...." If I mix a DNA sample that does _not_ contain FOO with the enzyme, no change occurs.
You know when you watch CSI shows* and the DNA evidence is presented as a bunch of light and dark bands? Those are produced by mixing the suspect's DNA sample with a standard cocktail of enzymes, treating them so that the DNA molecules are electrically charged (DNA might already have a charge, not sure), placing them at one end of a container of a standard gelatin with known pore size, and applying an electric field for a standard amount of time. The cocktail of enzymes cleaves the DNA at multiple different substrings and breaks down the DNA strands into multiple smaller substrings. Different size molecules will progress through the gelatin at different rates, so you can effectively compare two people's DNA to see whether the substrings match. You _don't_ know the person's genetic sequence, just whether or not his or her's DNA contains the same substrings as the DNA of the person that you're comparing it to.
* I'm assuming those TV shows present the DNA evidence this way. I don't watch them.
Disclaimer: I'm not a biochemist by any stretch. This is based on my memory from school, so don't take my explanation as absolute truth :-)
You have described sequencing and fingerprinting. These services use SNP sampling.
Sampling has been described several places in this thread so I won’t repeat that here;
Sequencing: https://en.wikipedia.org/wiki/Whole_genome_sequencing
Microarrays: https://en.wikipedia.org/wiki/DNA_microarray
* Full DNA sequencing. This is a multi-million dollar effort taken on by large collaborations, usually to generate a reference genome sequence for a species or particular line. Typically a variety of methods are used, from 'shotgun' style illumina, 454 longer reads, and now some newer even-longer-read methods. Long reads are important for generating accurate assemblies. Generally, some tricky portions will need to be meticulously amplified and sequenced with traditional Sanger sequencing to clean up. This gives you a complete sequence, including all inter-gene sequences, repetitive portions, etc.
* RNA sequencing. This method is the core of many molecular biology labs; you take expressed sequence (i.e. RNA that has been transcribed from DNA), reverse transcribe it to DNA, and then sequence the DNA. This gives you sequence information about the expressed part of genes (this is limited, though, because during the assembly process you'll reject many reads that don't map onto the reference sequence in standard analysis) but mostly gives you an idea of the relative expression of different genes. When people talk about gene expression going up or down in certain mutants, conditions, etc., they're usually talking about RNAseq (or qPCR if only a few loci are involved).
* 'Sampling' in this case refers to chips with various DNA fragments on it. You pass the sample DNA over it, and compatible sequences hybridize together, which is then detected via florescence. These chips are designed to carry sequence fragments that vary in the population. Together, these variants form a DNA profile. The quality of this profile depends on the design of the chip. The sequences included are generally loci that are known to affect particular diseases or to be especially variable in the population for assessing ancestry.
In cost magnitude, this goes from millions to thousands to tens.
The price of a whole genome sequence has fallen dizzyingly fast, faster than any technology of which I’m aware. We’re close to the point where it’s priced like any basic blood test. Exciting times.
sampling is a high resolution photo with large holes in it, a full sequencing is a full photograph with varying resolution and quality etc. There are still some areas of the genome that are inaccessible even with full sequencing. Improvements are being made with long read sequencing.
I've heard stories of affairs being discovered through DNA testing. Curious if the denials might have validity.
If two samples had enough similarity to say someone is a child of someone else, my guess is that's pretty accurate.
The whole concept of geographic ancestry may have fundamental limitations at the level where people are quoting percentages.
Do you have any personal experience with this? Or anything you can cite? I'd be interested.
The credit given to the intelligence of society at large here falls somewhere between "cute" and "astounding".
I would recommend flipping through a tabloid and then considering such content sells millions of copies each print. And then reconsidering your thought that it's "pretty well known that these services are estimates only to a large degree".
You could easily make the point that many people won't understand something without coming across as judgmental and elitist, but failed to do so.
He said "Of course there will be errors when you have to extrapolate from a sample." The common man doesn't know what extrapolate from a sample even means!
I'm sorry but suggesting society as a whole understands genetic testing methods and techniques is just so far absolutely removed from reality (you're perhaps justified if you work for Illumina or are a faculty member of a University) that the point needed to be made even if harshly. Maybe I just interact more with "non hacker news" types than the average poster here, but that post quite frankly seemed so absurdly out of touch as to remind me of the Lucille Bluth quote from Arrested Development - "I mean, it's one banana Michael. How much could it possibly cost? 10 dollars?"
Sorry I couldn't help myself. :)
For example, you're happy to fire off about how dumb everyone is for reading tabloids because the finer points of how people unlike you consume media are not particularly important. And, likewise, the typical person buying a DNA test as largely a conversation starter isn't going to invest a great deal of effort to understand how DNA tests work.
Tragically people are notoriously bad at long term predictions of what will matter to them and how that relates to a broad corpus of knowledge or habit of skepticism. The result is that people are so ignorant they’re under an illusion of being knowledgeable, and at that point they’re more or less a lost cause.
https://en.m.wikipedia.org/wiki/Dunning–Kruger_effect
It might help to know enough about DNA to wish to avoid these services, especially given how they cooperate with law enforcement. Mostly though, it’s just really hard to calibrate your corpus of useful knowledge from a position of profound ignorance. Plus a lot of people really are quite stupid, and ignoring that in favor of some superficially positive message about mutual respect is potentially dangerous.
I'm not a biologist, but apparently it's not the case that twins have exactly identical DNA:
"Identical Twins' Genes Are Not Identical" ( https://www.scientificamerican.com/article/identical-twins-g... )
My wife did the same with the same result, as did a friend who had purchased the original 23andMe kit and wanted to get the newer data from the new chip.
So while I don't doubt the accuracy of the linked article, my anecdotal experience is that the 23andMe DNA tests have repeatability at least.
> Despite the popularity of ancestry testing, there is absolutely no government or professional oversight of the industry to ensure the validity of the results.
> It's a situation Gravel finds troubling.
What is up with this incessant clamoring for a nanny to supervise every little thing?
This is published on a page where, thanks to "oversight", I have to click a little spam box warning that this page, like every other damn website on the internet, uses cookies. No thank you. I really don't want to pay more taxes for the Central Bureau of Making Sure You Know You Might Be 7% Less Eastern European than Your DNA Test Indicates.
And I guess we remove all oversight now because of a bad example? Pack it in, FDA. We're done monitoring food safety thanks to cookie banners.
Your parent makes the argument, very reasonably imo, that the overhead and cost of regulating these types of company's results are not worth the upside. People make lifestyle choices based on things they hear from palm readers - we don't use the government to regulate and test the validity of psychics. I think this falls into the same category.
That was my point.
> Your parent makes the argument, very reasonably imo, that the overhead and cost of regulating these types of company's results are not worth the upside.
I think we read different comments; all I got was a snarky condemnation of government oversight.
A 1974 study by University of Chicago economist Sam Peltzman concluded that since 1962 the new rules had reduced the rate of introduction of effective new drugs significantly—from an average of forty-three annually in the decade before the amendments to just sixteen annually in the ten years afterward.
... The severest criticism leveled at the drug lag is that without access to a drug available elsewhere, seriously ill patients will suffer or even die. Peltzman raised the subject in his 1974 study. He noted pharmacologist William Wardell's estimate that because the relatively safe hypnotic drug nitrazepam was not cleared for use in the United States until 1971, five years after it was available in Britain, more than 3,700 Americans may have died from less safe sedatives and hypnotics. After earning the Nobel Prize for chemistry in 1988, U.S. drug researcher George Hitchings of Burroughs Wellcome Company said of an antileukemia drug he helped develop before the 1962 amendments: "We went from synthesis to the commercial drug in three years. That is absolutely impossible today."
http://www.econlib.org/library/Enc1/DrugLag.html
At this rate, the benefits of FDA regulation relative to that in foreign countries could reasonably be put at some 5,000 casualties per decade, or 10,000 per decade for worst-case scenarios. In comparison, it has been argued above that the cost of FDA delay can be estimated at anywhere from 21,000 to 120,000 lives per decade. These figures would seem to support the conclusion that the costs of post—1962 regulation outweigh benefits by a wide margin, similar to Peltzman's results of a 4:1 cost-benefit ratio for the 1962 amendments.
"The Safety and Efficacy of New Drug Approval", https://pdfs.semanticscholar.org/feb9/53ab293f24a14b3b3d4b74...
If a company is making health claims there's already abundant regulation in place to prevent you from making unfounded statements. If you are making health decisions based on an Ancestry DNA test without consulting a doctor, you're clearly on your own.
https://www.fda.gov/ICECI/EnforcementActions/WarningLetters/...
https://int.customercare.23andme.com/hc/en-us/articles/21769...
But if you want to experience complete freedom, you can always go out there in the boondocks, dance with the wolves. Let that experience teach you the value and importance of the "nanny".
Are these tests just novelty tests? Are they intended to be? Is it clearly communicated to consumers that they are just novelty tests?
If they are just novelty tests, but they're not being upfront about that when they take consumers' money, is that acceptable public behavior? If that's not acceptable behavior, what is the right social response? That's the point in the discussion where GP appears to be.
If these are novelty tests, and are being marketed as such, would you mind linking to a page on, say, 23andMe's website that describes them as such?
If not - if, in your own words, these are "novelty" tests but the companies involved aren't being forthright about that - do you think a government response is so inappropriate as to dismiss it out of hand?
I'm not sure if you're serious, but 23andMe also offers tests for genetic risk factors of some serious diseases. I don't think people can verify the accuracy of these tests in their kitchen.
The incessant clamoring of these companies to sell themselves as a medical device platform.
Snark aside, it's fine that they have things set up the way they do for ancestry reporting. A couple of percentages here or there do not make a lot of difference when you are using it in a recreational use case. When you start getting into reporting potential medical associations with SNP variants which could affect clinical treatment decisions, it stats to matter a lot.
It is because corporations keep lying and behaving horribly.
Well, they all gave different readings. I repeated the running in place test. Some went up and down much more than the others; maybe 10 bpm more. I tried re-arranging them on my arm and doing the running in place test again, you know, maybe it only works on the wrist since the blood is closer to the surface there? Nope, the readings were all over the place still. It was not super scientific of me. I mean, I was some strange person running in place in a store with a bunch of iWatches strapped to my arm.
Maybe the watches have some sort of learning algo in them that was getting messed up with all the folks trying them on. Maybe I was wearing them 'wrong'. Maybe I should have waited 10 minutes between jogging sessions. I don't know.
But, to me, these things have a long way to go. A lot of the bio-tech and bioengineering out there seems to be a bit of snake-oil right now. I really do want it all to work, I think that would be a great boon to us all [0]. If something like the iWatch can't give repeatable readings in nearly any way, then it's all just a gimmick and not useful.
[0] I mean, can you imagine if Theranos' tech actually did work?! That would change medicine for the better in incalculable ways. I can see why that company's plan was so intoxicating.
Far from being just a doodle, the PPG readout ("ECG-like trace") comes from changes in skin reflectivity that indicate relative changes in blood pressure. [1]
Obviously this isn't as good as an actual ECG (and can be disrupted by movement), but it's far more information than is needed to accurately track heart rate. These devices can be as accurate as a pulse oximeter [2], though they don't account for the movement of the user.
[1] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4822420/ [2] https://www.ncbi.nlm.nih.gov/pubmed/28464700
Unrelated: It is called the Apple Watch. :)
There are tons of people that test these devices against chest straps online, and almost all of these bands give results that are surprisingly accurate.. or at very least about what you'd expect out of a wrist based monitor.
ML only works for these types of measurements when you have some kind of reliable source of truth or fitness function to compare against.
I can't say that I know exactly how these things work, but I wouldn't be surprised if they were built to work with the blood vessels and skin thickness of a wrist, where watches are worn, rather than the skin thickness+hair+different vessel locations of a forearm.
Different parts of the body are used for different things. It's why blood oxygen monitors that work on fingers don't work on toes.
An iWatch is not the pinnacle of wrist biometrics, however. I'd be a lot more concerned if a high end Garmin got it wrong, that's their whole business.
Still not nearly as accurate as the chest-based models as mentioned elsewhere, but for 95% of people it's plenty.
Wrist devices will give inaccurate pulse counts if the band is too loose, skin is dirty/wet, or your arm is moving around quickly. If you need accurate numbers in all circumstances then a chest strap which detects electrical signals is a much better option. I usually wear a Garmin HRM-Tri strap while exercising and it transmits a wireless signal to my fitness tracker.
The only proper comparison would be with a chest strep, which are much more accurate.
Also, it really differs per person and how you wear the watch. Just wearing it a bit differently can change the results significantly.
Sweat also gets them messed up easily.
All reasons why, if you want accurate data for workouts, you should always use a chest strep. Even if they are much more awkward.
I've always used a chest-based heart rate sensor until I started trying wrist-based about 4-5 years ago. I've used countless brands and models (ranging in price from about $49.99 up to about $399) on my wrist - and literally none are as accurate as the chest-based strap. The chest strap responds immediately to increases and decreases in heart rate. Whereas the wrist sensors always seem to lag, sometimes by up to a minute or so.
The wrist-based sensors always seem to differ depending on where on the wrist they are worn - high, low, inside of the wrist, outside of the wrist, etc - by maybe 5 bpm or so.
In either event, if you really want to test the accuracy of the wrist-based sensor, you should really compare it to the output of a chest-based strap.
In fact, the results will largely be personal - because the electric signals may differ per person greatly in different areas of the arm.
Needless to say, my apple watch is worn daily, but when riding, I have gone back to a chest-based strap.
That all being said, there is a guy that reviews these gadgets and will often wear 2-4 sensors while working out, and post an overlay of the results from his workouts. The baseline should generally always assume that the chest-based straps are accurate. His site is called DCRainmaker, if anyone wanted to look up his comparisons/reviews.
A faster response could be got by measuring the time between heartbeats (probably a low but even number of beats).
I mentioned it to the employee who was manning that table, and she told me the demo watches had some pre-programmed data for demo purposes.
I was puzzled by this since heart rate monitoring is a key feature users care about. It occurred to me that heart rate data of everyone who has tried on the watch might count as the kind of medical information you shouldn’t publish on demo watches.
Imagine ridiculous stories like “CEO of X’s heart rate was 130bpm when he tried on the new Apple Watch, says journalist who tried it on right after.”
Although people might give you strange looks :)
https://www.macrumors.com/2017/05/24/apple-watch-heart-rate-...
That's an overly generous explanation. Fitbit-like devices are notoriously inaccurate[1], and the simplest explanation seems to be that their inaccuracy is unintentional.
[1] https://www.independent.co.uk/life-style/health-and-families...
https://hipaahealthlaw.foxrothschild.com/tags/class-ii-medic...
Are we claiming that every product needs 2 FDA applications: one for the end user and one for a special run of demo watches? I believe (but did not check) that demo watches would be either exempt or go unnoticed considering the amount of data it could gather...
The wearing a bunch of them at the same time is going to be wrong, yeah. Wrist mounted fitness devices are calibrated to read on the top side of the wrist. Even reading on the bottom side of the wrist would be a bit different, way up the arm? Very different.
It is possible to make accurate devices that read there, they just need to be constructed to do such!
Wearables are also meant to give an accurate rating over the long term. If you go running for 30 minutes, are the overall calorie burn, average HR, max HR, and min HR, all correct? Tracking in perfect sync with the heart is nice, and of course everyone tries to do that, but if the number is off for a second here or there, well so long as the overall result is the same the usefulness hasn't really changed.
Finally, due to the way optical HR works, random unexpected, movements are really hard to deal with. It can take some time for an HR sensor to lock on to the heart rate. This isn't too long (measured in seconds, and each generation of devices is getting better and better), but it relies on regular patterned movement, such as arms pumping while running or walking, the regular motion of a bench press, etc.
Stopping and checking the HR value every few seconds to see if it is correct is literally the worst case scenario for a lot of wrist mounted sensors!
Ground truth is strapping a heart rate strap to your chest, putting on the wearable, going for a run as you normally would, and comparing the two at the end.
Multiple sources have done this, and found the Apple Watch to be best in class when compared to directly measuring HR.
tl;dr: Stopping and checking wrist mounted devices can confuse them, Apple Watch, and everyone else in this field who is serious, are tested against HR straps and they come within spitting distance of accuracy.
What I found was that the heart rate correlated well with taking it manually and/or a finger heart monitor (pulse ox cheap from Amazon).
But with both devices you could actually tell when it was clearly wrong independent of comparison. The point is you more or less (at least I found) could figure out when it is right and wrong just based on past use and what you would expect (feeling your pulse). So if it was off it was grossly off and obvious. That is my experience with the watch that I have had for years. Either 'correct' or 'vastly off and easy to tell that is the case'.
Personally, I would love to help create legislation aimed to protect consumers from perceived and actual problems related to DNA sequencing and analysis. I don't know how though and I fear it'd fall to the back burner in the current US political climate. Our customers are not only US but international though and that presents yet another dimension to the challenges therein.
This article states:
> Whatever your ancestry results, don't get too attached to them. They could change.
That is absolutely correct. I've stated why in a previous comment [1]; Some DNA analysis software employ stochastic algorithms. That means that the answer they provide can be different if run more than once.
The article also states:
> Despite the popularity of ancestry testing, there is absolutely no government or professional oversight of the industry to ensure the validity of the results.
That is also correct. I've heard peers state that the industry isn't regulated outside of the field of medicine during product discussions. I think that's something which needs to be addressed: leaving it unaddressed can encourage predatory business behavior.
[0] https://news.ycombinator.com/item?id=18196717 [1] https://news.ycombinator.com/item?id=18196984 [2] https://news.ycombinator.com/item?id=18564380 [3] https://news.ycombinator.com/item?id=18569659
They are not allowed to sell health-related reports anymore in all of europe and other countries. While i think that some regulation and standards would be good, banning it overall was a dumb measure.
(1) reasonably guarantee that my data not only won't be distributed - but will be destroyed (on their side) within a short time frame
(2) give me access to all of the raw data and methodologies used in their analysis
Does anyone know of such a service?
The fact that none of these services allow for anonymous submission should be a huge tell about their real motivations.
But ultimately any kind of DNA analysis (that exposes the raw data) is intrinsically de-anonymizable, no?
Sure, you can be anonymous, but you're the son of John and Jane Tompkins, the brother of Emily Tompkins, the cousin of Mary Smith, and the father of Dave and Buster Tompkins.... huh, I wonder who you are.
You're incorrect. At the company I work for, which is one of them mentioned in the article, a customer pays for DNA sequencing and analysis. Who pays for it is often separate from who's being analysed. We often have customers who pay for products for their family, friends, or otherwise. The name on the DNA sample is not verified in any way shape or form (and how could we verify it beyond just billing information?) You can opt-in or opt-out of public matching. You can download your raw data anyway (GDPR requires this if you're an EU citizen).
> should be a huge tell about their real motivations
I would like it if you'd elaborate on this.
I don't have an opinion about this, but they're probably implying that the business model is the same as Google, Facebook, et. al., which is selling user data to third parties.
Carly's results have a larger 'Broadly European' percentage, but that is a catchall that includes: Italian, Eastern European, Balkan, French & German, Iberian (Others category), and Broadly Southern European (Others category). Therefore Carly's 'Broadly European' traits could be used to match Charlsie's results.
It is unfair to call the 23andMe results 'different'. They are getting the same results, however one twin is getting more specificity.
There was a controversy about Korean & Japanese ancestry results from 23andMe where ethnic Koreans are sometimes classified as >30% Japanese. These two groups shared common ancestors since ancient humans migrated via the Korean Peninsula to the Japanese archipelago centuries before the modern nations of Korea and Japan came into existence. Slapping either one of these specific labels on shared ancestry is wildly inaccurate and short sighted.
Years of highly visible advertising by these companies which implies they're accurate? All of those “discovery your ancestry” posters they put up around here had specific countries and locations.
The landing page shows me this headline "This new year, commit to a healthier you - inspired by your DNA." I'm sure they have it legally covered somewhere in their TOS but it's still not good if these are inaccurate or estimates.
If the post analysis shows nearly identical DNA that's great.
The whole '% from this or that country' has to be necessarily fuzzy, because that's just a probability game anyhow.
Part of the old-school American tradition many regions (mostly rural, which is where over 80% of residents lived at one time) was to have a bunch of kids and encourage them to spread out as far & wide as they wanted; this still continues in some places. The logistics of 'keeping in touch with family' have some pretty hard limits past a certain point, and recordkeeping can be haphazard/fragile.
A great many people in North America have significant trouble tracing their ancestry back past their great-grandparents.
This back of the envelope estimate suggests that 99.6% similarity for somatic samples of identical twins is not unexpected.
The other factors you discuss are more likely to cause issues. One major one is the error rate of the microarray because of the high number of genotypes being assayed. If the chip has an error rate of 0.1% and 500k tests are performed you would expect 500 mis-calls. I used to work for Illumina, who makes these chips, but I worked on the DNA sequencing side of things so I don't know the actual error rate off the top of my head.
However people are really worked up about a new online service that encodes multiple analog music sources into lossy mp3 samples using different analog encoders each time, and the service sells an "identify matches similar to this song" as a service that people think is the same level of exactness as the old fashioned court-of-law .wav file bit by bit comparisons, but its actually more of a lossy best guess pattern matching service instead. Yes they DO use the same general technology, mostly, but its quite different in purpose and outcome.
An even better analogy is we are VERY used to an online service that OCRs a scan of a music CD and outputs the musician name every time based on OCR of the disc itself, and now people familiar with that technology are VERY confused by an app that listens to your cell phone mic and often squirts out the musician name based on that raw analog sound sample.
Its more a miracle the new tech works at all, than a scandal that both business models aren't identical in accuracy.
Are there services that you can pay extra that will do a full DNA sequence and have access to 23andMe's comparison data?
Or am I being paranoid?
"The company said it approaches the development of its tools and reports with scientific rigour, but admits its results are "statistical estimates."
It is very likely that there's a fine print on their procedure that everything is a Guesstimate and while their algorithms will process things the same for every sample, there's just too much variations that are unaccounted for even in such controlled environments.
What would surprise me for example is if someone sent in their DNA for analysis twice on the same company and get varying results. That would mean that the process itself is not well-established.
I was watching a BBC documentary about how Rome influenced the history of Scotland. The announcer, who has a very pronounced Scottish accent, took a DNA ancestry test. The results (IIRC) indicated Germany, Italy, and eastern europe. (The announcer interpreted this last one as Dacia) Come to think of it, he doesn't look so different from Metatron.
The twin results not matching is kinda disturbing, though.
Identical twins are supposed to have identical DNA, right? Or did I really misunderstand a lot of school.
They are comparing a low resolution map the genome. The genome is 3+ billion nucleotides, only 700,000 (0.0002%) is being compared to find 0.04% difference! that is shockingly similar!
Not surprisingly, the results are not reliable.
Feed the unreliable result into any classification algorithm, and you still get unreliably results (actually I was expecting worse)
Not only that, but you only read small parts- SNPs. So there is no bijection; just a rough correspondence. On the bright side, it's good for privacy.
I think you’re describing short-read sequencing here but this isn’t what any of these services does. They do microarray genotyping rather than sequencing. And in neither case is there “inference”. True there’s something called imputation but it works quite differently and I don’t think (though I don’t know) that 23andme uses it for ancestry. I’m also not sure what you mean by “bijection” in this context.
I don't think anybody's ever claimed this type of mail in test is 100% accurate, and most of the results were within a couple percentage points of each other.
> According to the raw data from 23andMe, 99.6 per cent of those parts were the same, which is why Gerstein and his team were so confused by the results. They concluded the raw data used by the other four companies was also statistically identical.
If they were siblings, ~50% of the (several hundred thousand) SNP calls would be the same, not 99.6%. (Which is about what you would expect from identical twins, since that's where either one disagrees with the other, and errors usually don't strike twice, so a 99.6% overlap roughly implies a per-SNP accuracy rate of sqrt(0.996) = 99.8%, which sounds reasonable given the difficulty of sequencing and the inherent noise & randomness.)
On a fun side note, identical twins are not exactly 100% genetically identical, which is how rapist/murderer identical twins are being convicted these days based on DNA evidence; but there are so few unique mutations to each twin, and the new mutations wouldn't be on the common-SNP tests, that you have to do the equivalent of like 4 whole-genome sequences (IIRC, you need coverage of ~120x vs the more usual WGS of 30x) to get enough evidence to override the prior of any difference being just a repeated sequencing error, and it's very expensive.
99.6% is hardly an impressive figure. I would expect 99.9999% between twins at least. Chimpanzees and humans already share about 96% of genes.
https://news.nationalgeographic.com/news/2005/08/chimps-huma...
This reminds me of when economics try to predict what the GDP will be next year. It's almost always wrong, so why would ever listen to anyone who is always wrong?
It's mostly a scam, in my opinion.
Also, you should not underestimate how not knowing where you ancestors came from is disturbing for many people.
EDIT: for those who disagree, check what just the 1st two PC can give for European DNA after a simple rotation of the axis to match a map : http://blogs.discovermagazine.com/gnxp/files/genmap1.jpg
I think it's fair to say there is a scientific base.
I don't think it's a good thing for people to attempt to derive any aspect of their identity from their genealogical ancestry, it has no bearing on who you are as a person, and has the potential for great harm. I've seen it used to give people a false sense of culture and belonging to a community that they previously had nothing to do with. If you felt a connection to a particular culture or identity, should we really be telling people that their genome is responsible for determining whether or not they can associate parts of those cultures with themselves?
Racism results from a failure to separate identity from genome. If you listen to the dogma of a white supremacist, it's heavily predicated on associating the accomplishments of their "European" forefathers with themselves. If you listen to the dogma of a black supremacist, it's heavily predicated on assuming the identity of the Egyptians or Israelites and associating their accomplishments with themselves.
I really think that if an individual is truly "disturbed" by not knowing the precise nature of their lineage, that is an individual in search of an identity on a plate, and therefore highly corruptible. The motivation to learn more about one's ancestry should ideally not surpass the level of an intellectual curiosity.
I still think it can have a positive role, if only to help the person emotional investment.
> Is the usual "I'm part Irish part German part Cherokee" meaningfully more satisfying than "I'm west African?"
For that, "I'm west African" is less satisfying than "I did trace my roots to Mali" which can lead the person to study the history of the country, Timbuktu, etc.
Theoretically, they are similar, but since you inherit a random 50% of DNA from both parents, on an individual scale, you might inherit a misleading percent from your ancestors.
Let's say 3 of your grandparents are all full-blooded Italian. Like, their families have been in the same village for centuries. And one of your grandparents is from England (and is similarly from an old family in a small village).
So, theoretically your DNA results should 75% from that region of Italy and 25% from English region.
However, lets say from your mom, the parent who is 50% English and 50% Italian, you only inherit a mix of 15% Italian and 35% English (15% + 35% = 50% mom's DNA). It's random, so this is possible, especially on a small sample of ONE person.
And from your Dad, who is 100% Italian, you obviously inherit 50% of your DNA as from that Italian region.
Now, you theoretically should be 75% Italian and 25% English, but since it's random, your true DNA reads as only 65% Italian and 35% English.
This is why it's mostly bullshit to me. It's impossible to know what percents your DNA randomly passed on to your ancestors.
Sure, in our example, the two places where correct, Italy and England, but the percents are way off.
And that's honestly WHY we get these tests. We generally know where our families come from, we just have no idea what the percents are.
So, the whole point of the service is stupid since they equate DNA with ancestry.
No, not everybody.
Even percentage that are a bit off are better than nothing at all.
It's hard for me to imagine why anyone would feel any significant connection to an ancestry that they needed a DNA test to know they had.
Some people have not even such luck, and do not know their ancestors at all. Records are incomplete and some people lie. So instead, they try to grasp to what they can, using anything they can. DNA is but a tool there.
For me, DNA offers answers to questions about my family history, things that otherwise I would be unable to ever know.
I find it psychologically helpful. It helps me relate to ancestors I didn't know I have: who they were, what they were doing. When did they immigrate, what were their hopes, their lives, their challenges.
For me it is part of a healing process.
I don't think it will turn me extremist. And the data is not perfect. But it just feels nice to be able to relate to others when they tell you they are from xx country, their father from yy, their great grandfather from zz, etc. It lets you joint the common story.
Some family have had complex paths, with records lost. Dead ancestors can't tell you much about where they came from. I think it's just sad. I would prefer anyone to have access to the possibility if they want
Have you looked at things like thermometers? They don't offer more than +- 2F accuracy for those under $100. Some meters, etc only offer +- 5% accuracy or worse. Those aren't shams.
It would be helpful if folks would state their credentials/experience with respect to the subject area in their comment.
Mine: none whatsoever, but am interested in reading informed discussions with folks with either credentials or experience in subject matter.
So essentially just like every thread in every online discussion group then?
Depending on the actual goal of having people submit all their information for entertainment purposes
And new relatives
Could have been worse- you could have said "female, we know..."
Besides the needlessly dismissive tone, you are neglecting how know where the family came from is important to some people
switch (rand() % 100) {...}