Theranos and Zenefits are a feature, not a bug, of Silicon Valley
qz.com
qz.com
I think it's a partly a dysfunction of some investors, that they get excited by the plan and forget to drill into what actually works. Investors who consistently drill into the current situation aren't surprised when things are a little broken, because that's true of every startup.
Meanwhile, the realistic company that was passed over runs out of money and folds.
So the sensible 10 years for 1X that will make you profit but not overwhelming profit don't get funded, so the 10 years for 1X have no choice but to lie and get the funding they need, and investors get stringed along with all their sunk costs because most of the time that 1X payout is still a lot more than their sunk costs.
That's a hypothesis not well supported by data or theory.
On the other hand, average investor feedback does not create a learning opportunity for entrepreneurs who are pitching. Smile f-ing, and backdoor calls to other investors is the language of pitch feedback. It creates a dichotomy where many entrepreneurs feel everyone else is embellishing and investors can't give feedback with candor in any case. Therefor, an internal narrative comes to life that its rational, and even in shareholder's best interest, ie, a duty of a CEO, to stretch the truth in a way that gets the company the resources it needs.
If you lie to people to get your hands on their money and misrepresent your work, you shouldn't be lionized as some kind of entrepreneuring maverick; you're a danger to the public interest. This is worse than making a calculation error that results in harm; it is willful and deliberate harm at the scale of a mass murder.
I'm not trying to knock the software development profession, I have a lot of respect for and derive a lot of joy from the work. There's just a difference between "As a member of this association of professionals, I declare this a safe and appropriate design, and am personally and professionally liable for failures in this design" and "I have push access to master and can deploy code to prod".
In the vast majority of sane engineering orgs there isn’t gross negligence and developers are very cautious (hence the popular evil PM / incompetent manager trope) to the point that they barely move at all.
No, but is most software work really engineering? Personally, and atthe risk of completely going off topic and starting a debate which has little purpose, I don't think so. I don't consider duct taping together various frameworks and third party libs to create yet-another-CRUD-app "engineering".
Are there safety concerns that a lot of engineering doesn't have? Sure. But it goes both ways. In her field of engineering there are limited impactees and mistakes while potentially fatal have minimum monetary value. I can fuck up and cost my company millions in an instant. Or expose millions of user's personal information. Different risks, and I'd still probably take the a stolen identity over death, but at the end of the day it's a similar job.
Software development has different constraints. An edit -> compile -> test cycle is free, but formal verification of code is obscure and unwieldy. Consequently, our best practices are more iterative and empirical than front-loaded and model-driven. But we do have some. (Not denying that software engineering is much less mature than other forms).
A team that's doing good high-cohesion low-coupling components, fault tolerant design, unit testing, integration testing, version control, code review, capacity planning, telemetry, staging, canaries, blue/green or incremental rollout, feature flags, etc. is acting in accordance with the body of knowledge called "software engineering" even if they're doing it for a boring CRUD app. Someone with a big pile of spaghetti code on his laptop is probably not doing software engineering, even if he's cracked P=NP. (He might be doing computer science - have you seen how researchers code?)
Capital reaps the rewards, capital should bear the risks. If capital wants to use standards and certifications to help select the least risky staff, great, but getting the law amended to transfer your risks onto your underlings is just evil. If it happens to us, I highly doubt it's going to come with any commensurate increase in exposure to upside...
"move fast and break stuff" is bad PR if you're facebook. It's a game-over felony if you're making medical devices.
Good example: the Freestyle Libre CGM is a recently approved glucose monitor for diabetics. The European version lasts 14 days and takes an hour to start up, the US version lasts 10 days and has a 12 hour (!) startup time. It's the exact same hardware with artificial restrictions added to the software for unclear reasons.
I'm in the US and my Dexcom takes 2 hours to startup, and that's really only because of the sensor temperature adjustment, otherwise it's pretty much instantaneous.
> In the US, FreeStyle Libre’s accuracy has improved a bit with these changes – the average measurement error relative to a reference value is 9.7%, an improvement from around 11% in Europe (and on par with 9.0% for Dexcom’s G5). (https://diatribe.org/abbotts-freestyle-libre-approved-us-rep...)
Which is absolutely ridiculous. 12 hours without a CGM is much, much more dangerous than a 1.3% margin of error.
It looks like on the first day, comparing blood glucose with FreeStyle Libre CGM, there was a significant difference in accuracy, with a lot more (~20% more) readings falling into "zone B", or "points that are outside of 20% but would not lead to inappropriate treatment" (chart https://www.liebertpub.com/action/showImage?doi=10.1089%2Fdi..., from paper https://www.liebertpub.com/doi/pdfplus/10.1089/dia.2014.0378).
This is just one quick thing I found on Google Scholar.
Given that this CGM can also act as a regular fingerprick blood glucose monitor during that 12 hour overlap, I think the FDA likely wanted to err on the safer side of using well known and well tested methods for monitoring to avoid any unexpected issues. Is there sufficient evidence of the benefit of CGM over fingerprick tests to believe that this caution could cause actual harm?
This is a question you could only ask if you were looking only at the data without understanding the problem at hand.
To answer your question: as a user I'm not sweating Zone B. That doesn't even register as a problem. Zone C is the only zone I care about, and Zone C situations can happen quite easily with fingersticks too. The entire game is imprecise -- every single day I make decisions based on incomplete data. You're going to have a very hard time convincing me that 11 hours of moderately imprecise data is more dangerous than having 11 hours of no data.
There are many users who would not. They might make a rash decision based on that data. If the data is in Zone B a larger percentage of the time in testing, in much wider general use there will likely be cases that weren't caught in clinical trials in which it could land in Zone C.
During this 12 hour calibration window, you can still use fingersticks, which you have to do for calibration purposes with other CGMs anyhow. And it likely would be a fairly minor change for them to support doubling up on sensors, so that you could overlap them in order to avoid the 12 hour window; but that change would require further testing and approval, so for now the 12 hour calibration window is left in. It was only fairly recently approved with this 12 hour calibration window, so I wouldn't be surprised if the company is working on support for pairing two sensors to one device to support doubling up like this.
Those aren't comparable situations. With my Dexcom I test in the early afternoon and before bed and that gives me 24 hours worth of extremely reliable information.
Here's a hypothetical: let's say you test and your blood sugar is 150 mg/dl. What does that tell you? Almost nothing. Maybe your sugar was 200mg/dl 15 minutes ago and you're dropping like a rock or it was 100mg/dl 15 minutes ago and you're spiking. Maybe you've been at 150mg/dl for the last two hours. You can't really tell, so you test again in 15 minutes. Let's say it's 100mg/dl -- looks like it is dropping, unless your last reading was on the high side and this one on the low meaning nothing's really changed. So you test again in another 15m and maybe you can start putting together a picture of what's actually happening.
This is a terrible process and only works if you're sitting at home doing nothing. What if you're stuck in traffic or on a crowded subway car? In a meeting at work? At the gym? On a date? In a movie theater? It can become overwhelming quickly and that's when mistakes happen. And remember this is all compounded by your body's biology: you have to think about the food you ate, how much insulin you took and when you took it, when the last time you exercised was, how much stress you're under, etc, etc, etc.
And keep in mind fingersticks don't perform that much better than CGMs. They're still susceptible to error (https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3317395/) and much more susceptible to human error. Failing to wash your hands properly, not drying your hands properly, not letting the alcohol dry completely after a swab, or leaving your strips in a hot car can all cause your readings to be wildly incorrect.
Another thing to note is even if your Libre is off by a huge amount (say 50mg/dl) the trend will still be accurate. Meaning if your blood sugar is constant over an hour the Libre will give you a reading that's also constant. This is extremely important because I'm much more likely to take action on trends (e.g. rising/falling blood sugars) than on individual data points.
You said this in the other thread:
> I would think that 12 hours of having to use a fingerstick during calibration would be preferable to twice a day fingersticks.
I will tell you it is definitively not. Currently Dexcom takes two hours to calibrate and those are the longest two hours of my week. The ability to schedule those two fingersticks is much preferable to 12 hours of radio silence.
> You are on HN, and likely reasonably technically inclined.
Sure. I'm a "good diabetic" but that's the nature of the beast for a disease who's only treatment is constant vigilance. There's a reason why I've never gone into a hypoglycemic coma yet know people my age who are already struggling with neuropathy. The lowest common denominator here is extremely low, and if an error margin of 30mg/dl or a single false reading is going to cause you serious problems you have much greater issues with your diabetes management than your CGM.
I don't really care that much if I'm 70mg/dl or 100mg/dl if the line is horizontal. Both values are, roughly, in the "OK Zone" that I just don't care about better precision.
Even if I'm going low, the difference between 50 or 70 isn't nearly as relevant as the rate of change. If I'm going down, but the slope is gentle enough, I know I probably don't need much to get back to normal, while if it's steep going down, I probably need to eat something regardless if I'm 50 or 70, or 80.
It may be that the device is more highly variable in the first hours giving the impression of spikes or drops where there are none.
It's hard to tell without the raw data.
But what it comes down to is, there was something about the accuracy during the first 12 hours that gave the FDA cause for concern. Whether the FDA is overly cautious or not is an open question; I do know that in the US we get worse health outcomes for more money than most other developed countries. How much of that is due to our terrible system for paying for health care, which is privatized but with enough regulatory capture and inefficiency to drive prices way up, and how much of that is due to higher costs and lower access to medication and devices not being available due to excessively stringent regulations from the FDA would be something interesting to study.
I think that the FDA could probably loosen up its regulatory process just a little bit and provide a net benefit. Not sure with how much there is wrong with health policy in the US, and partisan politics around it, that's very likely, however. It would be interesting to see if there have been any studies done on this.
They're moving closer to the "move-fast" thinking (although not the "break things" part fortunately) under the new administration: https://www.economist.com/news/leaders/21739157-drug-regulat...
Well, a quick search of Google Scholar turns up at least one citation in which the Freestyle Libre CGM was significantly less accurate in the first day of use than in later days, indicating that it needed more time for proper calibration:
https://www.liebertpub.com/doi/pdfplus/10.1089/dia.2014.0378
I expect that the differences are based on some actual findings, not just "unclear reasons," but you may have to do some research to find out.
The FDA is notoriously cautious, possibly somewhat excessively so, but there is some good reason for that caution; medication and medical devices can have unexpected behavior and side effects and it can take a lot of long term study to find them. One example of the caution working out well is the Thalidomide case, which was approved in many other jurisdictions but not in the US, and which had significant side effects on fetuses.
What's the right regulation? I don't know.
Anecdotally, as a diabetic, the margin of error in that paper is much less scary than going 12 hours without a CGM. Poor data is much better than no data, especially if I know the data is going to be a little questionable the first day. I was completely ready to switch from Dexcom to the Libre until they announced that restriction.
The decisions were being made by different people. It's not like the FDA decided that Americans require a 12-hour window but Europeans only need 1. The FDA is just more cautious.
People will benefit from those extra 11 hours, but they'll also be hurt by the lower accuracy. It's completely reasonable for different people to weigh these costs and benefits differently.
11 hours without a CGM is a much greater risk than marginally lower accuracy. As a potential user of this system I've been unable to locate research to convince me otherwise.
Because the FDA is very conservative, in part because that conservatism prevented the thalidomide incident back in the day.
When it comes to novel treatments, or monitoring devices, we don't know what we don't know. The FDA errs on the side of well-tested treatments and devices.
Extra inaccuracy during the first day could lead people to make mistakes that could prove dangerous or fatal. You don't have no data during the first 12 hours, you just have to fall back to the fingerstick method, which the FreeStyle Libre still supports.
It's possible that as more data is gathered in Europe and the US, it could be found that this extra inaccuracy is not harmful, and the European version approved in the US. Or it could be that the company is able to improve the device to improve accuracy in the first 12 hours, and get a new version approved.
It seems to me that other CGMs like the Dexcom require fingerstick calibration at lest twice a day, so I would think that 12 hours of having to use a fingerstick during calibration would be preferable to twice a day fingersticks.
The FDA is necessarily conservative (in the practice, not political sense) but actively pursuing changes like you ask: look at their interest and experiments with adaptive trials.
Out here in the valley they are considered by folks outside life sciences to be stodgy and bureaucratic. But they can move fast. In one (paper) submission I made we dropped one sheet of the 1600+ page document when copying them. Within a week of sending it to them they contacted us for that sheet and we sent it. My board of directors was very interested in how long it took us to reply (down to the minute) and when I said it was less than 10 minutes there was a big sigh of relief. Compare that to the months they gave 22andMe to reply to notices of gross violation that the company had ignored.
Making medical devices that give fake results usually isn’t even a crime, let alone a felony. Otherwise almost all of the makers of home glucose monitors would be in prison.
Or walk into your local supermarket and see how accurate the cheapest food thermometers are, if you want an even more basic example.
The former is an error that probably requires the manufacturer to inform the FDA and send out a Customer Letter (a low-level form of product recall). The latter is fraud.
> Making medical devices that give fake results usually isn’t even a crime, let alone a felony.
It very much is a felony and people go to jail. The FDA even can and does make armed (!) raids on facilities when they believe there is fraud.
> Otherwise almost all of the makers of home glucose monitors would be in prison.
Home glucose monitors in the US only need an accuracy of 20% and the manufacturers have to prove it. And their labeling is regulated just like any other medical product. I don't know why you think that their results are fake. A distal finger stick of capillary blood will not give the same results as a venous draw analyzed with GLP, but nobody is claiming that it should.
> Or walk into your local supermarket and see how accurate the cheapest food thermometers are, if you want an even more basic example.
These are not regulated products, unless you are using them for commerce (in which case they are regulated by your State, not the FDA).
So in other words a major category of medical devices, which are needed to prevent you from dying from eating undercooked poultry or whatever, aren't even regulated, then that seems to support what I was saying.
Sure, if you buy any of the top digital thermometers on Amazon. But if you buy a random analog food thermometer at your local dollar store then it's not necessarily going to be much more accurate than a random number generator.
For purposes of this discussion: a food thermometer doesn't work directly on the human body and it is not a medical device regulated by the FDA.
(If you are using a food thermometer on a human body, that's still probably not the FDA's jurisdiction--more like local police or the FBI.)
https://en.wikipedia.org/wiki/Medical_device
I had basically the same same experience as you in terms of buying something at the supermarket that was very obviously not giving accurate results, and also just returned it and bought another one that actually works. But if someone were to not notice, the results could be potentially catastrophic, especially if the person was pregnant or had a compromised immune system.
Medical thermometers are required to meet a much higher degree of accuracy because differences of even a single degree can have serious implications, and are correspondingly more expensive. As they're intended to be used in the body and not in food, they're also more fragile than their culinary counterparts.
I know that when a public company goes bankrupt, the investors with preferred stock can, at best, get back all their money, but they can't make a profit.
Grouping drastically difference offenses in the same bucket obscures reality instead of making it more clear.
Zenefits is closer to Theranos on that scale (and closer to Uber and a couple of other entities that seem to enjoy breaking the law and letting others pick up the fall-out).
The fact that Zenefits survived as a company is remarkable, but it makes their intentional deceit and fraud no less.
Zenefits sidestepped some occupational licensing rules that only exist to protect the market of incumbents.
Zenefits made fraudulent representations about the training of their sales representatives, many of who know diddly squat about what plans they were actually selling to companies, and Zenefits' incompetence in actually transcribing data across incompatible systems. This resulted in numerous employees being denied timely access/use of their benefits, including health insurance. Zenefits absolutely belongs in the same category as Theranos and both companies should thank their lucky stars that their executives aren't in prison cells right now.
There will always be unethical people willing to exploit the commons for their own profit. Some will do so spectacularly, as a society it's our job to ensure that the EROI of such actions are < 1x so that they will lose out in the long run. Elizabeth Holmes was thoroughly unethical and will spend the next decade paying for that choice, I doubt she profited from it.
On that front, why is no one at Theranos jailed and why is that company still operating?
Most folks think in terms of absolutes, like "good vs bad", "right vs wrong", "steve jobs", "freedom", "jail", "fraud", "ponzi scheme", "criminals", and don't care about the details. People are easily misled by labels, like "Little Marco" or "Lyin' Ted" or "Crooked Hillary"
All of a sudden people are calling Theranos a ponzi scheme when it's doing something completely different.
The CEO took a deal where they will be paying a fine and will not run a company for 10 years.
The President is fighting the charges.
Ambition shouldn't come at all costs.
If Silicon Valley as a whole is a bit scammy, it doesn’t serve us to defend it on that score or spin it into a positive.
It is illegal to misrepresent your financial statements, thus lying about the present. But you can sell whatever you want about your future, like changing the world and making it a better place and what have you. The Silicon Valley feature is the unbelievable faith in the potential of an idea/startup. I don't think lying about your current financials or KPIs is a feature.