Contractors See Weeks of Work on Health Site
nytimes.com
nytimes.com
They could very well now find themselves in a position where it is humanly impossible to untangle it all and make even the smallest changes. I'm calling it failed at this point.
Edit: Title was changed from: Healthcare.gov "contains about 500 million lines of software code". This comment reflects this fact which seems unbelievable, but if true, I feel would likely mean that it was mostly created by code generators that probably copied and ossified original errors and misconceptions to the point that it is no longer possible to fix. Its a total failure, the money is gone, but they'll keep throwing even more after it, until the tiny, disjointed systems they develop in the mean time as "temporary solutions" eventually become the "real" system.
It's great that they can all fit in the Rose Garden!
Don't worry, the compiler will optimize it.
While "efficient" is not a word that comes to mind when thinking about the government, 500 lines of code would be 1.6 lines for every man, woman, and child in this country.
:)
So for example, if they used Java, that line count is including the lines of code found in the JDK.
I see no other way of getting to such a large number.
I kept reading the article and on the last you find this gem:
According to one specialist, the Web site contains about 500 million lines of software code. By comparison, a large bank’s computer system is typically about one-fifth that size.
That to me suggests the number is bogus, because it is so obviously a quote from someone who has no idea what they are talking about.
And while it's a poor metric, lines of code is not completely useless.
It's kind of weird to think that VS 2012 has more lines of code than Windows XP.
[0]: http://blogs.msdn.com/b/visualstudio/archive/2012/06/20/the-...
So you're saying that Contractors are under no obligation to meet standards? Or just that this contractor would be exempted? Or perhaps software in general would be exempted?
( Apparently NIST is involved somewhat : http://www.informationweek.com/government/enterprise-applica... )
Hell, when I worked at Amazon my manager used sprint story points as individual productivity metrics. Oh God.
Our industry doesn't have a consensus on much of anything, least of all anything regarding counting lines of code.
Maybe if they include all of the possible HTML the site could generate, but counting potential code seems perverse even beyond the usual cluelessness of the mass media when talking about software.
To go all the way up to opening day, have these problems and then offer that five million lines of code need to be redone is, well, I'm at a loss for words. I am trying to think about how badly one could fuck up five million lines of, say, Python. That's a lot of code. It's almost like saying that you set out to write a portal for the ACA and mistakenly wrote an navigation system for a new spacecraft.
In many ways this is great. I can't think of a better example of how bad big government can be and often is. It is hard to argue with what is unfolding before our very eyes: An attempt at centralized control that succeeded on the legislative front but is failing miserably in execution due to nothing less than incompetence at all levels. And, to boot, it is costing hundreds of millions and will probably cost billions if it doesn't implode under it's own mess.
Look at this in sharp contrast with what I happen to think would have been a far better approach: Free market decentralized healthcare. All they had to do is open interstate borders to launch all available companies into competition and slowly open other gates in order to promote free and open competition. Yes, judicious regulations would have been required here and there, but always with the goal of having an ecosystem that is as free as reasonably possible.
In this scenario at worst the government would launch a open-source projects to develop open and well-debated APIs for publishing and sharing all pertinent information. Various sites could then connect to any available insurance company through these standard API's in order to gather, present and sell insurance to anyone. Instead of having a badly executed centralized attempt to develop a portal you'd have thousands of entrepreneurs jump into the fray for an opportunity to offer the best possible services. Very soon the best-in-breed services would rise to the top and continually evolve better and better offerings.
Cheaper, faster, better. All three.
Instead we got this shit.
There was an article on here last week (I think, my memory might be failing to that end) interviewing some individuals who claimed to be former contractors for CGI Federal. They also claimed that CMS, the agency responsible for integration (with no integration experience) essentially demanded that they perform no testing on the system until about 5 days before October 1st. I don't know how true these claims are, but given the nature of this whole fiasco, it somehow doesn't seem all that surprising.
> Look at this in sharp contrast with what I happen to think would have been a far better approach: Free market decentralized healthcare.
Yeah, and the US government seems to favor regional monopolies that almost cleanly map to congressional districts. Maybe there's a correlation and someone's getting a kickback somewhere along the lines...
> Cheaper, faster, better. All three.
> Instead we got this shit.
More expensive, slower, worse.
I do have to wonder. If the government elects to go the route of a single payer system after they've largely screwed up the launch of this mess (and remember, ACA isn't fully implemented--the last bits of the law don't come into effect until 2018), is anyone is going to trust them at this point to do so?
What we are seeing right now (~.5B waste) is a drop in the bucket compared to waste in the US system as compared to socialized systems. We pay ~2x as much for comparable health care (per person per mean PPP-adjusted GDP/capita) [1]. That is about 250b/yr we are spending on a (more closely) free market system that keeps promising to be more efficient than its social counterparts and keeps failing to deliver, or even deliver in the same ballpark. I realize that HN is a site about code, but it's important to keep some perspective here. $500M (itself an inflated figure) is 0.2% of $250b. If it is hard to ague against $500M of waste, it surely must be nearly impossible to argue against $250b of yearly, recurring waste, no? How do you ignore these figures?!
If you're going to argue that our system makes up the difference with better outcomes, I want to see numbers. I happen to know that they don't support this argument, but I want you to do the search so that you can't dismiss my sources by assuming they were cherry-picked.
> I happen to think ... Free market decentralized healthcare ... would have been a far better approach
Of course, $250b/yr includes the cost of providing care as well as administration. Perhaps administration of a free market system is more efficient? Not when you compare the US (31% administrative overhead) to Canada (16.7%) or the UK (6%) [2], [3]. This isn't a surprising result, either: I recently listed 7 reasons why exchanges should be more efficient than current individual plans [4] and you can add the profit margin and advertising costs to that list to come up with 9 advantages a single-payer system would have over a free market health care system.
Markets are good at optimizing things that can be quantified, so perhaps a better comparison would be between medicare and private insurance (not overall administration, just the insurance part). Again, the government systems come out ahead: Medicare overhead is 3.6%, Canada's insurance-related overhead is 1.3%, and overhead among private US insurers is 11.7% [2].
> Cheaper, faster, better. All three.
[citation needed]
See [5] for a more complete overview of the pro-single-payer argument and a truckload of reputably-sourced studies and comparisons (as opposed to hand-wavey promises) to back it up.
[1] http://en.wikipedia.org/wiki/List_of_countries_by_total_heal...
[2] http://www.pnhp.org/single_payer_resources/administrative_wa...
[3] http://nhsrationing.org/tag/administrative-costs/
Lookup the history of laser eye surgery over the last, say, ten years.
Free markets work. That's irrefutable. The problem is we've never had a free market system when it comes to medicine.
Unfortunately, when it comes to policy we do not get to cherry-pick the conditions under which the rest of the system operates, and the other parts of the health care market run under conditions that demonstrably throw the free market for a loop (hence the system-level failures vs socialized systems I pointed out above). Cancer treatments cannot be paid for out of most people's savings. "Customers" do not always have the chance to shop around between providers (w/o insurance) and those that do use insurance experience a 99% deleveraging effect because only the 1% that have extraordinary costs are in a position to examine the value of what they bought. Do you have a proposed mechanism of breaking information asymmetries, re-leveraging disproportionately expensive consumers, and standardizing the coverage provided by free market plans? Do you have a way of pricing the 5 externalities I listed into the insurance providers' budgets? Has your proposal been implemented in a dozen other countries and proven to be comparatively cost effective for decades?
What is your answer to the figures that show that our system in aggregate is 2-10x less efficient than comparable social systems?
Anybody can cherry pick a counterexample. Anybody can speculatively extrapolate it out to a convenient conclusion. But the single-payer advocates can point to a mountain of directly relevant apples-to-apples evidence regarding the industry as a whole that does not require extrapolating across markets or cherry picking conditions / good years. To me, it appears to be a mountain of evidence vs a pebble. Do you really think the the implicit strength of extrapolation from the (very) special case of laser eye surgery carries weight against these empirical apples-to-apples comparisons?
I think an issue is that America has a system of government that lacks a long-term focus. Every 4 to 8 years, we get a new 40-100 year agenda. This leads to tinkering with these social programs and the original goals can get lost, muddled.
That, and our government is not structured very well to deliver services with a largely unaccountable bureaucracy overseen by a divided and paralyzed government. Our government in the process of failing to deliver a service right here (with the exchanges at healthcare.gov).
I could see single payer systems working well elsewhere. In the states, I think it would be implemented badly at 10x the cost. Just my opinion, but I think we should try developing a competitive market for healthcare and reduce government regulation in it and see how that goes. Then, when that flops, we can try out socialized medicine.
And anybody can do what you are doing. You have an obvious bias against free markets. That's fine. However, I will argue that, if you live in the US, almost everything around you is the product of free market forces to a large extent. Perfect free markets don't exist. That's a given. Please don't point out that this or that market isn't perfectly free. I get it.
Here's the difference with our views.
My view is centered around the idea that government should be limited to a very small area of responsibility and influence in our lives. I won't get into the details. Suffice it to say it would be a small fraction of what it is today, in rough strokes perhaps reducing it by 50 to 75% --gradually, over a period of 25 to 50 years.
Your view, if I read between the lines, feels like it is almost diametrically opposite mine. You don't have anything in your HN profile so I can't even attempt to understand where you might be coming from. For all I know you work for the government and are actually being paid by the current ACA/Obamacare "machine" in some form. I don't know.
I, well, I've been an entrepreneur since forever and fully understand what it means to live a life where you make sound fiscal, social, educational and personal decisions based on the idea that you --not the government-- should be responsible the well-being of your family and community. That's my frame of reference, not sure what yours might be.
Going back to the size of government, I would, for example, push to de-militarize our country over, say, 25 to 50 years and encourage others to do the same. We could and should take the lead in making this a better world. The military alone accounted for 25% of our federal budget. We don't need them. I know many think we do. We don't. The world needs to make supporting massive military forces a thing of history.
I would also eject government from most areas affecting our lives. Take the Department of Education as an example. The US keeps testing significantly sub-par when it compared to other less developed countries. Yet we spend more than almost anyone per pupil and we have a massive DoE. I don't think they are effective. Gone.
Anyhow, you can continue along these lines and remove government from all of these areas. Gradually. Over twenty-five to fifty years.
Are regulations required? Are they important? Yes. Of course. In certain areas and with moderation. That's a different topic. No, I am not advocating a wild-west guns-a-blazing free market. Controls are almost required in certain areas. This would be a very important function of a limited government.
When it comes to healthcare the free market approach would create efficient markets quickly due to competition. If I could go to Nevada and get heart surgery for less, with excellent quality and even a free slot machine in my room during recovery, hey, that's fantastic. California would have to find ways to compete. And they would. Technologies would be developed in order to make the medical business more efficient at all levels. Within a span of ten to twenty-five years the medical industry would be transformed into an incredibly efficient machine providing above-par services for everyone. No tax dollars or government involvement would be required outside of putting in place sensible and limited regulation in order to help, not hinder, the development of a useful free market.
Corner cases will exist. In other words, the good, the bad and the ugly exist in all markets. Free or government originated and controlled. That's human nature. Neither school of thought can guarantee that bad things will not happen. Stories abound about people in highly socialized medical systems having to wait months or years for treatment. The free market equivalent might be not having enough money or insurance to pay for something or horrible cases of medial malpractice.
How do we deal with the cost of ownership situation? I'd say much like we do with auto insurance. You own it. It's yours. It does not live and die with your employer. You get to insure yourself for anything you want and for as much as you want. I barely saw the inside of a doctor's office during my 20's. My guess is I would have been receptive to the idea of a low cost catastrophic health insurance policy with a relatively high deductible. If my auto insurance guy had said something like "Martin, for another $50 per month we can offer you catastrophic health insurance with a $10,000 deductible" my guess is I would have gone for it without much thought.
Should medical insurance be mandated? Yes and no. It would be really easy to setup a situation where, for example, if you want to drive a car you have to also have health insurance coverage. This is just an example I am pulling out of my hat and it is not well thought out. The intent here is to say that, yes, I understand that an idea market would benefit from wide-spread market participation. In this hypothetical nearly everyone 16 and over would purchase health insurance to some extent or another. That would take care of a huge percentage of the US population.
You would personally be responsible for the cost of your health care. In other words, if you go to a hospital and don't have insurance you have to pay the bills yourself. This is no different from going to the dealer with a car that broke down and no extended service warranty. This would setup a situation whereby everyone in the US would buy medical insurance, just like they buy car, homeowners and renters insurance. You know how they offer you insurance for your new TV or refrigerator at the appliance store. Well, al lot of people buy it.
If you were free to shop for health insurance from any provider I can easily see companies competing for your business everywhere. I could, for example, see State Farm and Alstate offering people various health insurance policies along with their homeowners or auto policies.
What to do about the really poor who can't afford any of it. We help them, of course. The details are not important here. We help them. Period. If you can't possibly pay for your insurance we, as a society, have a mechanism through which you will have it. This is where the idea deviates from the auto insurance model.
What to do about the whole pre-existing condition situation. Well, your insurance policy has an "uninsured motorists" element? Perhaps every insurance policy would have a nominal "pre-existing conditions" fee that we collectively contribute towards dealing with these costs. This would be the socially responsible way to deal with situations where you have someone who might require half a million dollars of annual healthcare bills per year due to their unique situation.
Oh, yes, all healthcare insurance should be 100% tax deductible.
All of this would work and evolve very well with minimal government intervention. Laws and regulations would be passed in order to serve the free market and reduce government involvement to a minimum, not hinder it.
The beauty of taking such approaches is that --due to the lack of set-in-stone laws-- we can always let observe them for a while and THEN decide whether they are working or not. The problem with passing legislation like the ACA/Obamacare is that you have 40,000 pages of legislation and rules that is convoluted, unmanageable and incomprehensible to such an extent that no citizen can even begin to comprehend it. And, to make things worst, we don't have the freedom to move away from it and modify it in an evolutionary and dynamic way. It is set in stone and it takes a tremendous amount of work to change it's path. I mean, they can't even get a medium complexity website to work correctly on launch. That should tell the entire story.
The most fundamental issue here is that it is nearly impossible to have dozens or hundreds of people to create legislation that is, for lack of a better term, perfect. It takes years, decades to painfully evolve these things. This is where free markets have an overwhelming advantage because they can evolve as fast as humanly possible. The power of entrepreneurship and free markets has been massively demonstrated by the explosion of this little thing we call the Internet.
Anyhow, we'll probably disagree. And that's OK. To each his own.
So where are the 12 countries of comparable levels of economic development that are operating with 1/2 to 1/10 the administrative overhead of the social systems? If it's so easy to support your argument with evidence, why are you arguing with philosophical generalities instead?
I understand how the free market is supposed to work and why it usually does work. I'm still a huge fan, contrary to what you read between the lines (I agree with you on deregulating education, for instance). More on that later. But something is amiss in the US health care market. The socialists are beating us. By a lot. Not overall -- I do not believe their higher taxes provide a better net return for someone like me, not by a long shot, and so I do not plan to move -- but it is possible to section out health care for purposes of comparison, and our system has a 2-3x worse value proposition along that dimension. We agree about the general principle that markets tend to have better self-corrective properties than rulebooks. Where we disagree is that I think healthcare is an exception. A glaring exception that we have invested trillions of dollars into trying to fit with our free-market-is-better worldview and still find that it's an exception.
What we already pay for medicare and medicaid would provide universal coverage if our system were as efficient as Canada's or the UK's (in terms of percent PPP adjusted GDP/capita spent on health care), so this isn't even a philosophical question about how much we want to spend on government-mandated charity. It's a question of getting value for the money we are already spending. You could postulate that the medicare/medicaid administration is spending money inefficiently, but two facts suggest otherwise. First, the overhead of medicare/medicaid administration is ~4% (vs 12-30% for private insurance), so the money isn't disappearing into the bureaucracy itself. Second, medicare/medicaid drive a harder bargain than almost any other insurance plan (ask your doctor next time you visit, the administrators do not like dealing with medicare/medicaid), so it's difficult to argue that medicare/medicaid are responsible for our inefficient health industry
We have two options:
1) Keep spending 20% of our GDP on health care (vs our neighbors at 10%) while we try different permutations of rules and hope to land on one that fixes the free-market feedback mechanisms and kicks it into gear.
2) Cut our losses and copy/paste the well-tested solution, using other countries as a template until we have health care at 10%GDP rather than 20%GDP.
Option #1 has been in the whack-a-mole stage for decades (the feedback mechanisms are broken by default, more on that later) and shows no concrete or theoretical signs of letting up. I'm for option #2. The experiment has been run, the results are in. Healing the feedback mechanisms in the health care market was a harder problem than it appeared. Saying that the next set of regulations or deregulations will make our health care market do a U-turn seems crazy to me, like saying that communism is the way of the future in 1990 while the USSR is busy collapsing. This time it's our turn to cut our losses, thankfully in a less traumatic manner.
> I can't even attempt to understand where you might be coming from.
MSCS student, ex-libertarian. I took a health care policy class in undergrad to satisfy a requirement and it changed my worldview. I had heard that the US health care market was inefficient, so I thought it would be a good place to look for opportunity. I thought that maybe I could use my data-fu to help make it better and possibly turn a profit while I was at it. The health care policy class seemed like a good place to get a 1000-ft overview of the industry.
As I learned more about the specifics of the health care market, I realized that the problems weren't due to a strange historical trajectory, barriers to entry, obsolete/misguided regulation, failure to adapt, monopoly/lock-in, or any of the challenges I was familiar with from the software world. Not fundamentally, at least. The usual problems have analogs in other countries, yet they're doing just fine compared to us. The "traditional problems" explain waste on the order of billions while the efficiency gap that needs to be explained is on the order of trillions (per year). I have already listed the explanations that I think best account for the gap, but to make a long story short: the health care market is the perfect storm of factors that frustrate the free-market optimization process.
> When it comes to healthcare the free market approach would create efficient markets quickly due to competition.
So why has this failed to happen? Our current system does not suffer from monopoly or regulatory capture. You are free to shop around between insurance plans, yet they operate at 3x the overhead of medicare/medicaid and 2x the entire (not just insurance) administrative overhead of the UK's NHS. What do you plan to change and why do you think it will succeed where past efforts have failed? I have an explanation for the failure, and it is not one that deregulation will fix:
Customer feedback through buy/no buy decisions is broken in the health insurance industry. In other industries that also aren't plagued by monopoly, dissatisfied customers push back by not buying a product. In the health care industry, a strange situation arises where customers don't know that they've been duped. The bulk of costs are concentrated in a handful of customers (those that get cancer, get in a terrible car crash, etc) so if an insurance policy figures out a way to dump the expensive patients, 99% of customers never realize that the piece of mind they have bought is worthless and they never demand better. This isn't hypothetical, this is precisely what happens: by design, private insurance doesn't actually shield one from the cost of a debilitating injury or illness, leading to the statistic that 62% of personal bankruptcies in the US were caused by medical problems even though 78% of those people had insurance [1]. The insurance companies make sure each person has just enough "responsibility" so that they can dump their own responsibility if it ever actually comes due.
Innovation in the insurance industry happens largely by finding new, clever ways of fooling people into thinking they are covered for something that they are not.
> How do we deal with that situation? I'd say much like we do with auto insurance. You own it. It's yours. It does not live and die with your employer.
Paying for insurance through your employer is a legitimate market innovation that decreases statistical, legal, and bandwagon-effect risk. Your employer provides two services to you: they aggregate your bargaining power with the rest of their employees, driving a better deal, and they decorrelate you with the specifics of the plan because you will take what they give you (bypassing the bandwagon effect).
This isn't speculative, the market has already spoken, and declared that individual insurance is more expensive (if you think otherwise, go form an individual insurance company and sell it for less). As a libertarian, you, of all people, should respect this. The PPACA's exchanges attempt to mimic the beneficial effects of an employer without going through an actual employer. I suspect it will work at least a little bit, after adjusting for the fact that we have sealed some of the loopholes individual plans used to dump sick people, which will necessarily increase costs. But I don't think we'll see administrative overhead fall even to 2x what we see in a typical social system, because most of the market problems are still there, and a single-payer system is still the limiting case of addressing the problems in [2] individually.
> What to do about the really poor who can't afford any of it. We help them, of course. The details are not important here. We help them. Period. If you can't possibly pay for your insurance we, as a society, have a mechanism through which you will have it.
This creates a discontinuity in marginal incentives (or reduction in marginal incentives if you pro-rate between income brackets) that single-payer doesn't, in addition to retaining all the problems of small plans listed in [2].
> we can always let observe them for a while and THEN decide
The cost of continuing to experiment with the free-market solution is 45,000 deaths [3] and $1.3T every year ([4], assuming we could figure out how to emulate one of the ~16 single-payer systems ahead of us in health rankings [5] right now).
I would not call this "beautiful." Health care just isn't as compatible with the free market as the internet. We've tried extraordinarily hard to make it work but it hasn't happened yet. A better analogy for free market health care would be the Hurd, scaled to $2.7T/yr. It's a beautiful vision, but there are some major architectural and compatibility issues that have been proven to separate concept from reality, all while there's a perfectly serviceable Linux kernel available that is
* 2x as fast <--> costs 1/2 as much
* less buggy <--> USA is dead last in top 17 health rankings [5]
* more compatible <--> doesn't pay 2x for drugs because of size disparities when bargaining w/drug companies
* has a smaller memory footprint <--> observed lower administrative overhead in medicare vs private, universal vs private [6]
[1] http://www.businessweek.com/bwdaily/dnflash/content/jun2009/...
[2] https://news.ycombinator.com/item?id=6582753
[3] http://news.harvard.edu/gazette/story/2009/09/new-study-find...
[4] http://en.wikipedia.org/wiki/List_of_countries_by_total_heal...
[5] http://www.theatlantic.com/health/archive/2013/01/new-health...
>> I can't even attempt to understand where you might be coming from.
> MSCS student, ex-libertarian. I took a health care policy class in undergrad to satisfy a requirement and it changed my worldview.
Ah. OK. I get it now.
To be clear. I am not attacking you personally but rather making a comment on the frame of reference from which you are basing your views.
First. Academia is chock full of socialist extremists. Not sure how well you are/where able to see through this if present and filter it out.
Second. Don't think for a minute that a class such as that actually reflects reality without getting out to the real world. In other words, trust but verify.
Finally. You are wrong about such things as medicare being wonderful. My wife is a doctor. I won't go into some of what I've learned other than to say that people are suffering horribly due to the failings of that system.
In looking at our system versus other you seem to be ignoring the effects our litigious society has on the cost of, well, everything. I won't go into what we are paying for malpractice insurance (without even the semblance of a negative mark on her entire professional history).
You are correct that the academic frame of reference played a large role in developing my worldview, but bias swings both ways. I think it's telling that I am able to back my arguments with specific measurements and comparisons while you resort to philosophical generalities.
Economic incentives in the software industry tend to match competence, effort, and risk-taking with just reward, at least to some degree. Developers tend towards libertarianism because they and their associates are able to make the market work for them. Favorable economic conditions shelter them from market failures (health care costs 2x as much? Eh, it was a benefit anyway, and a small fraction of a typical yearly salary in any case).
Unfortunately for academics, the good they produce is neither excludable nor easily measurable, leading to systemic undervaluation by the market (I'm not talking about CS research here so much as the hard sciences). They are more directly exposed to the market's hairy underbelly and tend to be more skeptical of the whole endeavor because in their day to day lives they and their associates have a much more difficult time coupling value creation to market-provided reward.
> Don't think for a minute that a class such as that actually reflects reality without getting out to the real world. In other words, trust but verify.
Don't think for a minute that the tech/startup scene (to include your employment experience and entrepreneurial activities) actually reflects the bulk of America or the world. Your real world experience is far too narrow to be relevant to the issue at hand. You cannot decide how to manage a forest by staring at a blade of grass. As an in-demand professional (married to another in-demand professional) your experiences are not only nonrepresentative but provably atypical. You need numbers, not self-gratifying philosophical principles to see past this bias.
> You are wrong about such things as medicare being wonderful.
I never said that. I think our hybrid system is a monstrosity.
What I did say was that medicare had low administrative overhead compared to private insurance plans, defined as E(1-cost/payout). In other words, private plans are provably failing to deliver on their promise of greater efficiency by a factor of 4 (and these are the efficient employer-aggregated plans). I also said medicare drove a hard bargain as compared to private plans, a claim which I can substantiate [1], in order to head off a potential attempt to shuffle the blame for the US's inflated costs onto medicare.
The hybrid system is a monstrosity, but I conclude that the private half is dragging down the public half, not vice-versa. My reasons:
* International comparisons (universal systems are beating us by a factor of 2 at cost metrics and matching our quality metrics even if you ignore the ~50M people our system dumps entirely)
* Explicit reasoning about why the health care market is a rat's nest of edge cases for the free market [3]
* Internal comparisons between medicare and private insurance (the 2 reasons I listed which you straw-manned into "medicare is wonderful")
> you seem to be ignoring the effects our litigious society has on the cost of, well, everything
Because malpractice insurance accounts for <2% health care expenditures [2] while the cost inflation (relative to socialized systems) that needs to be explained is on the order of 50%. I could not be bothered to mention these figures because I knew them to be small, while you apparently could not be bothered to even look them up, which unfortunately didn't stop you from trying to bluff me with certainty that the effects were large enough to play a causal role.
Doctors disproportionately care about litigious waste just like HNers disproportionately care about ACA exchange contracting waste. Both concerns hide the bigger picture. As bad as these two problems are, they are tiny compared to the macroeconomic problems. Billions vs trillions.
[1] http://www.californiahealthline.org/articles/2013/10/15/stud...
I propose they give the project to the HN community, divide up the $400 million figure amongst the participants and give us a weekend and I bet we'll have more to show than that of the so-called developers the government hired.
I am not underestimating the complexity of such a site, but seriously, it's not like it hasn't been done elsewhere to a degree. if I can do Internet banking, the a site for providing healthcare can be built a lot easier. They're both of similar scale and security requirements I would imagine.
This site seems like the greatest scam ever perpetuated. I bet the contractors are laughing themselves to the bank literally as we speak. Even though it's bad publicity, money is a higher priority than credibility for some businesses out there it would seem. Is it any wonder people don't trust the government? They can't even hire competent people to build them a website. This seems to be a recurring theme for local councils and governments elsewhere as well, so it's not just the US this is happening.
People need to get mad about this. That's $400 million dollars that could have been invested elsewhere like medical research, charities, food programs, rehabilitation for the homeless... A $400 million mistake has been made, and another $400 million will probably be spent before it's even ready.
What a waste of money.
> I bet not even mission-critical code written to power NASA spacecraft and satellites are anywhere near that length.
That doesn't seem like a good example. You want to keep the code in those kinds of things to a minimum, and that's probably not too hard.
Optimizing for readability is usually most important.
Readability is usually more important than reusability, in fact, although readability very often enables reusability.
But we're talking about mission-critical embedded code, which is a special case in many respects. Still, making it as readable as possible makes bugs easier to find.
Reuse is an emergent property of code, though. It's very tough to engineer for reuse upfront and have that effort be successful. It's best to keep your code simple and put your effort into making it well-tested. Then, over time spot the commonalities in everyone's code, and factor out the common bits to become reusable code if everyone agrees it's worth it to do so.
I wish there were a book about how to READ software. (If anyone knows of one, please speak up.)
One tip I picked up on my own over time is when you're looking at an unfamiliar project, don't read the code like a book, read it like a program. In other words, try to find main() and trace how things work from the bottom up, rather than trying to understand it based on how the authors have broken up the code into directories. I don't know yet whether this is truly good advice.
I guess I also suspect that the code for satellites and space shuttles can be kept reasonably compact. I mean, these systems are (ideally) not designed to be everything for everybody, unlike Linux or Windows or KDE or Visual Studio or Firefox or what have you.
Here's an example: http://spinroot.com/gerard/pdf/P10.pdf
And another: http://lars-lab.jpl.nasa.gov/JPL_Coding_Standard_C.pdf
Dunno how closely these are followed in day-to-day operations, but they clearly put some thought into it.
It is too late to get mad about any new/rising costs related to the ACA. It is here to stay and might as well be Medicare 2. "Law of the land" as the president is fond of saying.
Here's a fascinating visio diagram that shows just how much government has grown under the ACA. Anything red indicates new or expanded government:
http://www.jec.senate.gov/republicans/public/index.cfm?a=Fil...
Seems like the ACA created very painful integrations between federal systems and insurers, asking them to do many new things. Not a big surprise they're having problems. Some federal IT systems are archaic and integrations/change are costly and fraught with danger. The insider stories I've read about the Soc Security mainframe-based system are frightening!
Woe be to the phalanx of ACA consultants that has to integrate with those systems... :/
Reading this article, which at the highest level tells us the people in charge, the White House and HSS's CMMS (all government workers, the latter the integrator of the project) are still discussing how to fix this mess, hoping to have a plan by Thursday, combined with e.g. CMMS delivering changes to the contractors through the week before launch, and per this article "In the last 10 months alone, government documents show, officials modified hardware and software requirements for the exchange seven times.", suggests to me it could easily be years before the exchange works, and possibly never. We've seen this before (FAA, FBI, IRS, etc.).
All depends on CMMS (non-specialist civil servants and perhaps political appointees) learning big project management "on the job" or the vast majority of them getting replaced by an integrator who has a clue, either of which will take months.
That's just what's required to get sane high level management of the project.
In the meanwhile, can the individual parts done by different contractors get some serious progress? Well, first it depends on CMMS freezing requirements, and doing a good enough job of deciding what requires a total rewrite. Per this article:
"Quality Software Services Inc., or Q.S.S.I., a unit of the UnitedHealth Group, developed the identity management system, another major component that allowed consumers to register and establish accounts.
The identity management system from Q.S.S.I., which also taps into government databases to retrieve users’ personal information, was a particular source of trouble when the exchange opened. Change orders show that on Oct. 4 — after millions of people had been trapped in technological loops trying merely to log in — the government asked CGI to help it devise a new identity management system to replace the one provided by Q.S.S.I. But specialists said that approach was abandoned as too risky. Ultimately it was decided to fix the current identity system."
And we've heard from a HN member who was involved in a big university project that Oracle's identity software is fairly gnarly to get working. Which I have absolutely no trouble believing, because the underlying problem is gnarly.
But one wonders just how much panic was behind CMMS's idea of punting the existing one 3 days after launch. And you know the managers of both contractors had to spend a lot of time working with CMMS on just this detail ... working with people who are simply not (very) experienced in software development and management.
(Ask for my worst experience working for a government for the level of cluelessness that's likely involved here.)
So, getting to my point, you may say "It is here to stay", but "You Can't Fight Mother Nature" or the hard reality that computers do exactly what you tell them to do, not what you want them to do. How many months or years do you think people will stand for being fined by the IRS for not enrolling when they can't? How many will die because they've lost their existing coverage due to the ACA changes but can't sign up for anything, or anything they can afford without the subsidies which only the exchanges can (now) deliver? What happens if this not entirely nonfunctional system develops an adverse selection death spiral?
That's where only the very dedicated, the ones who really need a lot of healthcare, jump through the hoops, whereas the healthy 20 somethings and beyond, who are asked to do a 30 step (!) application on an unreliable system, punt, pay the fine but not the much larger sums needed to subsidize the 50-64 age bracket. Which raises the prices of the policies offered, which further decreases enrollment....
I just read between the lines when I hear "law of the land" and I think there's NO way the administration is going to back down here. It is going to be interesting to see how things go down. Perhaps they'll just hire human agents to take the place of the broken site?
Trying to do 7 million identity checks manually? Well, maybe something can be kludged to speed that up. On the other hand subsidy calculations had better be done by computer, and that's not working reliably right now; it could probably be Tiger Teamed into something that works and at scale, a simple calculator (heck, I bet it could be done in Javascript).
But again I get to the biggest problem of management. Starting at the very top, the CEO of this organization (Obama, per his Rose Garden event earlier today), they're in (public) denial of how bad the problem. In part they can't help it, they aren't experienced IT people, they aren't like us who can look at less than 1,000 words of clues and know "it's fucked".
Really salvaging anything out of this needs decisiveness. E.g. to get your kludge working for 2014---and again I note the critical detail of subsidies, especially since the only plan that can now be offered is gold-plated (e.g. coverage of your children through age 26) and inherently very expensive even before you get to community rating (everyone under 50 is to pay a bundle to cover the 50-64 set)---someone would have to decide to start it now. "Wastefully", in parallel with trying to get the site to work. As it is, millions are losing their old-fashioned high deductible major medical polices as of Jan 1st, things start getting really bad for a lot of people really soon.
If this nightmare drags out to 2016 (which today we can't discount, and that's even counting the website or its substitutes vaguely working), do you really expect it to be the law of the land by 2018?
I find some of it amusing - I've worked on some enterprise-level projects with business systems talking to other business systems, and I can only imagine how much eye-rolling is going on among the programmers that are actually trying to do the work. This is a project where some very visible, loud people have a vested interest in it looking bad even if it were flawless.
I actually think it would be kind of fun to be close enough to the technology to really identify the core constraints and be in a position to help alleviate them.
Certainly the U.K. NHS experience shows the running of such a system is political. I don't watch it closely, but as I understand it Tony Blair/New Labour doubled spending, but somehow functionaries and bureaucrats got the new money, not the front line people and institutions.
You think we can have anything less than a totally political discussion about euthanasia, most recently known as the Liverpool Care Pathway in the NHS?
Are any of the managers in the White House, HHS and CMMS in that category? Certainly not loud (nobody is speaking on the record: https://news.ycombinator.com/item?id=6586119), but perhaps anti-Obamacare moles?
Because the people running this show every sign of being true believers, and I seriously doubt they're playing much attention to the "waco birds" who are totally against it.
And we could more fruitfully spend our time contributing to the automated test pack.
No possible way. For a website. Did they write their own operating system from scratch? And database?
500 million lines of code would represent nearly every line of code that has ever gone into all shipped Microsoft software spanning 37 years.
But 500M is not credible, unless you are doing things like counting auto-generated code and open source libraries. Even then, you would have to be using 100,000 open source libraries to get near 500M LOC :)
> the separately designed databases and pieces of software from 55 contractors work together.
Factor in dealing with the multitude of state laws and companies they are interacting with, and you have a recipe for problems.
A lot of armchair quarterback technologists, but not a lot of precedent to compare computerizing a healthcare system in this way to.
> One specialist said that as many as five million lines of software code may need to be rewritten before the Web site runs properly.
Oh, so only 1%!
Tangentially, why are so few quotes in this article sourced? I see no reason why they shouldn't be. It's all throughout the article... They keep quoting "specialists."
"Stan Z. Soloway, president and CEO of the trade association that represents hundreds of government service companies" who parsed the contracts and independently confirmed that there's no prime integrator (i.e. the HHS CMMS is doing/failing that job, https://news.ycombinator.com/item?id=6569042).
One or more big insurance company CEOs; they're the ones who have to deal with the gubble the backend is delivering to them.
Various mostly bloviating politicians like John McCain.
Even yesterday's HHS blog posting (http://www.hhs.gov/digitalstrategy/blog/2013/10/making-healt...) is unsigned.
You know, "war on whistle-blowers" and all.
The NYT is a left-leaning publication. This seems to be a transparent attempt to try to soften the blow to the Obama administration's image by saying that implementing healthcare.gov was a Herculean task that no one could have gotten right on the first try because it required a half billion lines of code. It's no coincidence that this article emerged the day before Obama himself plans a press conference on this very issue.
Yikes! I don't think these "experts" are going to be fixing this anytime soon.
We really need a public peer review system for technical contracting of government projects. This is getting insane. These schmucks fleeced 500 million from our government (us) for a barely functioning website, and now some snake oil contractors are going to come in to "fix it" and fleece us for even more.
I can't imagine any way the contractors could have delivered a working system under the constraints we've been hearing, including most especially the unqualified Federal HHS CMMS government bureaucrats taking upon themselves the role of integrator of the 50+ parts of this system.
For that matter many people at all levels of this have been waving red flags, which the people running the show have studiously ignored. I mean, you know someone told them "you've got to freeze the requirements and give the programmers some time to get the job finished", not to mention "it needs more than one week of testing"....
Some of the remedial steps we're hearing about, like monitors for system bottlenecks, sound to me like things the integrator should have made part of the system from the beginning....
I feel like I did underestimate the potential costs of this software. How much, I couldn't say, but thinking about it this morning I realized that they probably had to integrate with the systems of (every|many) major existing healthcare provider in the country to actually make this work as desired. I can see now where that wouldn't be cheap.
Is it 500 million expensive? I'm not sure I'm qualified to make that assessment, and now that the initial shock has worn off I'm feeling okay with that. I'd need to see more data - particularly, the balance sheets for the project - before I could say for certain whether the cost was justified. I'd be interested to see that data, but as it stands I'll settle for chilling out about it unless more information comes out along those lines.
"but thinking about it this morning I realized that they probably had to integrate with the systems of (every|many) major existing healthcare provider in the country to actually make this work as desired. I can see now where that wouldn't be cheap."
Actually, that's one part that shouldn't have been expensive for the government, except in support services. As in they should have standard, sane input formats for policy details and output for enrollees that any participating insurer could adapt their systems to. IT must be a core competency for a successful insurer, since everything? they do aside from marketing is IT mediated.
There's been some reported troubles with getting the policy details right, although I think what I heard was about one or more state exchanges---with guaranteed issue and community rating it's rather more simple than normal, but then again a customer's FICO score can and I gather generally/always is used in the pricing. At the other end the insurers aren't reporting trouble in getting and parsing the data, just in it being self-evidently incorrect, e.g. some customers being reported as enrolling and canceling several times. Which they're addressing, while the volume is so low, by calling them up (everyone wants to complete the circle anyway).
On the other hand, you know it took a while for everyone to hammer out those formats, especially the input one. The output should be pretty simple, just needs to have all the particulars correct. E.g it would be a bad sign if an insurer was getting people enrolled in a policy they didn't offer.
I was not thinking rationally about this whole issue when it first came up and thus didn't measure my previous experience when I was thinking about the issue, but I actually have worked with healthcare systems integration before. Even with all government inefficiency/management aside (we were a private company), interfacing with healthcare providers is an extremely complicated and painful business.
For instance, there is a HIPAA standard interface protocol for things like Eligibility and Benefits and Claim Submission, and most (though still not all!) health care plans have some sort of system through which you can interface for that information - though often it's hidden behind miles of red tape and fundamentally different implementations for various custom fields, and different interpretations of the meaning of many fields. That alone is painful, and that part is supposed to be "standardized".
I can't even imagine the state of healthcare provider's policy estimation systems, particularly when it comes to interfacing with them. It's speculation on my part, but I imagine that each healthcare provider would likely have to have their estimation systems tied into. I wonder how many even had it exposed in such a way at the start of this project that you could tie into it, and how many of them had a wide variety of custom inputs to take into account in their estimating. Though you are right that perhaps the new law allowed/forced them to simplify this estimation process, in which case each of these providers likely had to modify the ruleset of their ancient systems before the integrators could actually work with it. Yeah, I could see even a simplified ruleset for policy pricing and eligibility as being a major factor in slowing down this process.
I certainly wouldn't have wanted to be a software developer on this project, thinking more about how bureaucratic and inefficient both the government and healthcare providers are. I still feel like, even then, 500 million is a dubious number - and I'd like to see where that money went. (how many people were employed on the project and for what cost, what were they being charged for hardware and datacenter(s), etc...) But as a software engineer, I would certainly not have enjoyed working on such a doomed project regardless of whether the money was as well spent as possible or if it was lining the pockets of those further up the chain.
Who is the contractor responsible for this? Perhaps the same people responsible for the Ontario EHealth scandal:
http://www.cbc.ca/news/canada/toronto/ehealth-scandal-a-1b-w...
I don't know if you were being facetious, but you're not far off:
http://m.washingtonpost.com/blogs/wonkblog/wp/2013/10/16/mee...
Chao said that he’d once held high hopes that the exchanges would run smoothly from the beginning, but that those hopes had been dashed. “The time for debating about the size of the text on the screen, or the color, or is it a world-class user experience, that’s what we used to talk about two years ago,” said Chao. “Let’s just make sure it’s not a third-world experience.”
http://www.forbes.com/sites/theapothecary/2013/03/22/cms-on-...
I'm so sick of seeing contractors fleece our government (and by extent, us) by pulling the wool over the eyes of those who don't know any better and making sweetheart arrangements.
I point out there's absolutely no way the contractors could have won with the insane management they were under, unqualified CMMS government bureaucrats doing the integration and constantly requiring changes, through the week before launch, some of which is detailed in this article.
I've tried a dozen times over the span of several weeks, and get to a new impassable failure state each time. I'm literally sent to broken URL's that have variable names "?id=%applicationID%" in the URL instead of the actual value. Sometimes clicking "next" just simply refreshed the page with no new content.
At least, that sounds like a great way to charge gubbmints by LoC.
One specialist said that as many as five million lines of software code may need to be rewritten before the Web site runs properly.
It sounds like they've been doing a little too much coding and not enough designing.
I like the idea of a government healthcare program, I really do. But this website... man, someone lined their pockets on this scam, big time.
Unless you want to posit that the contracts were accepted by companies that knew it could not be delivered in the time required. And even then, people are optimistic, and I strongly suspect they didn't think HHS/CMMS would be so insane about the number of changes demanded during the process, going all the way through the week before launch (!!!).
In comparison:
Windows 95 had 15 million lines of code.
Windows 98, 18 million
Windows XP, 35 million
Luckily they have about 8 weeks.
And I agree with the people who think that 500M lines of code is unlikely. Judging by some of the statistics that were cited in other comments (e.g., "Windows XP was 45 million lines of code"), this may be more lines of code than Microsoft has written since the company was founded (many of the lines of code in XP were originally from NT, etc.).
I feel a bit dirty for having to explain such simple things. All the armchair pundits here are declaring 'how can they screw up a simple website'. Nobody here (except if there are people reading this who actually worked on this project) knows how hard this project is, and how much integration is required. Personally, if somebody had asked me to make a quote for a project this size a year ago, and going on the information we have now, I would have said that it's impossible to build something this big and this complex within a year. Can't be done. IT systems of tax administrations of small countries that do less than this system have cost 10 years to get to a stable, workable state. How can anyone build this within a year?
Anyone have detail on language they used and/or frameworks(s)?
Someone should be held liable for this. They just wasted tax payers money on would be experts and have to spend more tax payers money to fix it. Disaster!
I strongly suspect that somebody is full of shit.