Could A Tech Giant Build A Better Health Exchange? Maybe Not
npr.org
npr.org
http://en.wikipedia.org/wiki/Gall's_law
The problem with dealing with the government is that they have a long list of requirements that they absolutely must have. As a result, the programmers will work to satisfy all of those requirements, and will let the unwritten requirements (like, oh, "must not fall over when more than a dozen simultaneous users connect") lapse. Frequently with software you have to trade off requirements against each other; when you go to implement it you realize that what the customer actually wanted is self-contradictory. The essence of good product management is being able to make these trade-offs smartly.
One thing that all the big companies realize is that to build a large working system, you have to start with a small working system, and then evolve it so it keeps working at all points. Google doesn't dictate exactly how the product is going to turn out at the end - it sets a general direction and product statement, and lots of individual product managers or tech leads then work on figuring out the details.
Being handed a pile of requirements that are sloppy and contradictory and then being told they are non-negotiable is kind of like being asked to build a submarine out of swiss cheese. No amount of money or expertise can make that work.
As context, take a look at Alabama's RFP for their health exchange. [1] It's 286 pages long. A version of it was issued just a month earlier. And this is just the RFP -- I assume the product spec was longer, much more detailed, with changes issued right into summer 2013 or later.
[1] http://www.aldoi.gov/PDF/Consumers/FINAL-Alabama-HIX-RFP-v47...
Anything is better than this big design up front with a million must have requirements.
I also don't understand why every state has their own portal. There should be maybe at most half a dozen different teams/designs/systems. We could expand on the startup funding idea by having each state act like a venture capitalist. They hear pitches and invest in the teams and designs that are most promising each round.
All code from all participating startups should be some open source license, preferably a Public License, but a BSD-style would work too. This allows the creation of an ecosystem where all are allowed to borrow from one another.
I can see the headlines and complaints now. People will complain that you're forgetting about $EDGE_CASE, like pensioners who worked abroa/new unmarried mothers/mothers who divorced recently/young children with $RARE_DISABILITY/etc./etc.
No matter what you do, so photographic sympathy story will have been "delibrately left behind". No politican wants to sign off on that.
You can't win.
It isn't about the tech at all, it is about getting the contracts. To pick those 15 companies, you would still need a proposal system, a validation system, an auditing system. Because the scope would be small and doable, and 15 independent contracts going out -- tens of thousands would flood in with proposals. Now you created a brand new problem, now you can't even START the contract, because picking winners has become to onerous.
The reason government gets so bogged down is an attempt at fairness and legal constraints in how they work -- the problems are actually really complex, even for stuff that seems simple on the surface.
State and local governments should learn from this and other episodes ( our failed DMV upgrade from a few years back for instance ); that an RFP and a procurement process that eats 5% of the budget before anything actually happens is bound to betray the public trust.
While I'd like to think that there was an easy solution; say appointing an 'implementation czar' and requiring open source solutions be the default. I don't fool myself as to the likelihood of that working any better.
Governments are human nature magnified and focused and as such they have all the human frailties; not to mention that everyone involved is pushing and pulling in a different direction.
That said; I do believe that all government IT projects should be open sourcing everything they do or have done for them ( not the data, the functionality ) so that it is open to public review and provides the possibility for public improvement.
At the very least if that $43 Million had been spent on something that left a github repository behind; Oregonians would have something to show for that money...
The only reason the government has to change healthcare.gov is because it's an issue that republicans have put on TV to use against democrats. That's it. If it sucked, but wasn't in the news cycle (like everything else), then the government wouldn't do a damn thing about it.
Generally, it seems stupidity of an organization scales with the number of people therein, private or not. Even corporations let horrible products persist if they don't hear enough whining. Government just has some extra features that make it bad at projects like this.
BTW, as an Oregonian myself, I find it irritating that the site doesn't work but we are still being subjected to the truly awful Cover Oregon ads (see http://www.youtube.com/watch?v=Xv2UUcXCo9g).
The musician is Laura Gibson http://www.youtube.com/watch?v=Xv2UUcXCo9g and the song is very hopeful.
But, buying all that airtime and all those billboards... http://www.wweek.com/portland/article-21410-live_long_and_pr...
Definitely showed the risk of a marketing campaign too far in front of the product. Not to mention that a marketing campaign for something with so much pent up demand is kind of goofy.
The project managers are total PMP dopes that know fuck all about software engineering. People get promoted to those positions because they are political. Not because they know what they are doing. Back in my big corp life from 5-6 years ago, I witnessed several multi-million dollar train wrecks. They were the epitome of sunk costs fallacies.
Oracle and IBM both have great developers working on products that are core technology that both those companies sell, but their software engineering consulting and project management is atrocious. There is a reason why companies like Thoughtworks exist. TW and similar firms come in to clean up the messes started by the Oracles and IBMs of the world.
On the other hand, I've worked with tough clients that want a solution "for everything" and force the vendor into a rigid waterfall / BDUF but in the same time keep changing the requirements as they go.
But come on, for 41 million dollars you can develop it as static pages, that get submitted to mechanical turk and will have change to operate this for years... I don't get the math.
I'm in the wrong business, I should start applying for government contracts.
If you assume that each dev has a salary of $100k, they're probably costing the company $200k for benefits, office space, etc. so $40 mill pays for at most 200 years of dev time.
Add in managers and a profit margin and $40 mill probably buys you a team of fifty devs for two years.
That doesn't sound too absurd. They may have failed, but $40 million seems like the right ballpark for a healthcare exchange.
I think the difficulty is really in schema matching/mapping.
So for healthcare.gov to talk to Hospital Insurance Inc. it would actually talk to the Health Insurance Inc shim and THAT would talk to the third party.
Now you have a small(er) definable project that a team can deliver and that can be tested. 100 shims later and healthcare.gov can talk to anything because they all (appear to) use the same interface.
Some challenges with "Just use REST/JSON and then Bob's your uncle:"
1) You will probably not have the source code of the system you are integrating with, which is likely a mainframe system written in COBOL.
2) The engineers who built that system are retired or dead.
3) The company which they worked for is no longer operating / was sold 15 years ago to IBM and then reverse merged into a different unit 7 years ago / lost all of its records in a fire / etc.
4) Your sole documentation is the dogeared paper copy describing file formats for their sole input method, CSV files, which was current as of its publication date, before you were born.
5) Your sole documentation incorporates by reference four other documents which, ahem, yeah.
6) You have a working test suite which accurately models the production system in all respects. It is the production system. Please do not run the equivalent of Accounts.all.delete when cleaning up from your test runs. (The system won't stop you from trying to do that. No, really.)
7) Praise God, there is actually a bridge between the mainframe system and the Internet, so at least you don't have to reverse engineer wire protocols. It was coded in 1996 by IBM's crack team of integration engineers. This will give you an excellent opportunity to brush up on your Java 3.4.1. It speaks REST/JSON, as long as as you spell REST/JSON "XML". Good news, though: your time spent learning the CSV file format won't be wasted, since the XML is just a straight mapping to it. Except for the four bugs in the mapping, which are dutifully recorded in a database in Hyderabad whose existence will be exposed to you 8 months into the project.
Welcome to legacy integration! We hope you enjoy your stay.
First, you can forget about having documentation of any kind or anything as recent as XML.
Also...
8) Nearly everyone you need to interface with on the customer side is apathetic at best. More likely, they are plain adversarial for fear that the work you're doing will eliminate their jobs. A single person can plop themselves in your critical path and refuse to cooperate - millions of dollars or public safety be dammed - there is nothing you can do other than work around them.
9) You will need to coordinate with half a dozen other subs under the contract prime. Some of those subs will be unassailable due to pre-existing relationships. Others will depend on talent with a 10 hour time difference. One will suddenly be in charge of "testing" your code in order to generate more billable hours. You will be in a constant battle to avoid being used and abused by the prime to make up for the deficiencies of other subs.
My first job out of college involved connecting to various pieces of hospital equipment. The equipment connected to our gateway (the project I was assigned to work on) via RS-232 cables, communicating using a standard protocol known as HL7.
Standard cables, standard protocol -- what could possibly go wrong? The answer: Everything. Everyone interpreted RS-232 in whatever they wanted. Everyone ignored the parts of HL7 they disliked, and added things that were "obviously" important. Getting decent documentation was nearly impossible.
The bottom line is that software development is relatively easy when you're in a vacuum. The moment that you need to integrate with someone else's system, things get complex and difficult. And when you have to deal with several vendors, each of which has implemented a superset of a subset of the standard, things get even crazier.
When I was editing the student newspaper in college, our editing/layout computer system stopped talking to our typesetting system. The vendors blamed each other for failing to do the right thing. We had technicians come in from both companies, and they basically yelled at each other, saying that he was not implementing the standards correctly.
So yes, the Obamacare Web site will go down in software history as a cautionary tale. But if they still haven't gotten the back end to communicate with the individual insurance vendors, then I fear that the debugging process is far from complete. And claiming that interoperability is a Small Matter of Programming is easy to say when you haven't actually needed to do it.
I once worked for a company that was all "cutting edge". We built Digital Video On-Demand systems for airlines, hotels. Python, REST, JSON, media streaming. Gorgeous things. We were invited to respond to a RFP for the transit system of the city I lived in, for platform displays. Bear in mind, these systems had been in place since the seventies, and consisted of two CRTs side by side, and you could watch the image drawing line by line down the two screens like a GIF downloading over a 300bps modem (it would take up to 15 seconds to draw a screen).
Alas, this would have been so much nicer.
We prototyped a HD flatscreen system, all sorts of niceties, news, temperature, more detailed train, trip, other information... then we had to integrate things.
We -eventually- dug up a line printed (literally on that blue and white tractor dot matrix printer paper) "spec" for the message format for the PDP system hooked into their network.
Prior to that we were pulling and reverse engineering a raw serial on the wire binary protocol with messages. Oh, did I mention there was a primitive packet system written for this that would preface messages with a destination, so they could be multiplexed - i.e. you could get "Train Destination (for train 1), first stop, second stop", then "fifth stop (for train 2 on a completely different display, to be ignored)" and then back to your messages.
When was this? 2007.
This will give them a runway till take-off.