This Obamacare contractor doesn’t take security seriously
washingtonpost.com
washingtonpost.com
Coming from the construction industry, there isn't a contractor in the world that cares about things which aren't in the contract. Implementing features which the client has not asked for, in the design-bid-build world of public-sector contracting, just means you will lose the bid to a lower bidder who won't do those extras.
Well, you'd think so, which is why that's essentially already mandated:
http://en.wikipedia.org/wiki/Federal_Information_Security_Ma...
http://en.wikipedia.org/wiki/Federal_Information_Processing_...
So if I had to guess, if they didn't protect the data then it's likely those controls were notin place everyone knew they were not and the agency heads said it was ok. All fed systems get tested, and the system certifier is supposed to be an impartial reporter of the facts (these are the vulns, missing controls, risks, etc. and they are almost always impartial). They report those to the DAA along with a recommendation to approve or not approve the system to operate. Given how important this system was, I highly doubt anyone would not have issued the ATO no matter what vulns it had. And that's not to pick on this system, sadly slap dash rush ATOs happen far too often in the USG.
If you spend a ton of time and money on a system, execs in the USG have their careers on the line to show that wasn't wasted and since they don't understand security it all seems like science fiction to them. They almost always ATO the system. And why not, no USG exec has ever been fired for having a poorly protected system broken into.
growupkids: Which is also true in government contracting.
"Public-sector" = "government".
In the construction industry, a contract might not explicitly say that the building has to comply with local building, fire and electrical codes - that's taken as a given. To construct a web site that deals with sensitive information without taking security into account is like building a skyscraper with no fire exits - not something that any honest or competent builder would do.
You can, and people do, and they are cited for it when caught.
Likewise, there are mandatory standards for Federal IT systems, particularly in the area of security.
Relying on the honesty or competency of your contractors go above & beyond their requirements is an inefficient and unsustainable solution in a free market. This is a clear case of Moral Hazard: In the same way that bad money drives out good, corner-cutting organizations out-compete honest ones.
That's because you have to have a license to perform that work. If you don't meet the code, you don't keep your license.
This is one of the reasons why McConnell et al have argued that we need professional licensure. Licensed Professional Engineers can lose their licenses if they sign off on half-assed plans.
In the public construction industry, every contract includes that language. It's superfluous, because no contractor that wants to keep his license violates those codes (all construction is inspected by local authorities).
Furthermore, in construction, those codes exist and are easily incorporated into contracts by reference, even non specifically ("all work will comply with local building codes and statutes"). Also, the liability for non-compliance, even if undetected by inspectors, is massive.
In the tech industry, I've seen "work will be performed to professional standards and best practices" and other such meaningless clauses, intended to give the contractee some standing for complaint if they aren't happy later, but it's hard or impossible to measure compliance to generic standards that mostly don't exist.
This is one reason why Engineers say that dorking around on computers isn't Engineering. They do have a point.
FIPS actually exist and are easily incorporated into contracts by reference.
Imagine a bank (well just an account), on the functional level, you just need to deposit and withdraw money. The fact that they have to secure it is a huge constraint, but at the functional level we're not far from a trash can, a parking lot or a gas tank.
I work in the information security industry--primarily focused on healthcare security--and you'd be amazed and appalled at the lengths that CIOs will go to say that they "don't need to be HIPAA compliant."
More and more, though--especially as high-profile incidents become more widely reported--security is starting to become an accepted norm. There's still not much budget for it, and it's still considered an inconvenience, but at least people recognize that spending some preventative dollars can save a significant amount in reactive responses (DFIR, legal fees, etc.).
The HN crowd might be interested to know that the worst security offenders I've seen (even worse than old legacy systems) are software startups. Why? Slogans like "Ship First," or agile development cycles that don't include QA/security testing as part of a meaningful SDLC. What good is shipping something every two weeks, if one of those releases loses your customer database?
My latest drive is to help organizations realize that security is a feature, not just an inconvenience. Once that idea is more commonly accepted, we'll all be better off.
If that's the case then shouldn't lawmakers, being lawmakers, pass laws so that future government projects include a clause in the contractual arrangement? Isn't that just common sense?
Seems like this is a trend with the US government, they try to find others to blame instead of just fixing the damn issue. They had this massive set of hearings about people abusing military disabilities (e.g. claiming for disabilities obtained before joining the US military) but didn't actually, you know, fix the freaking issue -- or even try to. They just berated the people being advantaged from it.
This is for two reasons:
1) Fixing something makes it appear, politically, like it's your fault, or at least savvy opposing politicians can spin it that way.
2) The incentive to fix the issue <= incentive to shift the blame. If one party points their fingers, the other must point back, lest the public think the first party is right. Shifting blame is easy, one can do it without working too hard, and it can fuel public debate which forgets about the original issue.
Many politicians care less about fixing problems and more about getting/staying elected.
They already passed a law creating mandatory security requirements for Federal IT systems, whether contracted out or built in house or any combination. [1]
[1] http://en.wikipedia.org/wiki/Federal_Information_Security_Ma...
A. The government wants all your data available to them. B. The government does not want all your data available to the public.
A != B
Edit: Not to come off as rude, I just tend to agree.
If you want to say that in that sense, the government is being hypocritical, perhaps that's true. At the same time, though, their goal here is (hopefully) to keep your data safe from outsiders, whereas we can "trust" them to be good with the data they backdoor from companies ;)
Pretty much every government IT projects I've seen fail, that's most of them here in Denmark, has done so because of politics, laws and regulations. The thing is: the big contractors like CGI, aren't going to turn down the money. They are only to happy to deliver a poor product because they are paid millions and never asked to account for the failures of these projects.
It would be better for the industry as a whole to say NO to projects we know will fail. I don't for a second believe that CGI software developers thought that they could make healthcare.gov a massive success, or even just plain decent. If they couldn't see that, then they shouldn't be doing software development in the first place.
Doing a postmortem this way is refreshing, except you'll never get the right answer: "With the time available (not the money) this was never going to work. The integration with legacy system and the shear amount of these integration points makes this an almost impossible project". In reality will get a lot of blame allocation and CGI won't be made accountable for accepting a project that should have be rejected by anyone with half a brain.
That CMS, responsible for integration testing, would delay that until 1-2 weeks before launch (see above change orders problem), see the tests failed, and launched anyway.
Well, I'm actually sure CGI wasn't surprised by the latter, but, still, I can't blame them for any of the above. The White House -> HHS -> CMS, all insufficiently experienced in running a major development project, turned out to be a customer from hell. Heck, given how important this site is, a key part of Obama's legacy, I and many others assumed the Administration would treat it with appropriate seriousness. We were wrong.
When the second or third of the changes happens, CGI should have told them "NO, this cannot be done professionally" and hand them their money back.
Did you notice how obsequious CGI's VP Cheryl Campbell was when asked the usual leading questions about the bureaucrats and political appointees running the show? This bit from the hearings is really telling (http://www.washingtonpost.com/politics/house-panel-grills-co...):
"Later, Rep. Leonard Lance (R-N.J.) asked the contractors whether they could conceive of “a more incompetent administrator” than CMS.
“I have no opinion on that,” Campbell replied."
Might be a lie, might be she's been jaded by so many customers from hell she really can't precisely rate CMS....
I'd also note that it wasn't necessairly the 2nd or 3rd change that were fatal, while the late ones clearly were. I'd further note that a lot of us are optimists, believing we can perserve in the face of tough situations including clients from hell, and we can take that too far when things become manifestly impossible.
It's also possible that CGI et. al. believed, certainly hoped, that when the first integration tests in the last 1-2 weeks before the scheduled launch resulted in total failures the administration would admit reality and delay opening a site that they knew was going to fail hard. I mean, this steady dribble of bad news is a zillion times worse than a one time hit in a program that's had a zillion delays and waivers already.
I guess I'm saying I'm not sure CGI could accurately gauge the depths of stupidity of this client prior to the most insane decision to turn on the known failing system.
Someone in the room retorted with "we didn't hear an apology when the government shut down either."
(You'd expect someone to get timeout for this kind of pettiness.)
With that said, one of the obvious problems was blame shift, as this article points out. The guy was (I don't remember this particular interchange) most likely outlining the fact that the question being asked of him wasn't relevant to "his responsibilities" in the contract. However, when responsibilities of a project are distributed, things fall through the gap.
This article is a bit too short to give enough space for the subject, but I think it does point out the most important issue - no one really had full control of the project.
What I saw was a small minority of politicians that seemed to have a handle on some of the technological challenges at-hand. Some politicians used their time to score political points completely unrelated to the technology issues - basically "our party is good / your party is bad" idiocy. Some politicians read prepared smart-sounding questions which they wouldn't be able to rephrase in another-way if they were asked to. One politician asked for a copy of the server logs so that Congress could get to the bottom of what was going wrong. And none of the company representatives were willing to accept that they'd done much wrong - instead taking every opportunity to blame people or entities not in the room.
No need to worry, folks! Your government is on top of things.
I wonder, how many Congress idiots does it take to fix the caching policy on a web server?
While I am pretty sure that most congressmen do not get it, but there are some very qualified and awesome technical people who work in the Government. Probably, someone has to have the balls and call it open source one fine day.
Generalization to open source everything that the Government does, or taxpayers fund, might not be such a great idea. There are some projects which are supposed to be secret (Like a NSA that's focused only on anti-socials) because they give our Government an edge over others.
Probably a competent agency with an oversight to decide on open/closed aspect can help, but the subject needs much longer and detailed debate. IMO, healthcare, education, civil investigations are perfect examples where we can go completely open source. Or get disrupted the valley way.
Requirements came in so late that development didn't even start until March of this year, and requirements were changing right up until the week before launch. Only one week of full integration testing was performed.
While open source should be used in government IT, that's not the main problem in government IT. The main problems are poor project management, a lack of accountability, a broken contract award process, and decisions makers who don't understand IT well enough to guide these projects to fruition. Open source would not solve any of those problems.
Those two lines pretty much nail the issue at hand. Not only for IT but for other aspects of Governance as well.
I'm not sure how to solve that problem but it's clearly an issue.
And for whatever reason, the military seems to have a very short memory when it comes to major weapon system procurement failures. "Hmm, the V-22 went over budget, killed a good number of people, has higher maintenance needs and operating costs than we anticipated because of the USMC's requirements. I know, why don't we tie the entire future of our combat airpower to the JSF and hamstring that project with the needs of the USMC..."
Long story short, the space constraints of operating from an amphibious carrier forced design compromises on the V-22 that significantly worsened that program. Now with the JSF, the same thing is happening with the VSTOL variant that the USMC wants.
This is, unfortunately, what every discussion among politicians regarding any sort of technology is.
I believe the part about end-to-end testing and HIPA were something that needed to be asked and was not answered in an acceptable manner. There are some technical folks on that committee and they even showed source code.
The media coverage and congressional circus is blowing this way out of proportion.
They've completely hosed up existing data.
Took me 2 weeks to get in, finally got in. Application created for my wife and I. System broke. Logged in a couple days later. My wife's data is gone. No ability to add it back in. No ability to delete current application. No ability to start a new application. It's a mess.
Registering and using vastly different. Given that it took them years to get this far, I don't have a whole lot of hope that this will be usable for everyone, with data problems sorted, by March. Possible, but not likely. Especially because the attitude of "hey, just go somewhere else to buy" is so quick to be used. If that's really the case, why did we bother to even put this together, if it's not at all necessary or required?
There's going to be massive loads of hidden bugs that will plague this for months, if not years. Having such a tight deadline on it is crazy because it's not at all how any large insurance agency works, nor is it how the federal government works.
If you need insurance and you just received a cancel notice from BCBS that your policy is gone as of Jan 1st it becomes imperative that this exchange work within the next few weeks.
This isn't a political game real people are going to be royally hozed and soon if this doesn't get fixed.
Feel free to come back and tell me how wrong I am if my faith turns out to be misguided.
Even if several paths are kludged for these most desperate millions of people losing their current coverage Jan 1, I can't see how any more time than 7 weeks is feasible, leaving 2 and a fraction to process them. And to know the system is really working, large numbers, 10s of thousands, had better be getting successfully through the systems before then.
For convenience?
And March '14 isn't the date that individual plans go kaput. Most of us in FL and I assume other states got notices from BCBS that our plans end on Jan 1st. That means we must signup before Dec. 15th on healthcare.gov.
My experience with healthcare.gov...
I think I must've been the first person in the US to successfully create and account and apply. I received my authentication confirmation email at 4am on Oct. 1st.
However... I still haven't been able to view any plans. When I click the "View eligible plans" button it goes to a 404. Sometimes it's a 50x, sometimes it just times-out. I've tried nearly every day since Oct. 1st. I've tried early, I've tried late, I've tried on the weekdays and weekends... I've tried re-applying, I've tried applying in another state we are thinking of moving to. From my experience healthcare.gov is radically busted.
So yes, you can create an account, enter all your personal information and "apply" but you sure can't see any plans that you are qualified for or actually you know signup.
However, healthcare.gov generally seems very quick, very responsive, I thought the UI for signup an authentication was good.
YMMV
As I understand it, If you are due a subsidy, as of now the only way you can get it is through the Federal exchange. State exchanges must use its subsidy calculator, ditto insurance companies (the latter is reported to have been a very low priority item in the system development).
As things look now (putting on my student of failed projects cap), a lot of people are going to be without insurance of any sort come Jan 1, especially because they can't afford whatever policies they're de jure or de facto allowed to buy. Which could easily kill as many people as lost in the Space Shuttle program ... except they didn't volunteer for something so obviously dangerous.
If anything, I'd expect a focus on certification to decrease quality. Those things are expensive & time-consuming and the only ones who invest in them are the people who expect to be in that consulting ecosystem for a long time.
There are processes, not certifications, that result in generally high-quality projects (see NASA's processes) at the cost of being incredibly slow and expensive.
To take something from the article as an example, the QSSI executive's hedge about security not being part of his contract illustrates one of the real problems with our government contracting system. The sad fact is that it is more likely to have technologically illiterate "Business Analyst" and (cost-conscious) "Project Management" types making critical technology-related decisions about the scope of a project and contents of contracts than people who are technology-savvy, such as software or systems engineers, even if said people have some input.
A companion problem to this is the nature of government contracting. The QSSI guy is just doing what contractors do: they fulfill the letter of the contract (sometimes completely, sometimes by handwaving) and then when the government complains that the product doesn't work the way they wanted the contractor wins an extension or a follow-on (paid) contract to do the "additional" work.
It works like this for a reason. The "Analyst" ass-clowns in the government get better job security, because they're overseeing a project that can have its duration extended almost indefinitely. The contractor owners get to reap huge contracting windfalls. The lobbyists for the contractors who continue to bilk the government continue to get paid, and the Congressmen who the contractors buy, er, provide donations and campaign support for continue to hold their seats of power.
Look it like this: from their perspective, badly run projects that don't actually accomplish what they need to are a feature of government contracting, not a bug or problem to be solved.
Most code I have seen widely in production is using old sql injection vulnerable code. No sanitization and no proper session handling.
You would think that people would care about something like security with all the privacy concerns.
I wish I had better words to say this, but I had just woke up and out the door to breakfast.
Sure, it's possible that this kind of danger would be then aimed at separate developers working in those companies too... but I'm not sure it's a bad thing. It would be easier to resist stupid requirements (I won't do it, because it would bring legal issues on me) and even the comments on SO would be different too ("if you do this, you will be sued", rather than "this is not a secure way to do it").
And it didn't matter what the QA reported, the site was hell or high water launch Oct 1st whatever they revealed.
As always, those with political power (reported to be the White House and CMS, maybe some of HHS) trump those who have to make it work in the real world.
Instead of the ridiculous way the gov't handled it, with constantly changing requirements and a "change the car tire while we are driving" attitude?
Sounds like the WashPost is trying to do some damage control.
Unfortunately this disaster is so huge that's not adequate to tamp it down. But it sure looks like some are trying ... hmmm, what is the broadcast part of the MSM aside from Fox saying?