Why 40-Year Old Tech Is Still Running America's Air Traffic Control
wired.com
wired.com
There's no conflict there.
Now NSA meant to keep US infrastructure safe, and attack US companies/people is a conflict. But the FAA being responsible for both safety and ATC is far from a conflict.
I'd prefer the FAA moved slowly and got it right. This isn't an area where young-hip 20 somethings re-invented the tech' in the latest web-2.0 fad. This is an area where you want mathematically provable correctness and unit testing up the ying-yang.
Get it right slow, don't get it wrong fast.
There should be ways of introducing new tech and features while slowly fading out the old ones however you don't switch from one to the other until it has been reasonably field tested.
One problem is they went to a defense contractor to build software instead of a software company.
I'm sure their software is great in their spy airplanes where it will never see the light of day and you can cover it up if it fails majorly however I wouldn't want it in the air traffic control computer.
Devil's advocate here:
Most software companies simply aren't up to the task. They're just not suited to building provable software from design through implementation often requiring the use of functional languages.
If you look at who does most aircraft software (or similar safety systems) it is mostly by certified engineers who just happen to "engineer" in software, rather than programmers who try to be engineers.
I'm a programmer, I am not ashamed of that. But realistically the way I work is in no way transferable to a safety critical system. In that situation every single function needs to have a unit test, it needs to have reliable input and output, and ideally it needs to self-identify hardware-caused undefined state and fail-safe.
I'm a programmer on a safety critical system. It doesn't require getting a PE, it requires a good quality system and a sizable amount of scorn towards the forces that want to reduce our profession to nothing more than a trendhopping slapdash self-congratulatory shitshow.
A functional programming style or (almost) pure functional language would help to with (semi) automatic static code proofing.
You also have to consider the compiler. Proving that your Haskell code is correct is largely pointless if you then compile it with a compiler that hasn't been proven correct. At a minimum that's going to mean compiling the code with a lot of the fancier optimizations turned off, which can lead to a rather severe degradation in performance in the case of a pure functional language.
My guess is that there are not currently any functional languages that (i) are significantly easier to formally verify than C or Ada, (ii) have predictable time and space behavior, and (iii) have implementations that are both extremely well-verified and reasonably performant.
Unless you are an engineer by accident, I would say you should be as prepared as you could be (albeit new to the field).
Honestly, the stance you share in this topic is why normal people historically has not seen computers and software as a reliable thing and continue to use half-assed software such as Windows for example.
But a bad example. Microsoft uses automatic proofing for their (and third party) device drivers. Windows NT series is very reliable and has one of the best OS designs. The Win32 subsystem and shell comes with a lot of legacy and is a different story.
That's not a problem. That's why it ever worked properly in the first place, let alone working correctly for 40 years. Why shouldn't your system last for 40 years?
FTA:
> Modernization, a struggle for any federal agency, is practically antithetical to the FAA's operational culture, which is risk-averse, methodical, and bureaucratic.
I don't understand the tone of this article. Risk-averse, methodical, bureaucratic.... that's exactly what I want them to be!!
Still a humongous undertaking, don't get me wrong. But at least it is a 5 year plan versus a 20-30 year plan which is the current thought plan as I see it.
* Don't bother, just let the planes fly wherever they want, but levy a fine when they crash.
* Put it out to tender every 5 years and fine commercial ATC operators when crashes occur.
Both of them shut the stable door after the horse has bolted, and would probably require them to operate at least a passive sort of ATC themselves in order to assess blame.
I have some sympathy for the idea that most modern software companies, especially VC-funded startups, tend to prioritize getting to market fast over making everything exactly right. I'm not even saying it's wrong for them and their market. But if we're looking for something that's well-designed and carefully tested before going to market, along with a process designed to produce that, this doesn't sound like it at all.
My one experience working on a government contract was positive. Basically the government set a deadline, some basic performance metrics and added clauses that fined us if we didn't meet those performance metrics. If were early and/or exceeded those metrics, we got a bonus. The end result was simple: Our company did everything it could to make sure we met the deadline and we delivered a quality solution.
How much do you want to bet that a big company like Lockhead Martin is really good at using its political leverage to make sure the FAA doesn't impose fines in the contracts when projects don't perform up to key metrics?
Ding ding ding!
I watched Northrop do exactly that on a system. They used their clout to persuade the program office on a system to rate them more favorably than they deserved (truly, they were consistently delivering late but ended up with positive marks across the board). Fortunately, this resulted in a big shakeup once it was discovered and reported. Unfortunately, NG continued to hold the contract (no one else could, practically speaking) and all that really changed was the personnel in the program office. I was gone by the time thing got "fixed" so I have no idea if it's actually improved on that particular project.
If you build enough rapport with some of the workers there then they will actually respond to your emails in a relatively timely fashion, otherwise you'll be completely in the dark. Once every so often you find someone who actually responds quickly and does their job well. You guard that person's contact information and only give it to people you trust.
To be quite blunt, almost everyone you encounter intends to be a lifer at the FAA and knows that simply sticking around will almost guarantee them higher pay and a fairly decent retirement package. There is zero upside for them.
They're playing retirement min-max'er and I'm playing honest citizen. They're winning at the expense of us all.
They get the credit when things go right. Whenever things go wrong, it's all on the contractors.
Based on my anecdotal experience, the government employees are simply untouchable. Even when they blatantly break the law, they still keep their jobs, and all the contractor and subcontractor employees lose theirs. Maybe they get a letter in their personnel file that keeps them from certain transfers and promotions, but they don't get fired.
The worst thing that typically happens to them is a furlough, which is a lot like a layoff in the private sector, except the workers can actually expect to be called back to work at some point. Sometimes they even get paid something for their inconvenience.
I have never seen even a single one of them assume personal responsibility for anything even remotely negative. It does not happen. The buck never stops anywhere.
All that means that it's crystal clear to Upper-Management that air traffic control is Hard!, and it must be properly resourced. It makes me feel safer (even if probably more risky due to human error).
The minute automation comes in, an army of Crazy! PMs will begin to demand that everything has to be done due Thursday evening, so QA can test over the weekend, and then the Bean-Counters come in and slash every specification of hardware because its not quote-Efficient!-unquote.
Would you really want to fly in a world like that????
To me there's no way to tell if the plane flies out of their space or drops out of the sky at that moment. If that's the case the search and rescue won't be better had MH370 happened in the US.
[0] http://www.technologyreview.com/news/533871/could-passenger-...
The problem there is primary radar coverage, not the ATC system. (MH370 turned off its transponder so the only way to detect it was via primary radar.) There's no point in spending huge amounts of money expanding primary radar coverage just to get a marginal benefit in an enormously unlikely scenario.
I inherited a system from a guy with this mentality. I had a rack of 12 year old Solaris boxes in production running software almost as old. It was a complete nightmare. Part of things not being "broke" is making sure they work well into the future as well. You're just building up a massive technical debt with a "aint broke" mentality.
And it would be very handy in a variety of cases for "one air traffic control center to take over for another with the flip of a switch".
Personally, I think some of the old, expensive tech like ground based radar is a better solution than using GPS. End of the day, we're over-dependent on GPS, the system is very vulnerable to attack, and any disruption of it would global and catastrophic, while ground-based stuff is at least somewhat localized.
The MH370 case is a red herring IMO. Say we knew where that plane crashed. What does it mean? Everyone's dead in any case, and the details of precisely what happened doesn't necessarily offer a great deal of value.
Take for example Vancouver's Skytrain. Last year, there was a failure of some equipment that resulted in a system-wide shutdown. Media got in and was appauled that Skytrain still uses equipment run on floppies. The old software/hardware wasn't what broke down, though--it was some NEW equipment more recently installed to manage the PA system.
Why fix what ain't broke? ESPECIALLY when people's lives are on the line.
Remember when that control tower in Chicago got attacked by some crazed employee? Do you know why there isn't just a simple DR switch over? Because the system is so antiquated, something like that is too problematic to implement. Thus all the cancelled flights for a week at the world's busiest airport.
>ESPECIALLY when people's lives are on the line.
Its more risky to have a pig with lipstick system run by COBOL graybeards one foot in the grave than to run things properly in a modern system with modern redundancy and modern features. Last I heard, the code that runs many of these systems is non-auditable because its just a pile of spaghetti code on top of spaghetti code. Its "works" in the same way an 80 year lady on her third hip can "walk."
Maybe the do, maybe they don't. I'd rather find the plane and figure this out than assume.
One could certainly make the point that occasional mysteries cost less than sealing up all the holes, but it's silly to say that knowing what happened to MH370 would be pointless because everybody is dead anyway.
I think the difference is that Waterfall is actually a better solution - as long as you actually know all of your requirements ahead of time, and they will actually never change. This may be the case in for embedded stuff like avionics, but is almost never actually the case for more business-like software that automates or assists with human processes, which is what Air Traffic Control actually is.
So it kinda sounds like they used a traditional aerospace Waterfall project plan, but the requirements gathering was a total fail. Sounds like they need a team more experienced in Agile development, even though it has more of a reputation for putting things into production before they've been really carefully tested. Of course, this is based off a lot of shaky assumptions from a vague article, so who knows.
As for the original IBM 9020 computers running Host, these were IBM 360 variants. The technology in these computers is interesting - instead of newfangled ICs, the computers were built from SLT modules. These are metal modules kind of like an integrated circuit, except there's a small circuit board inside with discrete transistors, diodes, and printed resistors. (I was looking at some SLT modules yesterday.)
Think of Uber, Lyft, and Airbnb: Outdated regulations slowed them down, but consumer demand is forcing the law to evolve. This back-and-forth is what lets tech companies move fast and break things without risking our safety.
As it is for those companies, it was also with early aerospace: Risks weren't eliminated. Rather, at first, risks were sequestered to advantageous subpopulations and taken by volunteers who, in theory, understood what they were about to do.
These issues screwed up HealthCare.gov and are screwing up the Department of Veterans Affairs and a dozen other agencies that need computers and software that work.
Probably true but lacks substantiation in this article.
To the author's credit, she's not a free market cultist in this article, and points out that other countries have better government managed air traffic control.
Software Runaways: Monumental Software Disasters.
Anyone with an interest in the management of software projects should read it.
The worst of these project failures, the most expensive and the most ambitious, was the attempt made by the Federal Aviation Authority to modernize the computer system it uses to keep track of what planes are in the air. The effort began in 1981 and ended in complete failure in 1994. The government hired IBM to do the actual work, and over the course of 14 years, IBM burned through $3.7 billion dollars. Nothing was accomplished. The project was finally shut down by Congress. Nothing came out of the project, not a single piece of software, nor even a line of code, was ever used for anything.
From the book:
It has been noted by everyone from the New York Times to the Vice-President of the United States that the main problem on the Advanced Automation System was "changing requirements". For those involved in large-scale computer systems, that is nothing new. No one can perfectly surmise the shape and feel of a system years in advance. Even replacing some aspect of a system you know by heart is not immune from thinking twice about it. ... [But] the requirements churn (it was called) on the Advanced Automation System was not normal. It was the result of our enchantment with the computer-human interface, the CHI. The new controller workstation, fronted by a 20" by 20" color display, because it was capable of seemingly endles variety of presentations, mesmerized the population of AAS like the O.J. Simpson trial mesmerized the nation...
The project was handed over to human factor pundits, who then drove the design. Requirements became synonymous with preferences. Thousands of labor-months were spent designing, discussing, and demonstrating the possibilities: colors, fonts, overlays, reversals, serpentine lists, toggling, zooming, opaque windows, the list is huge. It was something to see. (Virtually all of the marketing brochures - produced prematurely and in large numbers - sparkled with some rendition or other of the new controller console.) It just wasn\'t usable...
Rummaging through one of the closets at the far end of the hall on the fifth floor one day, looking for some standards document, I found an envelope left by someone who left the company - as many did after so many years advancing against stone, while the wheels of commerce were accelerating on what everyone referred to as "the outside". It contained "A Brief History Of The Advanced Automation System". It was printed by hand and left, perhaps inadvertantly, or perhaps with the hope that some anthropologist might some day discover it and make a pronouncement. In every important way, it is the truth:
"A young man, recently hired, devotes years to a specification written to the bit level for programs that will never be coded. Another, to a specification that will be replaced. Programmers marry one another, then divorce and marry someone in another subsystem. Program designs are written to severe formats, then forgotten. The formats endure. A man decides to become a woman and succeeds before system testing starts. As testing approaches, she begins a second career on local television, hosting a show on witchcraft. An architect chases a new technology, then another, then changes his mind and goes into management. A veteran programmer writes the same program a dozen times, then transfers. The price of money increases eight times. Programmers sleep in the halls. Committees convene for years to discuss keystroking. An ambitious training manager builds an encyclopedia of manuals no one will ever use. Decisions are scheduled weeks in advance. Workers sit in the hallways. Notions of computing begin in the epoch of A, edge toward B, then come down hard on A + B. Human factors experts achieve Olympian status. The Berlin Wall collapses. The map of Europe is redrawn. Everything is counted. Quality becomes mixed with quantity. Morale is reduced to a quotient, then counted. Dozens of men and women argue for thousands of hours: What is a requirement? A generation of workers retire. The very mission changes and only a few notice. Programming theories come and go. Managers cling to expectations, like a child to a blanket. Presentations are polished to create an impression, then curbed to cut costs. Then they are studied. The work spikes and spikes again. Offices are changed a dozen times. Management retires and returns. The contractor is sold. Software is blamed. Executives are promoted. The years rip by with no end in sight. A company president gets an idea: make large small. Turn methods over to each programmer. Dress down. Count on the inscrutability of programming. Promote good news. Turn a leaf away from the sun. Maybe start over.