The screen that set off the ballistic missile alert on Saturday
twitter.com
twitter.com
The problem is not knowing (or having someone who knows) how to design something better, it's treating good design as a priority. It rarely happens.
Don't know what the equivalent in the graphics design world would be.
"Hygiene" is a great word overall, for both cases.
[1]: https://medium.com/tragic-design/how-bad-ux-killed-jenny-ef9...
When I've been given PSDs in the past, either for review or for implementation, there's never been any documentation with any of the reasoning behind any of it. And when I've asked follow-up questions about why something was done the way it was, I've typically been met with either defensiveness or "that's how it is, now go and build it"
Something I brought up at my IT Support job yesterday regarding the crazy idea I had to actually document some processes and rules we have. Flow charts and tables and such. Having a reference you can use at any time is much more efficient than expecting everyone to remember literally thousands of facts they're told once in two weeks of training.
There had to have been quite the cavalcade of idiots behind the making of this system.
People blindly click through confirmation screens all the time.
In this case, the citizens of Hawaii paid for the bad system design with 48 minutes of existential horror.
But these kinds of mistakes happens ALL THE TIME with "enterprise applications." Oracle has similar horrendous UX for their EBS (Enterprise Business Suite), so does SAP, Siemens, etc. People regularly make costly mistakes in shipping and receiving, purchasing and manufacturing because they have to deal with shitty confusing applications that look just like that screenshot.
Totally not surprising that the internal application for public emergencies has the same awfulness as PTO request.
I expect the same employee in Hawaii fumbled the UI on their "time off request" for psychiatric counseling too.
The "normal" for enterprise and especially government software is dogshit though.
It's because the people doing the procurement are not the people actually using the thing day to day. They treat UX and design as extravagances in the face of spec-sheets and slick marketing decks.
Currently unscrewing one of those as my day job.
"Delete all production orders for period" - no confirmation, no ability to rollback (beyond me restoring from a backup) that kind of thing.
It's not that it isn't user friendly, it's that it is actively user hostile.
Hah! I built one of those once! The (very abbreviated) conversation went something like this:
"I need a way to bulk delete orders."
"Umm... Are you sure? I can't think of a good reason to delete a bunch of orders in production."
"Yes. We absolutely need this."
"Ok..."
A short time later
"Why does it ask me to confirm? Just make it do it."
"Sure..."
A week later, after restoring from backup
"Remove the bulk delete function. I don't know why we even have that!"
Really demanding managers refuse to explain their XY problems... This one never did learn.
The out sourced developers however...very much yes I can.
Half the things do the wrong thing wrong, the other half is the wrong thing sorta working.
There was clearly no attempt to understand what the user wanted or any attempt to ask the simple questions like "Why do you want X, is X related to Y, would it be handy to pull in related Y for display X" beyond the terrible code quality (and database architecture, I wish I was kidding when I say it took 49 to 69s to search for a quote (I timed it on repeated runs), it now takes ~400-600ms and I still think that's too slow (and I search more things) there is just a complete lack of any thought/effort or consistency.
Even the tables aren't consistent across the system - sometimes click a row opens the related entity (with no indication that's what it does), sometimes there is a "view" button in the row on the right, sometimes on the left.
It's just pile after pile of UX disasters/architecture code disasters, no wonder the original lead dev came close to a nervous breakdown, I just don't think he was capable of dealing with a system of this complexity.
Asking those questions is being obstructionist. Most end users don't understand the ramifications of what they are requesting. Explaining it all is an effort to bring them up to a level of education that would allow them to do your job. They pay _you_ to do your job and now you are just wasting everybody's time. Why do you have to make everything so complicated!? Why can't you just do what your told? Just add the button and stop asking questions!
Asking "why X, why Y, why Z" can come off as challenging their knowledge, and questions like "would you also need W?" tend to get answered "yes, sounds good" so now you've just made more work for yourself.
100% this - "Have you considered that our long lead times on inventory means we increase our holding costs?" is a good lead-in to exploring how to reduce that.
> "Have you considered that our long lead times on inventory means we increase our holding costs?"
What's frustrating about this is now we also have to understand the business overall, and that's kind of the problem in the first place: nobody has the patience to explain it.
So in addition to coming up to speed on whatever tech stack they have, we're going to have to find someone who understands and is capable of explaining the (sometimes completely foreign to us) business side of things and come up to speed on that too. That's a lot of ramp up time, especially when nobody wants to hand out raises and you're better off finding a new gig roughly every other year anyway.
I'm currently working somewhere with people that are great at explaining the business, and are willing to discuss at length the intricacies of how anything works at any time. The tech stack is abysmal, but being able to easily understand the business makes it worth dealing with the dumpster fire of a LoB app. I'd take it over the previous job, where we had greenfield project, bleeding edge tech, and money, but nobody could tell me how the hell the business worked...
That gave me enough of a grasp of the principles and more importantly the nomenclature that I could then talk to people in their language not mine.
To me it's a normalised series of tables to them it's WIP inventory.
One of the things DDD got right (if you remove the buzzword bingo/hype) was the idea of a 'common language'.
Unfortunately in my experience (couple of decades) most business people don't want a common language, they want you to understand theirs.
It is surprising how little theoretical knowledge people who work in a field use on a day to day basis.
> Unfortunately in my experience (couple of decades) most business people don't want a common language, they want you to understand theirs.
The “common language” of DDD was the language of the business domain, so that’s in line with DDD.
The practical problem is that the language of many business domains is often highly context dependent, making it unsuitable for direct use where ambiguity must be avoided even without context, and trying to namespace things to map to the specific relevant business contexts is often impractical.
I've found it's easier to understand their domain and map it back to mine than it is to try to build a shared domain.
Also I find business stuff interesting, it's just a nice change of pace from fighting with code.
But there isn't a "delete all" button, presumably because people would use it and complain that they wiped out all their email.
The end result, however, is the iphone email is completely unusable for me.
Consumer software is chosen by people that actually use the software. They can see that it is shit, give it bad reviews etc.
I used to work at a very large company who use Teamcity (think git for CAD), which was the worst POS I have ever used. A 90s Java mess that was so slow you could watch it redrawing GUI controls in slow motion. Everybody that used it hated it. At least some of the higher ups liked it, but they didn't actually have to use it.
I wish we could have a sane conversation about how UI's should be customizeable for the end user
Depends a lot on context and the type of application, but a lot of end users get confused about where the line is between UI/UX and backend functionality.
A lot of things that might seem to be UI issues might actually have implications for things like resource consumption and application stability. It's a tough balance to strike.
The issue with these tools is the company that is too shit to run them. Put the smallest box they could salvage that's already running a hundred apps. Put the storage on a NAS in another datacenter because there was no more local disk space.
In the end, the correct process is probably have two people activate the alert, so that two people have to independently not read the dialog boxes or whatever to send a false alarm. But maybe that doesn't work either, I haven't done any research to that effect and don't really know.
On some level, I like the interface. There is a list of possible alerts, and you click the one you want to send to send it. It's just that it doesn't account for the possibility that the user doesn't ACTUALLY want to send the one they clicked. The same problem exists with any powerful tool. Hammers don't check that you're hammering a nail and not your thumb. Cars don't ask "there appears to be a pedestrian, are you sure you meant 'accelerate' and not 'brake'?" With great power comes great responsibility.
Same thing with your hammer analogy. Everything shouldn’t look like a nail then! A real alert should look like a nail. A real ballistic missile alert should look like a railroad spike (aka really big nail). Test alerts should look like a screw. Etc.
> a different interface wouldn't necessarily stop someone who's confused from sending the wrong alert.
No. Just. No. It is a horrific design. This one will likely end up in a future UX textbook as a case study. There is simply no excuse for jumbling together a list of options like that. The list items are not even semantically coherent.Ultimately, however, the individual, his chain-of-command and the system designers have to be held accountable. The scary thing is that many people saw this and said nothing, almost certainly some people heard complaints about this and did nothing, somebody even signed off on this. It demonstrates an organizational problem, I think.
A simple fix would be to use a tried and true method that's been effective for decades. Have different popups that read
Type "REAL WORLD" to send this alert.
and
Type "DRILL" to send this alert.
So, yes, bring up a huge flashing red warning dialog with a single confirmation step. And make it easy to issue a retraction.
Trust me, if you get in your users way they will find some way to get rid of you.
1. Separate items intended to be tests from items intended to be live warnings. Draw a green (or yellow) box around the test items and a red box around the live items.
2. Require a confirmation before sending out a live item, and not just clicking a button. Put up a big red flashing warning that says, "You are about to send out a live warning, are you absolutely sure you want to do this?" and require the operator to type "YES".
> With great power comes great responsibility.
Yes, and designers here have a lot of power and responsibility to design an interface that is difficult to use in the wrong way, and doesn't treat options with wildly different real-world consequences as equivalent.
How many times to I test the SMS delivery system? Not very often, so I am going to be more careful with any popups. Maybe you only use your computer once a month?
These 48 minutes may well have changed a whole bunch of people's lives for the better, in the long term.
I guess you can think of it like waterboarding. Waterboarding doesn't harm anyone, it forces one to contemplate mortality, but it is also something so terrible that one would not wish on anybody.
Or imagine someone who, convinced of an impending death, donates all of their saved wealth to charities online.
Such misteackes in the cockpit of an airliner are deadly, which is why a great deal of effort is expended to prevent such mistakes. Unfortunately,
1. bad UI design is usually only obvious in hindsight
2. doing good UI design is expensive, and not always worth the effort
Of course, I face the same sorts of issues in designing a programming language (D), so this topic is of continuing interest for me.
That's only true if you have little or no experience designing or even using interfaces. As someone who has been bitten by bad UI design in the past, I've noticed and called out a decent amount of bad design well before it actually has caused a problem.
I'm afraid the history of cockpit design in airplanes decisively shows otherwise.
Based on the photos of the screen shown in the article, this one's dead obvious.
That requires a paper form. The payroll system in the state has failed to be upgraded for some 20 years or so of promises. Staff are paid via checks which are sent direct to their banks because they can't even pay over the wire.
Here's a good article about it: http://khon2.com/2015/05/11/costly-consequences-of-outdated-...
I can hear a forward thinking developer saying "hey should we at least have colored buttons or something on this screen so it's easy to see what is a test and what is not?" and the product owner/business owner/PM saying "no man just add a link it's faster."
Accessibility becomes an issue for colourblind people then.
I'm not sure "someone using this system might see everything as the same shade of grey" is a realistic concern at those levels of incidence.
"safety-critical systems with a large user base"
Surely a missile warning system would be operated by a very, very small number of people, on the order of twenty or thirty?
For instance, here's some icons from NATO, which used to be MIL-STD-2525. You'll notice that all the icons differentiate in both color and form. The NTDS standards for Naval systems are similar.
https://en.wikipedia.org/wiki/NATO_Joint_Military_Symbology#...
To me, the screenshot looks more like a list of user-defined presets than something that went through the original software development process. Maybe something along the lines of only superuser can define new presets, but lower users can activate them. That way, at least random "I love you honeybunny" pranks would be avoided, but the risk of skipping a real warning because of insufficient permissions would be avoided.
A physical UI would have an openly accessible switch for "drill" and "not a drill" protected behind a seal that needs to be broken, this is difficult to replicate in software.
[0] Which, to be fair to users, is a disease introduced by putting too many switches under covers which shouldn't be.
Good stuff but I think they could probably have gone with something else for "Neutral" - the distinction between a 16:10 rectangle and a square could easily be missed in a hurry, especially at small sizes.
For someone sending alerts, color vision is not — things like keeping cool under pressure, following procedure, interpreting chaotic information, etc. are.
One problem: William's in the toilet. He had a bad taco for lunch and honestly, he's going to be in there for a good 10 minutes. And there's a missile coming. Luckily Frank the IT guy is there and knows what to do but ... he's colourblind!
Oh dear. He's just sent out a dummy warning by mistake and now half of Hawaii is going to die. Well done.
"Everyone is going to die. Run but it won't help"... Is that want you wanted to send [Yes] [No]
I say this as someone who watches people log in on PCs and click past a useless (for them) Citrix dialog that includes a "stop asking" checkbox. Citrix added that checkbox some time ago, so these folks have been vacuously ignoring the entire thing at least once per business day for at least a year.
Perhaps a captcha style prompt could be added (basic math with random values?) to critical/damaging operations but I'm sure that would get pushback of its own.
"You can't require that our people do basic arithmetic when they need to send an alert! It'll slow them down!" Suggested response: "Are you telling me that you're allowing this kind of operation to be done by people who can't handle single digit arithmetic?"
They could consider a different UI response between a drill and active alert. Since drills are the majority of their workload they'd get dialog blindness to the drill confirmation, and then you'd design the active alert confirmation to be a-typical of the drill (for example a different color dialog, and have an "I agree" checkbox on the active, but none of the drill).
Just adding a checkbox and later a popup with an additional warning wasn't enough.
An undo function is obviously best, but ufortunately while technically possible, it took a while - possibly a day - to restore due to ripple effects in other systems, and it normally wasn't the person clicking the button that had any reason to notice something was wrong.
Unfortunately a lot of organizations don't want to invest the time/resources into creating it. Particularly when it gets into discussing exactly how the underlying data should be handled during the undo window (e.g. does it exist? Is it just flagged deleted? What about relations? Do all of our queries that touch this data check that? Etc).
That would apply to something that's used routinely, I'm not sure this system is actually used that often. Do these emergency systems send out "this is a drill" messages really that regularly? As a non-US person that sounds like a quite scary way to live.
Even if it is something that gets used regularly, adding additional levels of confirmation, depending on the kind of alert that's gonna be sent out, would already go a long way of preventing this kind of mistake.
For the real deal add some bold and flashing text along the lines of "This is the NOT A DRILL message, are you sure about sending this?" which pops up as a second confirmation screen, while drill messages have only one confirmation screen.
Per the explanation provided by the agency, this is a drill that is conducted at every shift change, so presumably multiple times a day. I gather it does not send any actual alert to the public, just simulates it (perhaps sends to a small pool of test devices?) But it's appearing more and more likely that we've been giving a misleading explanation for this incident so who knows.
Nah, real world experience shows that people who supposedly have common sense will happily put their payment info just about anywhere.
The Test Records probably do the same thing just to a smaller list than the real one.
So there is no real way to separate them.
Someone reviewed the spec for this. Someone modified the page of links. Someone reviewed that modification. Someone signed off on the change.
My superior would just have to solve the problem of getting the spec change cleared or find another developer. If that was somehow even seen as a problem by the superior - same thing - they’d have to find another developer.
But only to a point. People resist perceived change. You can improve the back-end all you want, as long as you don't make unexpected changes to the front-end. This includes something as simple as a clarifying pop-up, or a speed increase of a procedure.
... as long as you do not mind being fired.
Doing things that are immoral/dangerous/stupid because you need to eat is not an excuse.
Actually, with government contracts, you are legally obligated to perform only the task assigned. Providing work to the government without the government paying for it is illegal. And you are only being paid for what's in the contract. (This is actually overall a good thing. Otherwise, large companies with better margins could easily provide free value-adds to government customers to win contracts away from small shops with small margins.)
Now, in this case, yes maybe the contractual item could have been interpreted in a way that allowed a proper page redesign. However, I would never state that as a fact without having the requirements and also receiving agreement from the government customer over such changes.
This is clearly a case where all the downsides of bureaucracy were in full effect (someone might be afraid to touch something) but not the benefits (lots of security checks and balances such as in air traffic control).
When "trying to do it right" means you have to go through layers and layers of bureaucracy and probably ruffle a few feathers in the process, over and over again, I can see how some people quick learn to "just do their job".
This is human behavior. We can talk as much as we want about what would be the ideal solution here, and how we would never do it, but problems like this don't happen in a vacuum and are rarely one person's fault. It's a whole system that leads to this.
But what I was trying to say was that by either refusing to do it outright, o doing it right (without “asking for permission”). That is - I’d rather be jailed or fired than doing this. And I hope that goes for most devs.
The Anti-Deficiency Act only prohibits "voluntary" services, not "gratuitous" services. Prior to the passage of the law, it was not unusual for cash-strapped gov't agencies to accept "voluntary" services (i.e., no formal obligation to pay for them) but then pay for them later after being allocated more funds. Congress didn't like this so it outlawed the practice. However, if someone wants to donate services to the gov't and executes a written agreement to not accept payment, then it is legal for the government to accept the free services. See, e.g., https://www.gao.gov/assets/450/441639.pdf
I've seen lots of organizations try to save a buck by extending a previously existing system with in-house labor after the original contractor asked for more money to do the change. I would bet that the original contractor (if they're still involved with the project) hasn’t seen that screen because it's some homebrew patch put in by whichever administrator babysits the machines for the state government.
The problem with organizations maintaining their own rogue patches is that, if those organizations had people compentent enough to make changes well, they wouldn't have hired a contractor or bought off the shelf in the first place.
I have worked in shops that intentionally deliver bad software, and those that had higher ethical standards, and the only difference on the business side was that the bad software had higher budgets and more permanent employees (both more-permanent employees and more permanent-employees).
The problem is that all the people who can tell the difference between good and bad are employed by the contractors. The direct government employees are still counting SLoC and basing their UI requirements on the Excel spreadsheets that were directly copied from paper forms from the 1970s. I am not making this up.
The contract awards are still mostly based on who you know, rather than the quality of your past work, so given the choice between a slipshod initial implementation with a juicy back end in the form of continual maintenance and doing it right the first time and delivering something that never needs contractor support ever again, it isn't surprising that a lot of companies opt for the former even when most of their employees would prefer the latter.
<H2> Testing </H2>
...
<H2> FOR REAL </H2>
...
It's not perfect, but at least the test alerts and the real alerts are not mixed together.Then adding one new link with any reasonable categorization might mean changing the label on a separate link to make it more distinct, and no one wants to go there anymore.
I'd be very surprised if the ultimate cost of this mistake isn't far higher than the added cost of adding an "Are you really sure?" confirmation option.
This is why you don't let accountants make cost decisions in a vacuum. Of course if you're counting beans, the cheapest option is best.
In reality-based accounting, the hidden costs, consequential costs, potential damages, and costs of failure guarantee that cheapest is risky if you're lucky, and suicidally expensive if you're not.
The GAP are not designed to analyse those costs, so stupid decisions are made, avoidable consequences occur, and ultimately money is wasted, not saved.
We're seeing the same thing in the UK now where a huge government services contractor has gone bust.
In the UK it's policy to give government contracts to the lowest bidder. This makes perfect sense, if you're clueless and have no idea what you're doing.
In other countries it's policy to offer contracts to the second or third cheapest bidder, because this encourages bidders to to put in estimates that mean they're more likely to finish the job without cutting corners, more likely to include realistic margins to keep their business afloat, and more likely to estimate unexpected costs realistically.
The UK's policy has completely failed to save money. It's going to cost hundreds of millions of pounds in direct costs, and probably more than a billion in consequential losses around the rest of the economy, to deal with the fallout from a wholly avoidable corporate drama.
It's a catch-22 for developers in this position: you have to be able to justify every single major improvement from a position of cost-benefit to the business, but to do that adequately well enough to convince a client to spend the money requires up-front time and money expenditure that is hard to justify until after it's done.
I have seen a large number of government web sites, and most of them, especially at the state level (I am talking about US states), have a horrible UX.
Facebook has a dozen of fake drill buttons, and one real drill button, but you can't tell which is which.
Apple has a launch button but it doesn't work because the missiles are of a different brand.
Google's drill button turns out to be an ad.
A better idea might be an "Oops, belay that!" next to it so it can be cancelled just as quickly.
What some designers don't realize is that, in reality, they are also in sales. If they can't effectively pitch their ideas to non-designer audiences, it doesn't matter how good their design is; it won't be used.
It's not a coincidence that Paul Rand would include a hefty proposal book along with his logos.
I’ve written a few versions of this article over the years. I keep hoping I’ll see the day that it’s no longer prescient: “Why the DMV Website Sucks” https://hackernoon.com/why-the-dmv-website-sucks-2f27a367baa...
It makes complete sense to me that given a spec of "make a button send this specific text" you would end up with this. So it's very easy to send the initial message but sending a custom follow up requires contacting someone with access to the underlying system who can send a custom message.
In the same way that I may give internal users the ability to fire off SPECIFIC (maybe with placeholders) push notifications with the press of a button but not custom messages. Sure my underlying infrastructure can handle custom messages but the UI for such a feature does not exist (or at least is not available to the person who might have the ability to send preset messages).
All in all this was terrible for the people in Hawaii, no doubt. That said I WANT my missile alert system to be easy to activate to give people the most time I can to prepare. The best fix for this (yes a new UI/UX would be best but let's be realistic on how much time/energy they are willing to spend on this) would be a scary looking alert confirmation dialog of "Are you sure you want to send this alert to the whole state?" and maybe have to type a confirmation string.
There's likely a CMS of some sort for adding messages, so adding a false alarm one probably just meant someone with admin access just needed to fill out a web form.
UI changes, even simple ones, would take a little longer.
Having to type out YES or some phrase is a fairly common "Are you really sure you want to do this??" sort of thing. I gather they're also looking at requiring a second person's confirmation which, if practical and very unlikely to make it impossible to send any message, seems reasonable as well.
One reason of course is a combination of alarm fatigue and the fact that software has trained people to just always click yes when an annoying pop up appears.
Another is that people will think the have already clicked the correct thing button. So it’s less “Oh a confirmation box has appeared, let me confirm that I pressed the correct button” and more “stupid computer, of course I clicked the right button, that’s what I want to do!” without realizing they did in fact accidentally click the wrong button.
Of course they then need a way to test the button...
Have PINs required for certain critical operations and post those PINs in multiple places around the facility. Each PIN is different for each operation. That way the user has to look up the PIN before sending out the "you're going to die" message.
The codes would be per action not per person. So the "send missile threat" activation code would be, for example, 98790 for everyone and the "test send missile threat" would be 16289 for everyone. You'd have to look up what the code was for your message you wanted to send. All test message start with 1 while all real messages start with 9. They could be rotating so employees don't memorize them.
So to send an alarm by mistake someone would need to press the wrong button and also look up the wrong code and also ignore the meaning of the code prefix.
Just an example.
Though maybe just "this is not a drill" would be better/easier.
In any case, your proposal suffers from the flaw that in a real emergency, nobody will have the extra seconds to spare to look up a code, because that would literally be thousands of people dying per lost second, so the emergency codes will all rotate from '99999' through '99999'. And an employee would actually be responsible for changing the codes from '99999' to '99999' every time they are required to rotate.
Any reasonable person typing in “This is not a drill...” would pause half way through. It’s not the same as blindly clicking “Yes”.
Yes, in situations where the message does not fit within a pre-defined list due to an unexpected event. Of course it should still have security mechanism's to ensure that the transmission is properly authorised.
The biggest threat is probably misuse by politicians rather than the poor clerk!
Allowing a separate option for a clerk, maybe with confirmation from a supervisor or other clerk, to send an arbitrary message (with a separate approval chain) would be useful as well. In the Hawaii situation not having that option likely delayed sending out the retraction as they didn't have a built in.
This should also only be used for the real alerts and not the drills/tests otherwise you're training users to blindly type YES.
Maybe changing the challenge to "HOLY FUCKING SHIT YES THERE REALLY IS A FUCKING NUKE COMING AT US SEND THE FUCKING MESSAGE!!!" in the real challenge would work better. Of course, when you have to push the button like that, I'm not sure if you could still type straight ;).
But yeah: how do you properly write a test for a potentially dangerous operation?
If I had a bitcoin for every time someone clicked through the confirm dialog without looking...
If they are equal (for example a generic message like "Are you sure?") then it's almost like no confirmation, because people get trained to click it automatically.
I think that ideally each one must have a "nice" image about the alarm, that is very different from the other images.
Another possibility is to force the user to retype the message, so the user must read and understand the actual message to be send. (Remember to disallow cut and paste.) (Allow a small number of typos, perhaps a Levenshtein distance of 4 or 5, because the user will be probably nervous if there are some incoming missiles.)
Simple: you don't put the button behind the cover, you put the media with the scary message behind it. Then you can test the whole transmission system with less chance for mistakes.
Ironically, this was exactly the solution to the last big EBS failure, the "code word hatefullness" incident in 1971. Back then an operator had to load a tape into the transmitter, but one day he loaded the wrong one because the real and test tapes were stored next to each other. After the incident, they moved the tape to a different location:
http://conelrad.blogspot.com/2010/09/code-word-hatefulness-g...
> …In the past three tapes, one for the test and two for actual emergencies, were hanging on three labeled hooks above the transmitter… In the future only the test tape will be left near the transmitter. The two emergency tapes [will be] be sealed in clearly marked envelopes and placed inside a nearby cabinet.
So even if the spec is where the "false alarm" button was omitted, it's still a reasonable complaint that the system was designed without it.
There's a post just a short distance down that twitter thread to a simple layout change that would have made it difficult to choose the wrong option by simply categorizing the options properly. But I'm betting that list is autogenerated on a hardcoded template from some goddamn database somewhere in this enterprise app that makes a simple layout change like that a nightmare to implement.
Compare M. Milan's design to the mainframe menu screens I have listed at https://news.ycombinator.com/item?id=16158878 .
When someone is in a crisis situation, their actions have to be the SAME as a drill. Making someone do or type something different when things are already hectic is likely to send them into brain-lock.
The solution is something extra that pops up AFTER doing everything the same that says "Sending real alert in 10 ... 9 ... 8 ..." with a "Type: "Abort" in text field to stop".
If the operator brain freezes whether real or fake, the alert still goes out in 10 seconds. If the operator screwed up but doesn't brain-freeze, he can stop the situation.
That's pretty much the best you can do if you're trying to make your drill and your alarm almost identical in order to maximize crisis performance.
I can imagine that when someone accidentally sends out a real alert, the "Oh F" moment will last longer than the 10 seconds you give the operator to type those 5 characters, in the correct order, with shaky fingers.
"Give it a thump with your finger" https://youtu.be/YchEuqYjMi8?t=29
https://twitter.com/supersat/status/952612571122630659
It has a similar list of alert "templates" as this one, but in addition it has a confirmation box with a radio button "send TEST only / SEND EAS LIVE", and if you select the "send live" option you must enter an authorization code.
It would be interesting to see what the rest of the user interface looks like, if the hawaii one has something similar.
But regardless of excuses it should never take that long to send that message. You are going to have people with PTSD for years over this.
I wished the state would be sued by every single person in the state -- I am sure than both the guy who clicked the link and the UI would be gone presto. Probably some sovereign immunity that stops that though.
The only correct solution is to fire the person involved, as the next guy will look closer. Yeah maybe not fair, but when working with landmines you have to pay attention - and it is politically much more feasible than redesigning the system.
However I also object to that system in the first place, since it doesn't matter where you hide if a nuke falls close to you, and you probably don't want to survive.
How we've not wiped ourselves off by accident so far is a miracle.
[1] - https://en.wikipedia.org/wiki/Command_and_Control_(book)
http://nuclearfiles.org/menu/key-issues/nuclear-weapons/issu...
1979 Nov 9 Computer Exercise Tape
If you read the Washington Post account, it says the interface was a drop down menu of which this picture is not of. See:
https://www.washingtonpost.com/news/post-nation/wp/2018/01/1...
And perhaps not surprisingly some news outlets are using an unverified image from a tweet as a news story.
In the cockpit of every jet fighter is a brightly painted
lever that, when pulled, fires a small rocket engine
underneath the pilot's seat, blowing the pilot, still in
his seat, out of the aircraft to parachute safely to
earth. Ejector seat levers can only be used once, and
their consequences are significant and irreversible.
Applications must have ejector seat levers so that users
can "occasionally" move persistent objects in the
interface, or dramatically (sometimes irreversibly)
alter the function or behavior of the application. The
one thing that must never happen is accidental
deployment of the ejector seat.
The interface design must assure that a user can never
inadvertently fire the ejector seat when all he wants to
do is make some minor adjustment to the program.http://www.bbc.co.uk/news/uk-england-lincolnshire-37475298
We probably won't ever know how many people died as a direct result of the Hawaii BMD mis-alert, but I'd be surprised if the number was non-zero (think in terms of car crashes, stress-aggravated cerebrovascular accidents, and so on).
Similarly, consider the warning signage deployed around high tension electrical substations, and the consequences of ignoring it or failing to understand its significance.
The point is, in these types of situation bad design can kill, and we need to design accordingly and unambiguously.
I'm assuming you meant you'd be surprised if that number was zero. It's awfully hard to, in good faith, directly associate a message with a death. I mean if you wanted to make it seem bigger than it was, you could associate any crash or heart related death that happened in that 38 minute (?) period as directly a result of the warning, but I don't think that would be in good faith.
Statistical significance will depend on the size of the spike.
"not only will it kill you, it will hurt the whole time you are dying"
ref https://en.wikipedia.org/wiki/Death_of_Sean_Cunningham_(pilo...
The "training" they have is, if anything, an MS word document with a bunch of screenshots or a flash animation showing each of the functions that the contract required the developer to build in.
If they detect one then they assess it. If after assessment they determine there is a threat to anyone they notify the appropriate agencies.
A civilian in a totally different government agency is the one who pushes the notification button.
However, my knowledge of these things is a bit piecemeal and anecdotal. Does anyone know a good book on these things?
At least with airlines this seems to be largely abandoned now. My wife worked for a large middle eastern airline for a few years, and the crews were just assigned randomly. Well I guess there is a complicated algorithm that figures it out, but given the company has nearly 15,000 cabin crew it’s unlikely you’ll see someone you flew with again on another flight.
Also from the stories I heard people were promoted to managerial roles (on big flights there’s usually a senior for each class, and then a purser who reports to the captain) through seniority, not because they had any specific people skills that made them a good manager.
Crew Relationship Management is probably something else ;-)
>In the cockpit of every jet fighter is a brightly painted lever that, when pulled, fires a small rocket engine underneath the pilot's seat, blowing the pilot, still in his seat, out of the aircraft to parachute safely to earth. Ejector seat levers can only be used once, and their consequences are significant and irreversible.
>Applications must have ejector seat levers so that users can "occasionally" move persistent objects in the interface, or dramatically (sometimes irreversibly) alter the function or behavior of the application. The one thing that must never happen is accidental deployment of the ejector seat.
>The interface design must assure that a user can never inadvertently fire the ejector seat when all he wants to do is make some minor adjustment to the program.
Several indents like that make code blocks, which respect original new line placements and are usually unreadable on mobile.
A lot of people format quotes with italics, as in this example:
https://news.ycombinator.com/item?id=7485333
HN formatting documentation:
* http://www.vfp62.com/f14_rio.html
* https://www.theguardian.com/world/2009/nov/02/south-africa-p...
I'd argue that very few computer applications need something that works like an ejector seat lever. In most software applications you have more than milliseconds or seconds to make decisions about big, dramatic (and potentially irreversible) changes, and thus they should have multiple levels of confirmation.
This is a legacy software (and hardware!) system with a relatively small budget and number of employees that needs to coordinate with other large organizations (FEMA, HI-EMA, NOAA, etc.). I think the most interesting lessons to learn from this have to do with long term software maintenance. I'm sure folks at the FCC/*EMA knew that this UI was janky but why did they not have the budget/power to fix it? How do we ensure that the public sector can benefit from the technical advances that most people on hacker news take for granted? Curious to hear from folks with experience in relevant parts of the government.
Yes! And with re-engineering too! When you redevelop a system, you need to build up historical knowledge of its antecedents, and teach it to the current operators and maintainers. You shouldn't just start from scratch from the requirements.
In this case, there was an even older legacy system that had a similar incident in 1971, which they then mitigated. Apparently that lesson was lost.
http://conelrad.blogspot.com/2010/09/code-word-hatefulness-g...:
> …In the past three tapes, one for the test and two for actual emergencies, were hanging on three labeled hooks above the transmitter… In the future only the test tape will be left near the transmitter. The two emergency tapes [will be] be sealed in clearly marked envelopes and placed inside a nearby cabinet.
What I wonder about is whether those links generate HTTP GET-requests. Links do so by default.
If so, just need internal web spider or overzealous web browser prefetcher, and one day Hawaii might have a lot of false alerts going on...
GET-requests are not supposed to have side effects, like alerting a whole state.
When you have side effects, HTTP verb should be something else, like POST.
Of course, it's possible there's Javascript handler behind those links that generates a HTTP POST request. Overall appearance of the page does suggest otherwise.
I wonder how the confirmation page (if any) behaves and looks like...
Oh, and their display mode is set incorrectly. It's clearly non-native resolution (blurry text, bilinear "zooming" performed by the display, while individual pixels are in sharp focus) and wrong aspect ratio (the font appears too wide).
Perhaps 1024x768 on a display with 1920x1080 native resolution. Or something similar.
But seriously, as an occasional government contractor, this does not surprise me at all. This is par for the course on government software projects. Nobody gets fired for adding just one more menu item. Besides, good luck pushing a complete redesign through in a reasonable amount of time/money. With the bloated teams and processes they use, that would take years. Sadly, this is probably the "new" version anyway.
https://www.reuters.com/article/us-northkorea-missiles-japan...
No one intentionally set off these alarms.
- To allow authorities to listen for "chatter" from NK military deployed in Hawaii who might have been ready to carry out some action if an attack were launched, but who would not likely have been told when it would occur.
- To watch how NK responds internally to the idea that the US might be launching a counter-strike. Things like escape routes, comms, etc., could be observed via sigint.
- To create a "path" for the public to think about this kind of thing. It's been years since anyone in the US (public or media) has had to think about incoming missile attacks. The false alarm gives all the reporters an excuse to read up on the history, to issue warnings about lack of preparedness, etc. This means that in the event of an actual attack the information mechanisms will all work more smoothly. It's a dress rehearsal so that everyone is not stuck like a deer in the headlights if an attack occurs.
- The false alarm could have been triggered by someone on the payroll of NK. Note that the main goal of the military is to inspire fear, not to actually kill people.
In any case, as these systems are powered up, updated, overhauled or even replaced, there's testing going on. And when there's testing, there's the opportunity for just this type of gaffe.
Is this design horrible? yes. Is this out of the ordinary? Absolutely not. This is very common UX for systems designed 5+ years ago and there are still systems designed today that look like this.
This is an alert system, now picture your surgeon or air traffic control using something like this...
The problem is that two drastically different interactions are too easy to interchange by mistake.
This application could easily look-and-feel the same while having much a better user experience.
This screen is scary. Now imagine drugs used during surgery that look almost identical.
Here are some exampels:
https://twitter.com/EzDrugID/status/933871776152543233
https://twitter.com/EzDrugID/status/850259195618131968
It's frustrating that we know this is a source of human error and that we're still doing it.
> 1. TEST Message
>> DRILL - PACOM (CDW) - STATE ONLY
>>> Amber Alter DEMO TEST
Design aside, a standard naming convention would have gone a long way.
1) How in the name of everything holy did this monstrosity of an interface for such an important system get approved in the first place?
2) Given a monstrosity of a interface: Why are there, apparently, ZERO safeguards in place to make sure the human error rate at least has a HUGE chance to get close to zero?
Good to know. This means that in Hawaii, you should only think about taking action if the magic FALSE ALARM notice never appears.
If the FALSE ALARM notice doesn't appear within an hour, it might just be that the false alarm guy is on lunch break. If it doesn't appear in 8 hours, it might mean that a shift change occurred without proper hand-off. If it fails to appear in 24 hours, a nuclear attack might actually be imminent, and you should check other sources, to determine if the alarm was, in fact, real.
This protocol ensures that archaeologists will receive a valid record, omitting the false alarm notice in the rocks on the other side. Eventual consistency ensures that before the archaeological record is written, a false alarm notice will appear, if the alarm was indeed false. In all other cases, the alarm is effectively revealed to history to be a true alert, at some point in the future, as yet to be determined.
There was a 38 minute delay before the false alarm message: https://www.washingtonpost.com/news/posteverything/wp/2018/0...
Meanwhile an ICBM would likely impact less than 35 minutes after launch: https://en.wikipedia.org/wiki/Intercontinental_ballistic_mis...
- How much for a new web interface? 1k?! what?! that's outrageous!! just add another link, it will be fine.
Or they don't get it and they are hopeless.
No, they do get it. The DRILL link is one row away from non-DRILL link to avoid incorrect dispatching. In the mean time you can ponder if you need to announce a tsunami.
I would suggest that recent history [1] has shown we can have a 30m-2h warning, and that isn't good enough.
[1] https://en.wikipedia.org/wiki/2004_Indian_Ocean_earthquake_a...
This is the kind of thing we would want the gov to do and lie to us about because most people can’t handle the truth.
“Put the links really close to tether so we can use that as an excuse.”
This is a joke right?
It’s only going to make recovering from a mistake easier, it’s not going to prevent anything.
How could multiple people accept this as the interface? Put it on a separate page! Have an explicit confirmation dialog, or 3 of them. At the very least maybe it's own section with some padding...
Ridiculous.
How many people are you willing to kill to reduce the risk of a false alarm?
Really, there is no way to justify this as in any way "intentionally designed".
I agree that the 'real alert' should not be impeded and they could at least put a red or black rectangle around the 'real alert' button to make it stand out from the others.
Actually, considering that timescale and all the variables involved in detection and tracking a few seconds is less than the margin of error for predicting the warhead's arrival. While the goal should obviously be to get a (valid) warning to the public as quickly as practical, the "every second counts" mantra in this situation is overly dramatic. In fact, I think it is even counterproductive because it can lead to a "better safe than sorry" attitude that triggers unnecessary false alarms.
> How many people are you willing to kill to reduce the risk of a false alarm?
Depends. How many people might die in a false alarm? How many people might die if they stop trusting the alert system and fail to properly react to a true alarm?
Lol
I wonder if there is a clear plan, and if there's a better way to communicate that to the population. That seems like maybe it should be more of a priority then this alert message, which seemed to only sow confusion.
The "Duck & Cover" stuff is lampooned for being rediculous, but it's actually what you should do. The idea that everyone will die from being incinerated in a nuclear blast is a misconception. Most people in America would survive the initial nuclear blast from a full Russian strike (and China, etc. have even smaller arsenals).
In the short term, act like you would in a Tornado. Basement or if you don't have one, an room with no windows.
Longer term--stay in side for as long as you can because fallout will disipate. The longer you don't go outside the better.
1: Launch nuclear attack
2: Make coffee
“Around 8:05 a.m., the Hawaii emergency employee initiated the internal test, according to a timeline released by the state. From a drop-down menu on a computer program, he saw two options: “Test missile alert” and “Missile alert.” He was supposed to choose the former; as much of the world now knows, he chose the latter, an initiation of a real-life missile alert.”
https://www.washingtonpost.com/news/post-nation/wp/2018/01/1...
But (looking again) I see in a reply they say the image was provided by the office of the Governer.
https://mobile.twitter.com/CivilBeat/status/9532704728204574...
The amount of times I've heard 'Just add a button to do it' from 'backend developers'..
The point is, software is there to serve a user, and if you can't envisage the user using the software, then you don't understand the requirements. It's not a design question.
Too much responsibility is deferred as 'design' when it's actually a fundamental part of solving the problem at hand.
Not that design isn't also important, but the perception that usability == design needs to change.
Ironically, moving the ballistic missile alert to an actual physical button separate from everything else would have been better than... this. This looks like it's literally just an HTML page on an internal network. Do they send the alert command through the query string or something?
You don't need to question the narrative publicly like some of us do, but it's always good to keep an open mind about everything we're told, and sadly, that means considering the possibility that there have been more lies than truths told since Saturday.
Hopefully someday we'll get the real story, but I have a very hard time believing that this is it.
I find it sad that people have been conditioned to lump anyone who questions such a messed up narrative in with nonsense like that.
Let's not act as if we've never been lied to before. We have reached a very low point in both credibility and transparency, and just because I have serious doubts about this story doesn't mean I'm a flat earther or whatever else you're insinuating. Let's be mature here.
I can see a bunch of reasons for clicking that link, from just for fun to wanting more money to fix the UI to the more sinister of wanting to keep the populous scared so that they are more likely to support a war in the future.
No reason to bring lizard people into this.
The lack of an actual ballistic missile or missile defense response should really settle the question, IMO.
If we are able to identify the later story as real that means we have some other mechanism for determining truthfulness which could be applied to the current information.
Do you have any reason to believe we have been lied to? Can you point to any sources? Your vague statements suggest you know something but are unwilling to reveal it. That is a red flag for me.
Gives a whole new meaning to critical vulnerabilities. Sheesh.
A site is vulnerable to XSRF if it doesn't use tokens when performing critical operations, critical operations are (usually) performed using HTTP POST, which can be done via form submission... token generation and validation is done server side...
You can perform a successful XSRF attack in a browser with javascript completely disabled.
It would have used more than 2 colours.
It would have used consistent capitalization.
It would not have underlined everything.
It would have divided up menu options into related groups.
* http://www.cheapraybansunglassesa.com/wp-content/uploads/201...
* https://i.stack.imgur.com/8LzMo.jpg
* http://thinview.com/images/emulator.gif
* http://hunterstrainingassociates.com/images/ispf1_25.gif
* https://as400iseries.files.wordpress.com/2013/03/libraries1....
* https://seasoft.com/assets/seasoft.com/seasoft.com/public/up...
Instructions:
* Reply with your suggested fix to this comment. (That way we can vote equally between suggestions, instead of having them being scattered throughout the thread).
* First line is simple one-line summary, optionally followed by a blank line then some exposition.
Example:
----------
Simple fix: Seperate links into 2 color coded sections, TEST and ACTIVE
Put each set of links into seperate divs, with different background colors or striped backgrounds.
Put each set of links into seperate divs, with different background colors or striped backgrounds. As is, these different links are mixed up together making them accident prone. Undoubtedly a mini development boondoggle will result over this incident, in the meantime the above fix is easy and satisifies the real requirement.
Already suggested here: https://news.ycombinator.com/item?id=16158252
I'm not really sure where the problem lies, partly economics, and part lack of specialized knowledge. In this case that knowledge might be laws around AMBER alerts, how to respond to an RFP for such a system, or even just not knowing that such systems need to built.
How is it possible that they don't have standards while the OEMs do?
And I don't even think it's up for debate. If the design of the page just made someone accidentally alert a whole state of an incoming missile, then it's bad.
But I've read a lot of suggestions like spreading items across multiple pages, adding passwords, or big warning colors and lines around the live options that are not just obfuscating/distracting/annoying but are bound to cause errors and open the door for even worse design.
fredley 36 minutes ago If a system has critical safety components (using, or misusing the system could harm or kill people), all parts of it should be treated as such. This applies to hospital equipment, missile warning systems, cars, etc. Things like security and reliability rightly get a lot of attention, but UX is just as critical, as this event shows. There are plenty of case studies of poor UX on hospital equipment killing people[1]. When will people learn? https://medium.com/tragic-design/how-bad-ux-killed-jenny-ef9...
fredley 33 minutes ago This is not graphic design, it is UX. Graphic design is a component of UX, sometimes, but not here necessarily. Simply reordering the list, and giving it a hierarchy makes it much easier to see what to do: https://twitter.com/iamlucamilan/status/953201356545974272
If @Github would have built the Hawaii text alert service: https://twitter.com/mattiasgeniar/status/952929404925202432
Btw: any idea what this application is? looks like a HTML page ?
This is a civilian system. Detection and anti-ballistic missile systems are military, specifically NORAD and the MDA. I agree that NORAD should have an API, though I suspect integrating it with civil defense systems will result in more such mistakes in the short run.
PLEASE ENTER STATEWIDE WARNING CODE [ ]
then the operator types NUKEWARN or whatever in the box.
Much less error prone.
Probably a panel of unlabeled buttons
a terminal based system would be better than this, how much did this system cost?