Shitty Software Kills People
val.demar.in
val.demar.in
I remember a specially egregious case that one of the rental companies was guilty of: automated criminality
The software in question automated the reporting of grand theft auto to authorities. The rental company used it to streamline reporting of rentals to police.
The problem: false positives. The software would report drivers as criminals, when in fact cars had been exchanged midrental due to car failures etc, and the original car hadnt been returned under the normal workflow, which the software failed to account for.
Then that police report would chase innocent people for literal years, coming up to unsuspecting customers during routine speeding stops.
I always wonder whether contributor names to closed sourced software should always be transparent.
Excellent engineers should be able to take credit on their work and there would be more skin in the game. Particularly for employers that get suspect resources from to solve for budget constrains artificially imposed by business managers with no stake in the product functionality, to critical software processes like SREs etc
After a certain point, we recognize as a society that just because you make chairs, your not at fault if someone then takes a chair you made and then bashes someone on the head with a chair. It's the basher's fault, not yours.
you also don't know if they have any information given regarding possible workflow deviations, or if it was made when they had one workflow and workflow was changed without paying for changing the system. If that were the case it would be an example of shitty management gets people harassed by the police and potentially killed because getting harassed by the police at the wrong moment, or looking like the wrong person, in the wrong country can get you killed.
The police should pursue it cases like this, and arrest the decision maker in the organization that caused this to happen. If it’s been happening repeatedly, the person’s sentence should be increased accordingly.
Which is why bad software can hurt people a lot.
The OP's point is easily generalizable to so many domains; anyone who has written enough commercial software has seen bugs like these, though perhaps not with as severe legal consequences.
I had a friend - who happened to be pretty well-versed AND interested in legal matters - who was sued by someone for some car accident. The thing was, the identification came from a license plate, and he lived hundreds of miles away and drove a completely different car.
He finally got in touch with the attorney - who didn't care - and made him care. Took time and effort. Would have cost a normal person a pretty penny.
If such crimes are already documented as crimes on the federal level, then it'd only require an executive order from the President to get the FBI to start tracking that particular crime down.
Are you implying that the developers that created this back office tool should be liable for the false police reports?
If so then then counterpoint based on having made back office apps: the individual contributor (i.e. developer) may either (1) have little to no say in what the app is doing as they are simply implementing the requirements as provided and/or (2) have no knowledge of how it might fail (e.g. when the app is integrated as part of larger business processes)
If you build a system that orders coffee capsules, you want to make sure you don't order x100. But when you report 'thieves' to the authorities you open yourself up for plenty of lawsuits.. imagine the poor people being harassed by the authorities for some crime they didn't do, and will take plenty of time/money/effort to clean-up, all because company XYZ does not have a thorough UAT..
Unfortunately we don't have to imagine this. This is exactly what happened (Hertz was the culprit, see link downtread)
You may not even need that word "poor" inserted in there in fact, the average person in middle america may not have the resources to to avoid a night in county jail, due to a wrongful imprisonment / accusation.
> Are you implying that the developers that created this back office tool should be liable for the false police reports?
In other fields, having an engineer's signature (or the lack of) means something, legally speaking. Good luck getting any kind of insurance on a new construction that wasn't inspected or built by someone licensed.
The last paragraph where the institution disband the oversight and destroys results day before report is due is infuriating :O
At the very least, for this to be fair, that list should include management, up to the CEO.
In my opinion, absolutely not! The company or corporation holds all the cards: they manage the budget, the schedule, the hiring firing and promoting. The company is responsible to marketing and what sales people are telling customers. In most cases every bit of code the developers produce is wholly and entirely the property of the company.
Blaming the developer, to my mind, only empowers companies making software like this to make more of it and to sell it more widely. I would argue that we should see harsher penalties for organizations that produce and market (or mis-market) software that is causing harm.
> Then you have to print it.
> You fill it up with a pen, trying to remember, find or recover information which you might or might not have access to (but that you know for a fact that it’s stored in multiple databases behind some firewall). Then go to a physical location with your little printed form, queue up, and hope for the best.
Sounds more to me like an organizational inflexibility problem. Someone somewhere refused to change the definition of 'signed' and mandated that people print something. Someone decided that they couldn't autofill the form. Someone decided that submissions will only be accepted as physical forms.
This is sort of my hobby horse since I worked at a company that interfaced with healthcare providers that insisted that we accept faxed documents.
Bureaucratic operating procedures are often essentially malicious software that de-prioritizes the needs of ordinary citizens far below the priorities of the organization. An organization may pay some lip service to a mission of service to citizens, but usually the organization's actual, absolute top priorities are self-preservation and expanding its budget, staff, power, and influence. A secondary priority may be reducing fraud and punishing cheaters, even at the expense of honest citizens.
Though sometimes (e.g. California unemployment insurance) you get perverse outcomes where fraudsters are richly rewarded at the expense of honest citizens.
Online processes and APIs are great but we have to have backups and failsafes.
The failsafe is "find ways to give everyone internet access", which I think we've already done.
Libraries, public wifi APs, cheap 4G, dirt cheap internet capable devices, etc.
Foisting support costs off to libraries is another example of the tragedy of the commons.
No idea how you think this is a tragedy of the commons situation. It is definitely better to digitize everything and serve the 0.1% (certainly not anywhere close to 10%) with additional support that helps them use the digital service, rather than having non-digital fallbacks for significantly more expense.
Internet services becomes more of a win if you're remote, not less so, espescially with things like Starlink coming online.
I lived in a very rural area for about 50% of my childhood - we had no fire department, we relied on a well for water, our town's official population was in the hundreds, etc. If we wanted to get on the internet we had to do it at around 38400 and random packets would get corrupted because the lines were old and low-quality (I wrote my own parity tool in VB6 so that I could download a file 3-4 times and produce a not-corrupted version). Despite that, if I wanted to call someone up on the phone to speak with them or send a letter, that was just as easy and effective as it was when I lived in the suburbs.
When we eventually managed to get slow DSL years later it was INCREDIBLY expensive because in rural areas if you're lucky enough to have service at all the monopolist who offers it knows they can charge as much as they want.
If you don't have a sufficiently expensive phone. In an effort to prevent easily creating cheap fake accounts (presumably -- I don't actually know), phone numbers associated with low-end android devices are routinely banned from social media platforms, online banks, and any number of other organizations.
Any references where I can read more about this?
- https://www.paypal-community.com/t5/About-Settings/Tracfone-...
- https://www.paypal-community.com/t5/About-Settings/Paypal-do...
- https://www.reddit.com/r/TracFone/comments/g42vl9/phone_numb...
- https://help.id.me/hc/en-us/articles/360017839774-Why-can-t-...
At my company everyone has gone API-crazy. There's one activity in a team I used to be on that used to require a human review of a configuration change. It's something that developers on the team do just a few times a year. Someone got the idea that this should be fully automated, because I guess it would look good on their performance review if they built a new service?
What we had at the start was a description of what you were trying to do and how you could do it, but it included a manual review of a change request. It was simple, understandable, and required very little human attention.
What we ended up with was an API for a microservice with its methods half-documented and no "big picture" description of what API calls you were supposed to make in which sequence to get the job done. API calls would fail with obscure misspelled error messages, and you'd have to guess what you did wrong in the JSON file you were submitting. And then of course it ended up not working half the time, and you'd have to open support tickets or ping the entire dev team in chat to ask why you weren't getting the results you expected when using the API. Of course the initial response tended to be, "Did you read the documentation?" And the user would have to respond, "Yes. Here is what I did after trying to decipher the so-called documentation, and here's why. Here are the results I need. I'm not getting them. How can I figure out why?" At which point two or three very annoyed developers would ask for details on exactly which API calls were made with which parameters, and they'd poke into logs and whatever, before they figured out yet another untested corner case and manually fix the database entries while saying, "Well don't do that next time."
Sure, if you've got to scale to thousands of requests a week or something, building an API with a microservice might make sense. But there's a point of diminishing returns in trying to automate absolutely everything behind APIs. Some very low QPS things really should have a human in the loop by default.
At least the web is an open standard.
I’m sure we can find people who have died simply because they can’t navigate a webpage or an app.
As techies we assume having an app for something is the way to go - but there are many who do not have access and being able to fill out a printed form and submit it by hand is a better solution for them.
I haven't had a piece of governmental paper in my hands in my adult life (just over a decade). However, in Denmark we are very dependent on a working public IT infrastructure. After all, we are one of the countries that registers the most about their citizens.
Well, sure. But bureaucracy saves people too, and probably the balance is overwhelmingly to the second. It’s just hard to see things when they work like road safety regulations, food safety regulations, building codes and the like.
And when things work, few people complain about them; those who do may be ones who suffer a local inconvenience or lose cost-free access to a negative externality.
Agree, but I also think the point the author is trying to make is correct but maybe unclear - The main goal is to get people the vaccine, period. Everything else that is being done is being done in support of that goal. Not everyone has direct access to a printer, should that be a reason someone should not get vaccinated?
Paper ballots are great for elections, but there's no good reason paper is required to get someone Moderna/Pfizer/AZ/J&J et al.
About 10% of the people who showed didn't have smartphones or text, 4 people did everything only with phone calls. I wonder how many people who only did phone calls were NOT able to get vaccine slots?
Working with my aging mother, modern UIs are very complicated. Ever changing gestures, hidden menus, differences between machines, passwords, multi-accounts, etc. Heck my mom can't figure out the volume buttons on her iphone so she holds face time to her ear to hear. Heck even the default apps are hard on her. iMessage keeps getting moved to the home page and SMS to the 2nd page. So she sees the blue bubble and uses that to message us, even tho we are sending SMS. She needs to use the green bubble from the second page.
This is not to mention how bad all of these apps or websites are and the loss of a person to call to get clarification for anything not written on a website.
95% of my time yesterday was spent just answering nuances to the basic questions. For instance we were giving Moderna to the first dose people. Every time I announced that to the line, half of the 2nd dose people panicked and said "I need Pfizer".
For me as a programmer, keeping it all in my head was easy, it's just a 8 branch 3 level deep if/else statement.
For aging people with phones the information became inaccessible.
Those with paper printouts of everything were able to just breeze through and paper doesn't reboot, loose battery, or forget, and you can easily hand paper to other people.
The rate of vaccinations actually crushed the brand new system in place (which was there because the old system melted at 300 people a day). So paper was great. They just did all the data in put at the end of the day.
I'm actually quite worried with our current hodgepodge of low quality software we're greatly isolating older people, or people who are not using tech heavily every day.
And this is not just for crappy government software. I saw as many people unable to use their iphone and android basic software. With my mom, netflix, why does it even have a password she has to remember? Device authorization between her roku and a laptop is confounding, they're 2 totally different devices. It's basically impossible.
And it's not their fault! People have only a limited capacity for things. Imagine if your car changed how it worked every week or so when you sat down to drive - driving would be nearly impossible.
>>>> The number of people over 60 who didn't understand the difference between a text message and an email was around 30% (out of a sample size of 600 during my shift).
What is the difference, in plain terms?
It's easy to understand the technical difference if you remember that it's based on a historical quirk of cell phone technology and economics from 20+ years ago, that is probably not fundamental or useful today.
But try explaining it to a non techie: It's a different app, and happens to be more or less responsive depending on where you are located relative to the antenna towers, so the best explanation is just that you have to check for messages in two different ways on your phone, and not trust to receive a message immediately after it was sent.
Actually understanding the difference is as important as knowing why dialing a phone is called "dialing." For the rest of us, just make it work already.
The over 60 folks didn't care why text or email, they wanted to do what they needed to do to get their shots. The system we were using was setup for text. It was a tech choice. To them tho, it's a phone number, and they never text, that's what email is for. Why should they have emails and texts and phone calls and because it's hippa, an in web app email system to keep their data secure.
For my mom I'm referring to apple's pushing of iMessage over SMS... she doesn't comprehend the difference and apple's push to get people into their ecosystem is making it impossible to communicate with my mom without carrying around an iphone all day just to respond to her.
All these apps implementing in-webapp messaging systems that suck, then sending me an email with no information other than that I got a message in a system I look at maybe once per year.
Tech people overstating "tech quirks" rarely elicits anything worthwhile.
Also why are some messages green and some blue? Well, one in the phone company's native service and the other is iMessage. What's the difference? Well, green may cost you more depending on your phone plan. Why? ...
Both serve the same purpose, transfer a piece of paper. One of them does not get delivered if you forgot to lick another piece of special paper onto it (stamps) before dropping it into the mailbox, oh and btw make sure you drop it in the correct mailbox, it is not the same mailbox where the orher envelope works.
One gets delivered to the door while the other is delivered to your closest pickup point.
Remote delivery is complex, even the old way. Sadly such technology long ago stopped being about making things simpler for customers, unless the entire end-to-end chain is inside their walled garden and making it ever harder to get outside of it to where all your other friends are.
I agree with your overall point, but I am puzzled by this detail. There is no separate app for sending SMS on iOS. The Messages app handles both the iMessage protocol and SMS, has a green bubble icon, and by default, appears in the four-item dock at the bottom of every page of the home screen. Also, app icons don’t move between pages on their own.
I'm 41, and, you know, the sort of person who reads HN, i.e., can find my way around a command line, sling enough code to get out of a paper bag, etc. And I can't make sense out of many modern UIs.
Half of them are just forests of unlabeled icons. Like, ok, what is the thing that looks like a squirrel making out with a lawnmower supposed to do?
The other half are menus inside menus inside menus inside menus where finding the functionality you want is damn near impossible.
No government or corporate webform gives you all the information you need up front---no company discloses the full price until you're about to put the credit card in, no government tells you what documents you need until you're ten pages into the application.
And don't even get me started on fashionable startups. I just joined an organization that used (past tense, I helped kill it) Basecamp to organize communications, and that thing has a totally different arbitrary nonlinear ontology from every other preexisting human communication technology. What the hell is a campfire and why can't I find my pings and what were the designers thinking to just randomly drop all of this extra cognitive load on people's heads like a pigeon with diarrhea? UI designers, please, just stop!
Shiny and new is not necessarily better than easy and works.
UI designers are mad, mad I tell you!
The number of transactions where the full price is shown upfront vastly outweighs the number of hidden extra transactions
It seemed to be a rant based on a world which doesn’t exist
I am a software developer with over 20 years of professional experience designing and building systems. I recently got the COVID vaccine. While I was in the observation area, there was a screen projected advising me to set up my 2nd dose using my smartphone while I waited.
I searched for the clinic web site where I first signed up for the vaccine, and then I filled in the same form again that I used to get a my first appointment. One of the options was "Have you already had your first dose?" I checked that, and my expectation was that the system would determine that I have priority for setting the next appointment.
Instead the system kicked me into a waitlist. Fortunately, as I was leaving, the staff were adamant about asking whether I had my second appointment set up. I explained that I used the website for setting up appointments and that I was on a waitlist. They were confused and said I should have been immediately scheduled for my second appointment. Fortunately there were a couple of staffed stations set up where they could set up your second appointment if you had trouble doing it via your smartphone.
The staff member told me I had gone to the wrong website (??) and, in a slightly annoyed demeanor, asked me for my email address. They did whatever they do on their end, and I got a date and time set up.
I think it's perfectly reasonable to go to the website you initially used to set up your first appointment in order to set up your second appointment, and I'd expect the system to give you priority in getting your second appointment when you check the box indicating that you've already had your first dose. (Otherwise, why even bother asking??)
I guess the thing I was supposed to do was search my email inbox for one of three messages where there was an embedded link I was supposed to click on. But this wasn't discoverable, and none of the information on the screen told me that was what I was supposed to do.
If an experienced software developer misuses the automated system, I'm not sure how the general public doesn't run into problems using it. Of course I do have a knack for always wedging myself into the corners of almost any system I use, perhaps because I don't think like your typical member of the general public does when it comes to systems.
One of the reasons i'm getting sick of all this cloud crap and autoupdating software is that the changes are mostly cosmetic bullshit but I often have to re-learn or re-associate things. (or worse, constantly check my settings because they conveniently change in company-x's favour after a "reorganization" to give you more control or someshit)
Sometimes changes come along that are legit improvements. fine. i'll re-learn. but i'm just getting fatigued with bullshit changes.
The Massachusetts system seems pretty good with easy pre-registration and choice of ways to be contacted when you're up for an appointment. Problem with some of the other systems is you pretty much have to keep checking. I've been tempted to write some code.
[0] https://docs.italia.it/italia/developers-italia/gl-acquisiti...
I've very often told myself during the past months, witnessing (my) french government : we're really fortunate nobody is declaring war to us, because we would be run over in a few days, judging by the time it take for any administration to get things done properly.
They even make this case
Not software - "you have to download a bloody pdf. Then you have to print it. You fill it up with a pen...."
BBC - "Excel: Why using Microsoft's tool caused Covid-19 results to be lost"
But it worked. Excel is brilliant software with decades of work, it was the (Shitty) scripts that 'failed' ignoring BBC Excel headlines.
Except they got the scripts running, made a mistake, fixed it and moved on pretty quick.
The less insane use of Excel there was as human interface, where Excel files were used both to query external services, but also as input files/forms that could be sent as attachment and "cosigned" by appropriate person.
Yes it is stupid, but not everybody can download a PDF
Remember when the NY state comptroller complained about how the process was complex?
If there's one thing that GDPR should have put into people's head is not bothering with data you don't need. But nooo, everybody prefers to keep collecting the same data (either a form or advertisement data) then add another tick box. People's love for bureaucracy is something
(And I also might add, people's love for complaining sometimes it's bigger than their want of solving the problems)
Paperwork is not a bottleneck of the Italian vaccination program (and many other EU vaccination efforts). AstraZeneca not honouring their obligations to EU while prioritizing UK, is actually killing people. There are tens of millions of doses many countries counted on, which are not delivered. That's what's killing people, not the inconvenience of filling a paper form.
1. Go to the Smith's website
1.1. Endure some fucking chatbot to determine eligibility
1.2. Finally pick a date/time for the first dose
1.3. No options for the second dose, so I can't continue. No vaccine for me. How the fuck am I able to pick a first date and not a second? Shouldn't it be the other way around? Why even give me the option for a first date if no second dates are available? Why are you even making me pick a second date when y'all are doing the one-shot vaccines anyway?
2. Try Walgreens' website
2.1. No vaccines anywhere, so no vaccine for me. Cool. At least it was a quick response.
3. Try Walmart's website
3.1. Create an account, because reasons
3.2. Choose a store
3.3. There are inexplicably two options for "16+ with underlying conditions", and both have to be checked for me to be eligible. Alright, whatever.
3.4. Find out the chosen store doesn't have any vaccines anyway. If it doesn't have any vaccines, then why the fuck was it an option, Walmart?
3.5. GOTO 3.2
4. Try Safeway's website
4.1. Redirects to Albertson's for some reason, and no vaccines. Cool, thanks for the quick answer.
5. Try Sam's Club
5.1. Sign up for a "guest" account
5.2. Go through yet another chatbot for the basic eligibility questions
5.3. Go through a half-baked web form with a lot more medical questions.
5.3. Pick the first open timeslot.
5.4. Success!
This experience has (re)taught me that if this is what it takes for people to get vaccinated, then shitty web design will - not might, not could, will - result in actual loss of life. There's no way I'd expect my grandparents to have been able to work through this (luckily they started with Sam's, and as byzantine as it was, at least it worked).
Like, it's 2021. How fucking hard is it in 2021 to put up a simple web form without multiple megabytes of broken JS driving it? How hard is it to setup appointment scheduling? How hard is it to track inventory of a SKU? These things have been done electronically for decades now, and we still can't get it right?