Hacker News Reader

> Always a machine at the root of the problem.

I have a perfect example of this.

Years ago, my mom's coworker discovered that her bank card no longer worked. ATM's would error. Online systems would reject all transactions.

So she went to a bank branch to sort it out. After explaining to the teller her situation, the teller brought up her account details, looked up from her monitor and said, on a straight face: "You are dead."

What had happened was that she had had a full namesake who had also had an account in the same bank. When the other person had died, and her estate had closed the accounts, the person doing the account termination had only looked up account owners by their full name. Closed all of them, with the explanation "account holder deceased".

Human errors happen. So do computer errors. It only becomes a problem when humans rely only on what computers tell them. This particular bank teller didn't even realise that the facts stated by the computer were contradicting what was literally standing in front their eyes.

Critical thinking is a dying trait.

Isn't it exactly a human error that you are describing? The root cause was a distracted employee that input wrong information in the system, not the system being flawed...

Well the problem was that of identity. The bank assumed that a namesake was the same person, rather than relying on a more unique identifier.

The root cause was a system that allowed a potentially incompetent employee to indiscriminately close bank accounts without at least a date of birth or social security number confirmation. It's a problem that could have been prevented with a four to eight digit input field and an if statement.

The system isn't flawed only in the sense that the bank doesn't give a shit. They get to reap the returns on your capital either way.

Sure, the root cause was incompetence when closing down accounts but that is not the problem.

The real problem manifested in human blindly accepting what the computer told them. If there is no reason, or even NEED to apply critical thinking, why is there a human involved at all? This is not an indictment of automation over human labour. I would rather that humans are in the loop precisely because they have the ability to spot these kinds of errors - and help to correct them. Computers are good at repeating mundane tasks. Humans are not. We should be making most of their distinct abilities, not molding the two into same form.

Now, Temporal rightly pointed out that this might have been a case of attempted fraud. But if you are trying to spot fraud, spotting anomalies and being critical on what you accept as objective truth should be on the top of your mental map.

> This particular bank teller didn't even realise that the facts stated by the computer were contradicting what was literally standing in front their eyes.

It may be more complex though. The machine can be wrong, but so can your eyes. The previous customer coming to that teller with the same situation might have been someone who stole the identity of a deceased person. You can't just assume the machine is wrong because it disagrees with what you see.

This is also not the problem of machines per se, but of large systems - where different parts are handled by different people who don't know each other (or may not even be aware of each other's existence), and a set of fixed procedures (i.e. meatspace algorithms) are used to coordinate everything.

Are you sure the teller didn't just have a dry wit?

Or did they then proceed to refuse to deal with the living dead person?

So what happened to the money in the account? I mean, when you close an account(s) of a deceased relative at a bank (presuming you have the legal right to do it - it takes more than just "showing up" and claiming such), the money is generally withdrawn and processed as a cashiers/bank check (depending on the amount - for certain large amounts, there may be a wait period) - and given the person closing the account(s).

...so - what happened to your mom's co-worker's money?

The bank refused to admit error, at first. It took more than two weeks of constantly being on the phone and pestering before they reopened the account.

In the meanwhile, it was pure luck that she didn't have any critical expenses, such as taxes during the time window.

The bank never issued an apology.

Traditionally, the original paperwork machines were human-based and called "bureaucracy". They were well-known for being extremely infuriating, even before they were ported to computers.

The problem here is trusting the machine and not trusting the actual human and not giving human any leverage to do anything extra.

In all cases where machine and human are assigned to a job as a unit, human should have authority to override automated decisions.

There are two reasons why it isn't so: business doesn't trust its drones, and workflow tends to break down on unexpected input. But it should be.

With full audit trail of the overrides that makes sure the human gets sacked if they fuck up, sure. I dislike limiting human agency, but unfortunately, most employees aren't True Believers of the Corporate Cause, so you can't assume they'll be loyal and diligent all the time.

Full audit trail is the first thing you should put in when building a back office system.

Some good points you bring up.

This is not a response to your entire post, just a small one to "where we don't want a rigid machine-like process" - I think this is something that machines are perfectly capable of solving. We are not arguing about whether or not, in the ticket printer case, the machine should or should not print a ticket outside it's rigid specified rules (late by 1 minute etc). In fact I think that most of us would agree that as a matter of principle such grace-periods are a fundamental lubricant of a comfortable society. We just have (historically) had to plead our cases and rely on the grace of other humans to grant such meager exceptions to rigid rules. We are just arguing at this point about what the numbers for such grace periods should be, machines can easily implement them once we find a happy medium. My $0.02 anyway.

If we implement a 60 minutes deadline and a 10 minutes grace period, we might as well just set the deadline to 50 minutes and be done with it.

I'm an advocate for a hybrid approach. Get machines to do the bulk of all the work, and have a human handle problems, edgecases, and "bending the rules for loyalty".

I totally agree: generalizing something always has a cost. Not every experience with a machine is smooth, not every experience with a human is smooth. Therefore we need both! And so far only humans are able to improvise, understand, use emphaty... But machines are fast, scale easily, never stop working (haha)... Both is the wise choice!

Arbitrary deadlines like this are heuristics that we give to systems or people who don't have realtime access to the full picture so they can make decisions that are probably, on average, more likely right than wrong. The human checkin clerk can, at their discretion, call the gate and see if boarding is delayed, or find out how the security lines look, gathering additional information to make a decision, but the one hour deadline is a rule of thumb that they can use to make an initial call. But even with the additional discretion they have, they are still guessing whether the rules are worth bending for this passenger, what the real value to the airline is of getting that customer on this flight or bumping them to a later one.

A computer replacement that doesn't have any additional data, and isn't programmed with the discretion to make those calls to find out additional information, is going to just make the call that checkin is closed. It is probably an inferior solution to the human.

A computer replacement that has information about current security wait times, gate status, as well as information about all other flights, the location of the inbound plane, the estimated lifetime value of the customer standing in front of the kiosk, what competing airlines are doing, and the exact current company financials, can put all that data together to make far better decisions than the human checkin clerk could, in the best interests of the airline, and it doesn't need to be given arbitrary rules like 'turn people away if they show up 59 minutes before scheduled departure'.