USPS Isn't Paying 45,000 Rural Postal Workers This Week
jalopnik.com
jalopnik.com
We have been doing payroll for over seventy years. To fail to implement a payroll is a dreadful mistake, and a person is responsible
When civil engineers make mistakes like this there is a postmortem and we learn what the mistake was and how not to do it again.
Until we start taking responsibility, like real engineers do, computer programmers, we, will continue to let people down like this
Plenty of tech companies do post mortems. This is almost certainly a failure of management. If a single engineer caused this, the process is broken.
What the tech industry needs is professional indemnity insurance and competent management.
https://www.cbc.ca/news/canada/saskatchewan/engineer-designe...
Edit: And here is one from a crane collapsing:
https://www.constructionnews.co.uk/health-and-safety/housing...
Though... I have managed to avoid both startups and government work
I've had some experiences working on some popular software, and have definitely seen people trivialize the impact of a bug in ways that do not appreciate that end users are putting the software in the middle of their life, and hence not understanding what it means to the end user when stuff is broken.
So a programmer saying "dude relax, it's just a website" doesn't really hold weight to me. To you, a programmer who doesn't want responsibility or hasn't prioritized a bug, it's just a website. To the end user, maybe they are using it in a way you don't expect, they want it to work, and it actually is meaningfully harmful to them when it fails in some way. You can say it's on them for "misusing" your app but ... Again, that's blaming the user for your bugs.
A real world example I recall was a colleague who trivialized data loss and reasoned that nobody stores important stuff on that app. The customer complaint was that the "trivial" data that was lost was a recording of his deceased father, or something of the sort.
Negatively impacting peoples lives via advertisements is the raison d'être of many software engineering jobs.
Wow. I would be beside myself. Ecamm PhoneView is something I used regularly. They discontinued it and suggested moving to iMazing, but it doesn’t contemplate the same use case or workflows, and is a poor replacement albeit very useful and powerful for other uses.
This is why I strive to place myself in the customer’s shoes when building and improving something. Don’t take from them, listen to their feedback, let them guide the product within reason, and don’t lose sight of the fact that they’re paying you to work for their interests.
Most software probably doesn't require that level of rigor. IMO, those who work on critical software ought to be formally licensed like "normal" engineers who design roads, bridges, and so on.
I think about chiropractors when I think on this topic. They don't have the same training, the same level of rigor or skill (IMO). But they can still help people, hell some folks swear by it. But the point is, they aren't formally licensed in the same way and still make their living.
It isn't a perfect analogy, admittedly. Nor is it a welcome topic on this forum! Regardless, I believe some industries should require licensed software engineers.
> Okay this is probably something that needs a very high level of stability. But many apps/websites arent about life and death. If hackernews goes down no one dies. If a bridge goes down people might die. That is why civil engineering is more regulated (and work takes more time than if it wasnt).
Yeah, sometimes folks are working on something that doesn't matter at all, but that's the exception rather than the norm.
As a general rule, what we do can cost someone their life or their livelihood: sure my foul-up might not kill any users/ customers, but the lost ad revenue might cost some junior Nobody their job as part of budget cuts. That person loses their income for rent and for food, they lose their health insurance (maybe for their family), and I don't have a care in the world because my foul-up didn't impact me in any meaningful way.
Everything involves risk and we shouldn't punish individuals for taking a risk we told them to take -- but we often take risks without considering the consequences, and without _bearing_ any of the consequences. It's called a "moral hazard".
We do that in software too. It’s called an agile retrospective. Somehow it leads to more of the same crap.
And thats because management usually doesnt allow for proper engineering. They call that getting lost in the details.
So then we end up with things failing miserably again and again. But agile books get sold, mckinsey and other bastardisation factories generate profits, and so on.
Software “engineers” cling to their management driven paradigm and resist true accountability and responsibility for engineers because, ultimately, it’s easier and more profitably personally to hide behind the excuse that “management” made the decision.
Yeah just gonna stop you there...none of us were given the choice to be accountable and hide behind "I will lose my license if I commit/deploy this." I'm also an actual engineer in addition to a software engineer, and being able to say "I'm not willing to go to jail for this" works when you have a P.E. license in an actual engineering project, but definitely does not work in the software industry. It doesn't even work in engineering if you don't have a P.E. outside of "No this will literally hurt/maim/kill someone"
Yes, either way (software or P.E.) you might be fired for saying "I'm not willing to go to jail for this" but it's much less likely you'd be fired for it in "hard" engineering, because everyone else on the team understands that you're not being entirely hyperbolic when you say "I'm not signing this because I could go to jail." If you said that in a software project people would rightfully think you're fucking nuts and don't understand how the world works, because there's an infinitesimal chance a software engineer would go to jail.
Even the criminals who deployed the Uber Greyball thing didn't face any consequences, which truly shocked me, as that was blatant obstruction of justice.
I wish it was that easy.
In almost all software companies, with the exception of one where we built software security tooling, management asked for cutting corners, to which engineers objected.
Guess who got the blame when things went according to what has been said?
The engineers.
In one case it was so pathetic that the company lost tens of millions of pounds in revenue. There was a code freeze.
Turns out a marketing director _told_ an engineer to apply a change. That change tipped everything over. Fortunately the marketing director was “freed” later on.
Similarly in other companies people payments were either processed twice, not at all, accounts lost, user login details mixed or leaked and so on.
Every darn time technical debt was raised someone would say that we should not over engineer things.
But hey they did agile and unit tests.
My invoices got paid. It is what it is.
That’s indeed what makes it easy for management to consciously or subconsciously allow outcomes that do real harm like people not getting paid. There is no incentive not to be a %#^*up. At worst a manager looks bad. But they can always blame the engineers since there’s a system in place, so it couldn’t have possibly been the manager being a %#^*up.
A retrospective is not an incident postmortem, very different things.
A retrospective tries to look at how the agile process went wrong again this past sprint, but is not focused specifically about one production incident.
Companies should be doing incident postmortems on any incident of significance to identify action items to improve what went wrong on that specific case.
A retrospective’s documentation is essentially the smallest opaque piece of what an investigation would draw upon working backward. It happens fairly quickly after the causal factor is introduced and is likely to be completed before the software actually causes harm in production at a scale like this.
The results of a retrospective could make for a good source for recommendations, but only if it covered the causal factors.
If you can't make payroll with the payroll software, start hand signing checks, but don't blame "the software" and "the computer programmers".
For example, if you work for a state labor department… Public sector worker safety is regulated by… the state labor department. There’s an organizational bias to not rock the boat.
Any Federal or quasi-federal agency is a dictatorship. Some rural mail carrier in Montana or New York is 50 layers away from someone who has the power to affect change. You need the union to represent your interests and short circuit the org. Outside of the federal space, if a company fucked up payroll, the state labor department would unleash significant pain on the company. USPS is tasked with policing USPS!
If the union is failing some constituents, that’s going to drive a desire to get more effective representation. The benefit of focused representation may outweigh a smaller bargaining unit. (In general, it’s better to have 500,000 members than 50,000 members)
Jalopnik ownership is notoriously anti-union. They take a lot of swings like this.
The root failure here is USPS no question, but the union response is weak.
I wouldn't trust a Gawker site for any information.
The Union stuff is backstory and fluff to fill out the article.
My classes in finance said that rule #1 is to make your payroll (followed closely by pay your taxes). Failure to do this - even when the problem isn't cash flow, but IT failure - is inexcusable. It's the CIO's job to ensure that any risks to this are mitigated.
Fire them all. For cause.
Errors of this magnitude are either a symptom of some deep-running problem with an organization, or (much less likely) freak accidents.
In the former case, random firings aren't very likely to improve the organization; in the latter case, you'd be firing the people best qualified to do a post-mortem and making sure the same or a similar failure never happens again.
The rest of your post is more of the same “it can’t possibly have been me!” This isn’t a the website went down problem. People didn’t get paid here.
Dilution of responsibility is a real problem, but harsh punishment of everybody that, under some (your?) metric that made an error is definitely not the solution.
If you're truly interested in this (and not just angry and looking for somebody to take it out on, even though the root cause here is probably systemic – which is not the same thing as "inevitable!"):
The aviation industry and its culture of failure analysis and prevention is a great place to start, in my view. We have a lot to learn from that in our industry.
Important to note that this has been largely diluted in the US via regulatory capture, leading to things like the 737 Max disasters - so looking at the aviation industry a few decades ago is probably the way to go.
I believe that this USPS situation is a degeneration of internal fights that are lasting more than a decade now.. it is about money and authority, indeed
A lot of organizations, the company culture becomes so entrenched that almost no amount of C-level meddling will make a big difference. And this is especially common in government organizations (and in companies with very strong unions).
As far as I know, no one was fired. The Phoenix pay system for the government of Canada is a good example of so many failures. People really need to stop thinking that building systems with 10s of thousands of business rules is « easy ».
If I had to guess, it's an artifact from when people had to pick up their paper checks from their boss. Now with computerized direct deposits, I don't see why it can't be every week (or even daily).
The comment was asking why not pay monthly instead of fortnightly, which would be less frequently.
I was curious if there was some benefit for a company to pay more frequently when it appears to be more of an overhead.
Monthly seems too long for me from a worker’s point of view though, you’re effectively giving the company a short-term, interest-free loan at that point. And when you factor in the fact that your purchasing power is reduced by the time you get paid, and any opportunity costs, the real cost to the worker can be quite high.
The article emphasizes "this week" not because the USPS pays weekly, but because Friday of this past week was the standard bi-weekly payday.
That sounds like the most ridiculous thing I’ve heard based on the article.
USPS had a “catastrophical” payroll error and has been unable to resolve it.
This is USPS’s job and they haven’t been able to do it. The Union has stepped in and implemented a workaround that will be very helpful to the vast majority of USPS’s employees until USPS fixes its fuckup.
How is that a letdown by the union?
The USPS is, itself, an issuer of money orders. I assumed that's what they were referring to. I'm not sure at all how the union "implemented" anything here.
I gather the problem is that while the regular schedule time was accounted for properly and paid, the unscheduled time, substitute routes and Sunday / holiday Amazon package deliveries and that kind of thing was completely left out. Many carriers do a lot of that class of work in addition to a scheduled route.
Many folks who do a lot of that are going to be demanding verification of their pay for previous periods now too. Who knows how long they've been screwing that up? It gets insanely complicated with postmasters pretty much estimating hours and pay rates differing from person to person based on arcane rules no one really seems to know.
Where are you hearing this? Only thing I've heard in the last few years is that the USPS used to handle simple banking for people, and that it would benefit rural and underserved folks. I've not heard anything about the USPS "taking over consumer banking".
> One thing about the government is that you can't sue them for their mistakes.
Right, because for-profit banks have treated us so well in recent years...
And of course you can sue the government; people do it all the time. If you're harmed by a federal official in the execution of their duty, you can sue. You'll have to prove damages and negligence/malice, of course. Both the federal government and the state governments have explicit tort law laying out the circumstances within which a private citizen may sue the government.