Hacker News Reader

Comment by forinti | Hacker News Reader

I once worked with this awful third-party software that caused tons of trouble for our clients. We had an in-house solution on the works, but some people wanted to renew the contract and keep using the same buggy software. Sometimes I would spend the whole day just fixing stuff on this system.

I made sure all the tickets opened by our clients went to the people who wanted to keep the old solution. It worked, we moved to our system and we rarely have issues with it.

View on HN

I've found that the easiest way to effect change is by making the people who make decisions feel the pain of the problem. A lot of line levels do a lot of heroics in an attempt to keep pain from rolling up hill

Instead of the organization just doing the work to stop the pain, they throw bodies like so much human analgesic at the problem - I hope you'll let me torture this metaphor some more - and become addicted to this state of affairs.

Yeah I totally agree. I’ve met a lot of people who feel like a hero when they work crazy hours to “save the day”. But if the problem they fixed was caused by bad planning or bad decisions made by upper management, their heroics prevent the decision makers from learning how to run the company properly. Pain is an incredibly important signal. Our bodies need it to survive and companies are no different.

Go above and beyond, sure. But never ever be an invisible martyr. If it’s management’s mistake, management needs to feel some heat or they won’t learn.

A colleague once said that if you sacrifice yourself to make up for the firm's inefficiencies, they will never get fixed.

Which is basically what you've written. This concept should be discussed more often.

Tangent to this idea is the concept of 'back pressure' related to data flow, and the need to have a way to return feedback in a system.

I think reading this and other comments slid something in place for me about “Living Systems”.

What does the individual care more about tho, a chance at affecting change, or potentially getting fired?

Really shows the limits of an individual when their livelihood depends on the source of the problem

The trick to spreading pain upwards without getting fired is to communicate the situation upwards, repeatedly, ahead of time. “Hey I think that artificially imposed 6 month deadline is unrealistic”. “We’ll see”: Closer to the date: “That deadline is coming up fast. As I mentioned a couple months ago, I think we’re making good progress but I don’t think we’ll hit the deadline with the feature set we’re building. I think we need to cut features, push back the deadline or all start working crazy, unsustainable hours. And that’s not a precedent I’d like to start. What do you think boss?”. Etc.

If they don’t change anything, don’t suddenly pull all-nighters to then meet the deadline. A bad timeline was never your mistake to fix. And assuming you’re a productive member of the team, you’re not going to be fired for helping management plan like this.

I agree but my real experience with similar situations (twice so far) was at badly managed firms. I was ignored and branded a "trouble-maker"

The principle of spreading the pain upwards extends to making ownership feel the pain of bad management. But that’s a slow mechanism, and in the case of public companies, shareholders are so thoroughly isolated and insulated that it hardly works at all.

It works in the evolutionary sense - the worst compianies just die.

Counterpoint: SAP remains a going concern.

SAP is really fascinating. On the one hand, it apparently works. On the other hand, it seems to feed on suffering.

Hmm. That brings up an interesting take on “make work visible”.

>making the people who make decisions feel the pain of the problem.

At a macro level this is exactly why we're seeing such a strong disconnect between political/corporate leaders and "regular" people. The ones up top who make the decisions don't feel the pain of bad policy because they're so sheltered from day-to-day reality of most people

It's a really deep problem. Representative democracy, for all its faults, beats autocracy thanks to the improved incentives of leaders. Businesses as a whole, if they have to compete with others, face good incentives. But within a firm the incentives can be really screwy. One obvious response would be to abolish the firm and make everyone an independent contractor, buying and selling stuff from each other. That would improve accountability but at the cost of an enormous friction replacing a process that used to be as easy as asking someone to do something. Ronald Coase dedicated much of his career to the question of how to balance those forces, how far the boundaries of firms should extend.

Thanks for the reference to Coase, I'll be sure to look him up!

I often wonder how some companies survive despite bad or even catastrophic decisions by the management - it's only because the soldiers in the trenches kill themselves to get things done and to unf*k all the C-level incompetence.

And then it reinforces the management's conviction that they are absolute geniuses.

sometime ago this article was discussed here: "let it fail" :

https://www.maxcountryman.com/articles/let-it-fail

I worked at one place where we had to deal with a lot of off-hours on call issues. Our manager focused on quick resolution and never let it escalate past her. Most of the time she wanted us to simply reboot service or server. Never had time to find root cause.

It happened a lot affecting our personal lives significantly, waking up in the middle of night just to reboot/restart some service.

Over time, the whole team started ignoring on-call alerts, and worked real slow. Pretending that our internet was down or laptop is not booting up. Longer it took to resolve alerts, our automated systems would start sending alerts to higher up in the chain. Also it started to impact our SLA and other metrics.

Finally, they decided to allocate resources to fix and stabilize the system.

This is the art of navigating a bureaucracy: make the pain happen at the same point that the decision is made. It's not easy, and from the outside it's almost impossible, without being very dogged.

I once worked with new awful in-house software that (thankfully) depended on 3rd party software. We had the legacy in-house system parallel running, which was even more awful. The legacy's awful came from it being unable to do anything it didn't already do without extensive re-wiring, and a team who believed if they don't share their knowledge of it they had a job for life. The replacement's awful came from it being inefficiently programmed with large DB latency due to the team's belief at throwing more CPU behind queries instead of re-writing queries.

The teams sat next to each other but didn't speak.

The legacy team called in a bomb threat to a package left under the IT director's desk. It was taken very seriously. While their cause was already lost, that didn't help them personally.

The new system team eventually called Oracle, who came and re-wrote their queries.

When you say called in a bomb threat, do you mean they reported a package as suspicious?

I think it means they call the police (from a payphone if they are smart), and tell them there’s a bomb under the IT directors desk that’ll explode in x minutes/hours.

The call-in was an anonymous, apparently, call to the company's switchboard stating a bomb package had been left in the vicinity of the desk. We were evacuated to a separate part of the building for a few hours while it was dealt with.

Reply on news.ycombinator.com