Hacker News Reader

Comment by godisdad | Hacker News Reader

The solution never tried: share the spoils of efficiency with your teams. Give them a percentage or even a few basis points of what they save back as charitable contributions to an organization of their choice.

Watch how fast the bill will go down

View on HN

>The solution never tried: share the spoils of efficiency with your teams.

Funny enough, for all the problems with studying business administration, they do teach this as part of the principal-agent problem. If the principal wants something done, and the agent has no incentive to do it, you need to give the agent some of the incentive so their interests are aligned with yours.

At the same time it's something you rarely see implemented by BA types, because they don't like to share.

You’re making a new problem by creating that incentive. What happens when engineers are financially incentivised to cut corners? Do you need to tax their incentive pay for service disruptions? How complicated would the system have to get before the ICs realise that resource allocation is actually a governance responsibility, and they’ve actually just been tasked with doing the CEOs job?

Incentives are one of those things everyone knows we need, but nobody can design safely. A bit like security, we often come across systems that can be gamed due to unforeseen issues with the design.

Google "Hanoi rats" for an incentive blowup. Then there's myriad other incentive issues related to taxes and subsidies, all stuff that had the good intention of promoting good things and reducing bad things.

Sometimes its a meta-game: if a team sees a "share the spoils" scheme being trialled with another team, maybe they shouldn't fix their issues until there's a scheme in place for their team? What about the shape of the payoff? If you fix two issues in the same period, are you better off or worse off than splitting them over a boundary?

Even in sports, there's issues: there's an incentive payment to break the world record. Say you can pole vault 10cm higher than the previous record in your practices. But why break it once when you can break it 10 times? I read about this some time ago, should be findable online.

Rather they than just "they don't like to share", the BA types don't do this because in one of their other classes they have been taught about the downsides of such incentives, like:

- Corrosive effects on team cohesion, when separate ICs compete to see who can find cost savings first.

- The "cobra effect", where engineers will deliberately introduce cost inefficiencies so that they can fix their own bugs later.

- (Related to the previous point) Prevention of cost overruns is even cheaper than fixing leaks after the fact, but is very hard to properly incentivize.

Any class on principal-agent problems will hopefully include a chapter warning against asymmetric incentives and how they will be ruthlessly exploited by the agents. Engineers are not some exception here, in fact because of their training they are often extra good at spotting loopholes in the rules.

There's also always second order effects or externalities which this incentivizes engineers to hide or play ignorant about. To ensure group cohesion other engineers also have an implicit incentive to not bring them to light. So they get their bonus and in three months the cost savings is undone as those come to light.

Reducing cloud costs should be a one year exercise. There should be a department level incentive, meaning if the whole department/org meets its goal they get the incentive. There should also be a team level incentive to encourage individual teams to find cost savings.

Commissions, metrics, and goals and heavily used by sales across many companies. I’m sure there is some abuse but it is still heavily used.

Yeah I mean imagine if you applied these sort of incentives to executives, they would do silly stuff like take out loans to buy up the company stock or cut corners and destroy customer loyalty for a few good quarters of metrics before they depart to another firm.

Does the cobra effect happen because people are greedy and cynical, or does it happen because the people already have contempt for those offering the incentive?

Why not both? Or even better, why not a more noble third reason, like "people like their children to eat well and see this as an opportunity to make it happen"? Humans are complex and can have more than one motivation for any given action.

As a SW Engineer I have to say that this would not really work without some healthy dose of abuse.

Mangers at one of my previous workplaces were very vocal with "everything is running with slacks" so whenever I gave them an estimate on how long a feature would take to implement or a bug to be fixed (bugfix estimates are BS to begin with but...) they took it and multiplied it by 0.75 at most. Then bitched about it when it was over the time they assigned. Guess what happened.

Exactly this would happen with a cost-cutting incentive as well - if managers told us "reduce cost by 30% to the end of the year" then the cost would creep up to begin with. Incompetent management making stupid rules and stupid incentives is what drives me nuts :) .

The devil really is in the detail: "you need to give the agent some of the incentive so their interests are aligned with yours". The vast majority of employers don't pay SW Engineers enough and don't incentivise them enough so they give a shit. This really has to be a company culture, not forced but lived especially by managers, then it would work with engineers as well.

At Yahoo in 2015 we had the CFO challenge: find cost savings and get a bonus up to $50k. I think it was $1k per $100k in annual savings.

One guy turned off staging at night and saved $600k/yr.

Doesn't Yahoo have engineers in non-US timezones?

Or testing an emergency fix if something breaks at 4am. Problem with these types of metric driven approaches is that it incentives explicitly not caring about second order effects or externalities. Good for the person getting the bonus but bad for the company as a whole.

We had a particularly big staging environment for the small team we were. In fact we had 11 of them. This engineer, years earlier, had rolled out "bento box" environments where we could launch a full as many identical staging environments as we needed.

I believe we did keep an official staging running, but the 11 bento environments were the ones that shut down at night.

This was the staging for our specific team and we were all US based and on the west coast.

I can guarantee you that they spent more than $50k/month on all staging environments.

So he just got $6000 once for saving $600k/yr? Let's say the company didn't overspent on that, especially if you factor in that after 5 years €3M has already been saved. I don't know the incentives for sales, but I can imagine that theirs look better than 0,2% (if you just limit the projected savings over 5 years).

There's a saying that I can't quite remember, so I'll just say it in my own words: reward people based on a metric, and that metric becomes their main concern. Worse still, they will learn to game the metric.

Building a new system that doesn't have a baseline cost? If the cost saving bonus is based on a reduction in existing cost then you don't get rewarded for a new implementation. The business don't have a baseline cost yet. So what do you do? Build it to run at a higher price, run it for a few months and then cut costs.

Also, is the reverse true? Deny bonus if costs rise. Because if it is, then the engineer has to prove that the increases are directly related to increased customer activity. But what if increases aren't relative to customer spend? Etc

Goodhart's law - When a measure becomes a target, it ceases to be a good measure

Do this and now you also have an incentive to first spend money frivolously so that "savings" can later be easily found.

You should not announce this and use the numbers half a year after the announcement.

You have the same bad incentive when finance comes and say "cut 10% this year and 8% next year". You can porobably already cut over 20% today - after the needed work - but you will do the bare minimum because otherwise you will have less budget and possibility to cut even further next year.

Why to charity? Give it to them directly. That being said, you'll create bad incentives: for a team to create badly optimized infra cost then to reduce it and cash in the difference.

TripActions does this savings sharing when booking travel.

Any other similar platforms out there?

Ah a bonus for a narrow metric. Foolproof!

Reply on news.ycombinator.com