Optimism, Hubris, and Strategic Misrepresentation | Hacker News Reader

Optimism, Hubris, and Strategic Misrepresentation

albertcory50.substack.com

50 pointsAlbertCory16 comments

I have seen many failed or very expensive projects. An unordered list of issues.

1. Sunk cost. Quit before it hurts. 2. Non-technical leaders. Talk to experts when it comes to tech. I don't care if you watched a Youtube-vid. 3. Tech people that think they know it all. Don't build waste that solves problems that the business does not have. 4. Person cult. Anybody can make simple mistakes. 5. Department cult. Quit pretending that the guys at your department are smarter. 6. No scope. Define what to do. 7. No order. Prioritize what is important. 8. Premature or upfront decisions. Why create time estimates first and then figure out what to do. 9. Cowboys. Crazy time estimates that someone high up in the hierarchy just made up without talking to anybody. 10. Everything tech. Not sure why, but some dev teams can't add a simple feature without rebuilding half the tech stack. Features are built on current stack. Tech is built with tech decisions but may be necessary to add a feature. 11. Overlapping. Multiple people doing the same work. 12. Bad architecture. Architecture that does not simplify anything.

The article seems to confuse optimism with thinking everything is good. I think many people do.

Thinking everything is good means not having a reason to improve anything. Just eat some lotus, dude, it's okay . . .

https://en.wikipedia.org/wiki/Lotus-eaters

Optimism, on the other hand, is the belief in improvement. To improve is to accept and to signal that we are not in the best of all possible worlds so that we can work toward improvement.

"Optimism Bias" is a pretty well-accepted term in the research, and it's defined in here, which I cited:

https://www.cell.com/current-biology/pdf/S0960-9822(11)01191...

It means having a more positive view of your prospects than is warranted. Also, considering the risks to be less than they actually are ("it won't happen to me!").

What you're talking about is complacency.

I wonder whether the best way to go about these massive software projects is actually completely different to hiring some business on a massive budget.

It might be an interesting idea to instead take your massive budget, divide it into 10 pieces, and hire 10 independent teams to implement it. Just pick out of whichever ones are actually finished on time.

Smart, but will it still work if one of the teams fail? And if you control for interdependencies, are the still independent teams?

That's what I mean, you literally hire 10 different teams and tell them "hey make me a health insurance matching website".

Incredibly wasteful, but probably less wasteful than what they already are doing :D

Waste is entirely a matter of perspective. Calling it waste implies there's a practical solution that's better, when it may be completely unreasonable to posit such a hypothetical.

This approach is essentially an evolutionary strategy, and I would think twice about accusing nature of waste.

IIRC LHC has two separate experiments doing largely equivalent physics, complete with separate teams, buildings and instruments, precisely as a cost-saving measure. (They tried axing one of them at some point, and the budget for the other very quickly increased twofold and then some more. Of course cross-checking is also a concern, now that the US can’t build accelerators anymore.)

On the other hand, I don’t see how you could pull this off in an adversarial environment incentive-wise. If everybody gets paid, a winning strategy during the contract is to not really try; if only the winner gets paid, a winning strategy when choosing the contract is to avoid everything that looks even slightly risky. Supposedly there’s a whole not-a-Nobel-prize-winning theory of auctions—I wonder if it has something to say here.

> If everybody gets paid, a winning strategy during the contract is to not really try; if only the winner gets paid,

You pay everyone to participate and you give a bonus to the winner. People who don't make a reasonable effort won't get selected to participate next time.

This reminds me of the argument for allocating research funds using prizes more so than grants, which I have found quite compelling.

What argument is this? Can you explain more?

Phew, I found some breadcrumbs on this, I was afraid it was lost in the ether of stuff I've read and thought "huh, that's interesting and compelling" and then forgot the source of.

So here's an article[0] Matt Yglesias wrote for Vox in 2014 about prizes to incentivize pharmaceutical research, rather than patents. I don't think this is where I first read about this (because I wasn't reading Vox in 2014), but I think either he or Ezra Klein talking about it on a podcast may be where I first came across it. Here's the summary of the idea from there:

> A large cash prize creates an incentive to innovate just as much as a patent does, but offers several important advantages.

It's very possible that this is really only relevant to scientific endeavors that pop out patents, rather than anything that is currently grant-funded. But I generally like this idea of big incentives to find an answer to something, rather than micro-managing the funding allocated to different proposed avenues of research toward an answer. I hate that all the PIs I know spend over half their time writing proposals, when they could just be researching.

Here are a couple deeper links to economists writing about it (from that article)[1][2].

[0]: https://www.vox.com/2014/6/9/5788630/4-radical-ideas-for-pro... [1]: https://marginalrevolution.com/marginalrevolution/2011/12/me... [2]: https://www.project-syndicate.org/commentary/prizes--not-pat...

> > I’d like to be remembered as an innovator. I think it was General MacArthur said, you’re remembered for the rules you break, and you know I've broken some rules to make this. I think I've broken them with logic and good engineering behind me. The carbon fiber and titanium: there's a rule you don't do that. Well I did.

> [Note: According to Wikipedia, MacArthur is remembered for his accomplishments and his failures, the Occupation of Japan, “I shall return,” and being fired by Truman, not the rules he broke.}

Speaking of strategic misrepresentation… MacArthur is, in the initial quote above, presenting as having said that you’re remembered for the rules you break, but the response says that MacArthur was not himself remembered for the rules he broke, as if that was some sort of rebuttal.

Are you suggesting he's remembered for breaking the rule of being remembered for the rules you break, thus proving the rule?

No.

Reply on news.ycombinator.com