Hacker News Reader

Comment by pavlov | Hacker News Reader

Why is Twitter such a Waterloo for all these obviously accomplished people?

It seems like they've been assuming Twitter is the way it is because it was staffed by technically incompetent leftists, and if only they could apply their own get-things-done attitude and "neutral" politics, then the problem would be trivially fixable.

Where does this fallacy come from? Is it because of the illusory simplicity of the tweet format? Something like: "We just need to come up with the right algorithm and do an embarrassingly parallel run over these tiny 280-character chunks of text. How hard can that be. In my own Very Serious Day Job, I deal with oompabytes of very complex data. This tweet processing stuff should be child's play in comparison."

View on HN

I think there's a certain type of person, particularly common in tech, who thinks this way about _everything_; "oh, that's way easier than what I do, how hard could it be". A kind of reverse impostor syndrome. See the cryptocurrency space; it's more or less been 15 years worth of crypto people accidentally repeating all the failures of conventional finance from the last couple of centuries, because, after all, how hard could it be?

You hit the nail on the head. There are definitely these kinds of people and they are definitely highly concentrated in tech.

it's called dunning-kruger, the epidemic syndrome of silicon valley

It's definitely similar, but I think it's _subtly_ different (though it's often found in the same people).

Dunning-Kruger is, approximately "I'm good at the thing I do" (by someone who is actually incompetent).

What I'm talking about is "That thing that other people are doing is really easy; I'd be good at it" (the thing is not easy, and they would not be good at it).

If the person in the latter case actually ends up doing the allegedly easy thing, they may realise that actually they are not good at it, in which case it's not Dunning-Kruger. This is pretty common, I think; person barges in, saying "this will be easy, because I've decided the thing I'm good at is more difficult than it", admits it's not easy, and either leaves or learns. Alternatively of course they may retreat into full Dunning-Kruger; see the Musk Twitter debacle, which is _both_, say.

100 years from know, it'll be known as "the musk-trump complex"

the "Musk-Trump complex" is more commonly known as "narcissism". I think what is being discussed here is somewhat different.

fair :-)

I think, and, well, here's a phrase I've never used before, that may be slightly unfair on Trump. Trump does actually take 'expert' advice; he's just astonishingly bad at choosing experts (witness the amazingly weird lawyers he surrounds himself with). But he does seem to at some level realise that he doesn't know everything.

I think it's because not many non-wierd folks want to work for him. If he were just 5% better at supporting those who work for him, he might increase his success rate by 50%.

having worked in his general vicinity briefly, I can attest he doesn't take any advice unless it's pushed on him. His PR people, at least, seem to be constantly manhandling/damage-controlling the man

> Dunning-Kruger is, approximately “I’m good at the thing I do” (by someone who is actually incompetent).

Nope, but I can overlook because DK is misunderstood this way by almost everyone, and the authors have caused & encouraged the misunderstanding.

Dunning and Kruger didn’t test anyone who’s actually incompetent at all! The use of that word in the paper is so hyperbolic and misleading it should have been rejected on those grounds alone. They tested only Cornell undergrads. They didn’t check whether people were good at what they do, they only checked how well people could estimate the skill of others around them. The participants had to rank themselves, and the whole mysterious question in the paper is why the ranking wasn’t perfect. (And is that a mystery, really?) It is hypothesized that DK measured nothing more than a statistical case of regression to the mean, which is well explained by having to guess how good others are: https://www.talyarkoni.org/blog/2010/07/07/what-the-dunning-...

Contrary to popular belief, DK did not demonstrate that people wildly overestimate their abilities. The primary data in the paper shows a positive correllation between self-rank and skill. There’s no reversal like most people seem to think. Furthermore, they only tested very simple skills at and least one of them was completely subjective (ability to get a joke.) Other papers have shown that no such effect occurs when it comes to complex subjects like engineering and law; people are generally quite good at knowing they didn’t major in a subject.

Meaning people in SV are subject to the same cognitive biases as everyone? Knowing how to say Dunning-Kruger doesn’t exempt one from it’s effects, right? The paper didn’t show less skilled people estimating their abilities to be higher than skilled people, it only showed a self eval / skill curve that has a slope less than 1.

https://xkcd.com/793/

Perfectly describes your non-engineer neighbor or best friend when he encounters an idea he’s never heard of before.

> Technical people suffer from what I call "Engineer's Disease". We think because we're an expert in one area, we're automatically an expert in other areas. Just recognizing that helps.

https://news.ycombinator.com/item?id=10812804

you could also call it 'the halo effect'.

I think the more interesting question is why this symptom mostly happens to "engineers".

I've seen enough engineers presume they can easily become experts in law; I haven't seen many lawyers presume they can easily become experts in engineering.

Why?

It's certainly not _just_ engineers; you see it in the hard sciences and medicine to an extent, as well. Someone recently posted a study purporting to show harm caused by masks to HN, say; while its authors didn't appear to include anyone with expertise in the relevant medical specialties, they did include a chemist and a veterinarian. And, if you're a fan of Matt Levine, you'll know that dentists stereotypically tend to think of themselves as being experts at high finance.

But it definitely does seem to be especially pronounced with engineers.

(NB. I am a software engineer, and not a sociologist, so, argh, this is potentially getting a bit meta.)

Re: dentists. I have a few friends who are MD's who say they went into it to "help people," and that if they "just wanted to make money," they would have been "one of those tech CEO's." But when you look at how they run their offices and finances, you see that there is very little crossover between their medical skill into business. They just assume that they would be a successful CEO.

I've noticed it as a pretty widespread phenomenon for anyone who has the subjective experience of being competent and thinking that's enough to translate to other fields.

Super common in hot takes on politics, medical contrarianism, etc.

Though it's probably true that certain fields are more predisposed to it than others.

IMO, there's a certain level of arrogance intrinsic to engineering: To build something new, you need a belief, first and foremost, that you can build it at all, and almost as importantly better. Weeding out all the people who don't have, at least to some degree, that belief, and you end up with a disproportionate fraction of people who think that way about everything.

I can confirm I think this way about almost everything. Because I can’t see why I can’t be a lawyer, or a farmer, or a dentist given that I spend enough time to learn it.

“Reverse imposter syndrome” is a great coinage; I’m going to start using this!

Actually, on second thoughts, I should possibly have called it intruder syndrome :) (Reverse imposter syndrome could just describe Dunning-Kruger depending on which axis you're reversing...)

It's because of scale.

Very complicated algorithms and mathematical proofs can still be understood by a single person, and be explored by a small number of people who all know each other. Brain surgery is done by a small team of people. These are typical "smart people" occupations.

Something as simple as Twitter still needs machinery that spans across technical skills, needs 24 hour monitoring, and needs lawyer and accountant support, so nobody can actually to it.

People think they can do it, because it's easy to spin up a demo that sends messages to a few thousand people and then shut it down again. They don't think about how to scan for CSAM, or how to respond to foreign government censorship requests.

I'm a senior developer, and I have to admit I'm one of those guys ;).

WhatsApp was 55 people big when they got acquired, and to me that sounds about right.

Twitter employed 7,500 people. 7,500!!!! So please tell me where the complexity lies? Surely not in the front-end code I can tell you that.

Let's compare it to something WAY-WAY-WAY more complex, like a game with multiplayer, awesome mod tools, etc.: ROBLOX: 2,200 employees. Do I need to mention they wrote their own physics simulation engine and keeping realtime multiplayer going?

So please, explain this to me: how is Twitter more than 3 times more complex than Roblox???

Maybe I'm wrong, that's very possible, I've been wrong in the past. But just explain this 1 thing then: Twitter needs more than 3 times the manpower than Roblox?

Well, when Elon Musk took over and gutted half the staff, I distinctly remember HN full out of outrage and predicting (in hindsight, "impotently wishing" would be more accurate) doom and how Twitter will go down any day now.

Then nothing happened. At least, nothing that I personally observed as a casual Twitter reader. The goalposts were moved to "it will go down with the New Year's Eve spike", and once again nothing happened. Then the narrative became "the cracks will only be noticeable in a few months", and here we are and yet again, nothing.

So Musk and Geohot came out as the saner voices of that whole debacle. Of course Geohot said exaggerated things like "you only need 40 engineers to run Twitter", but if it turns out it takes 300 engineers, then I would consider this as Geohot being proven mostly right.

Did you see the news about DeSantis yesterday? Musk convinced him to announce his presidential candidacy on Twitter, and the live stream just didn’t work.

I don’t think that qualifies as “nothing happened” when features used in high-profile events fail, with the CEO and a potential future president left on the line. Any other platform wouldn’t have struggled with a stream of this size.

I guess you might say that’s just one thing, and other than the CEO’s live streams not working, everything is fine. But there are numerous other examples of accumulating paper cuts and failures at Twitter. I think this is close to what most of those doomsayers expected would happen.

> the live stream just didn’t work

Are you sure ? Others say 6.5 M listened to the livestream that was delayed 20 mins

so was it no longer live, or did they encounter 20 minutes of technical issues that delayed the start? cuz either way it seems pretty obvious that at least for some amount of time it didn't work

They had to switch to David Sack's account to do the livestream and I think there were about 700k listeners that were on at the time of announcment. The issues weren't just infrastructure related, Musk had challenges with his mute button and it was creating feedback because he and Sacks were next to each other on their phones.

But yeah, it could have gone better for various reasons.

Google also recently had a total failure in a public event. It's not necessarily saying much about Twitter.

https://mashable.com/article/google-ai-maps-search-event-bin...

> the AI falsely said the James Webb Space Telescope took the first ever picture of an exoplanet

> During the announcement about a new Lens feature, the demo phone was misplaced and the presenter wasn't able to show the demo

> Google seemed to say, "let's pretend this never happened," and immediately made the livestream recording private after the event

There have been outages, just not as catastrophic as predicted.

I think that depends on who did the predicting :)

There was a lot of "ooh, it will catastrophically fail within weeks", which was fundamentally an assumption that the previous team was entirely incompetent. (Any halfway decent team tries their hardest to build resilient systems, not things that need hand-holding all the time.)

The current trajectory is exactly on the expected failure path predicted by anybody who does actually work on large systems - a steady increase of smaller failures, punctuated by the occasional large failure. (Cf. DeSantis announcement)

In essence, a reduction in staff will result in worse SLO results. It will result in less coverage of edge cases (technical and UX). Smaller teams are more constrained to travel on "the happy path". And the fact that marginal utility of additional engineers decreases means you can usually reduce teams a lot before impacting that path.

In complex systems, reductions also mean you're more vulnerable to a black swan event being irrecoverable, but that still requires a black swan first.

it really is a testament to how well engineered Twitter is/was. I well remember Musk gloating about how the architecture was stupid and he'd fix it. Twitter would be long gone if his remarks were anywhere near the reality

Guess you don't remember the fail whale? Twitter was held together with gum and bailing wire for a long time. Yes it got better, but I'm certainly not going to use it as the example for good engineering.

> they've been assuming Twitter is the way it is because it was staffed by technically incompetent leftists

I don't think anyone argued Twitter was run by technically incompetent people. Where was this, if so? By leftists, yes, and by far too many people, yes. Both were argued repeatedly. But those things are now proven objectively true. The Twitter files showed just how systematic their enforcement of left wing orthodoxy was, and Musk fired most of the staff yet the site kept trucking and even launching new changes which is more or less the definition of having been over-staffed.

The wep app itself is easy.... it's everything around the tech that is hard (scaling, regulatory, moderation)

Reply on news.ycombinator.com