Hacker News Reader

>suddenly you have tons of teams working on what seems to be fairly niche aspects of the company's product

At scale it tends to be worthwhile, even necessary, to engage fully with the complexity of all those "niche" aspects.

When you don't understand how something could possibly take so much effort, it's possible all the people working on it are idiots and you could do it better in a weekend. (When you spot situations like that, think of them as startup opportunities...)

It's also possible they're doing a good job hiding the complexity from you.

I think there is a lot of truth to both your comment and the parent comment.

Everyone underestimates the complexity of systems they aren't personally familiar with. Ask the average person how many parts are in a modern automobile and they'd probably guess too low by an order of magnitude.

But it is also true that large organizations get weird when money is easily available. You get a Cambrian explosion where without selection pressure to ensure people and teams do real, useful work, everything starts to seem like a good idea.

Determining which organizational complexity is essential and which is accidental is likely the quintessential hard problem of business.

This is a very nice description of what I've noticed moving to a large organisation. There is also a lot of busy work. As the organisation grows, communicating and negotiating between groups requires extra effort that often ends up being the role of a new management level, and groups have self-preserving incentives to stay relevant leading to lots of entropy (busy work, repeated work, hero projects that are just there for internal branding). As someone who has recently been put in this level of management I spend a lot of time having to negotiate out of new project initiatives invented more to attract exec attention or satisfy a rogue execs dream project. Makes me really look forward to moving back to working in a smaller company.

> Determining which organizational complexity is essential and which is accidental is likely the quintessential hard problem of business.

Three questions.

(1) "What would the impact on our business be if we built the best possible, state of the art feature here?"

(2) "What would the impact on our business be if we half-assed a solution in a quarter of the time?"

(3) "What would the impact on our business be if we utterly failed?"

(Usually, the different between 1 & 2 is negligible. Sometimes, between 1 & 3. Don't ask people to justify success. Ask them to justify why we should spend any more time than slightly-above-failure.)

Those questions assume an existing framework where "the feature", "state of the art", "half-assed", and "failed" are all well-defined. But my experience is that the framing itself is most of where the challenge lies.

You end up with team A who wants to add the feature, team B who thinks it's an anti-feature, team C who thinks the entire application where feature lives should be scrapped, PM D who says users can already accomplish what the feature does using existing features, UX design E who says the fact that users are still asking for the feature means all those other features must be poorly designed and we should fix those...

That second paragraph summarized the last 5 years of my professional career.

In my experience, I've always listened to the last UX design E guy. Every time we worked on what he said the users asked for, we did well..

The problems caused by "half-assed" solutions are likely somewhere down the line, in distant future and their true costs impossible to estimate.

Anything less than perfection will incure avoidable future costs, and perfection approaches infinite time to deliver.

Ergo, standing in the present, one should do ones best to estimate future costs and decide what percent of perfection seems optimal.

In reality, when asked to self-estimate, too many teams (and especially naive PMs) select perfection as the optimal approach.

Every tech company thinks they’ll just build their own CRM until they try.

See also: build systems and UI frameworks.

That’s a false dichotomy. It’s likely that that are both not idiots and what they are doing isn’t necessary.

When large companies have such excess engineering power to blow, they tend to say yes to all requirements and scope creep takes over everything. So no, the engineers building the abstract framework factory are not stupid, but they should have just built the report dashboard instead of the tool to generate tool generators.

As soon as an organisation gets above a size limit, organisational politics becomes the most important factor in any decision.

The decision to agree to a feature request starts to consider Who the feature request came from before any of the practicalities of the feature itself.

They're not idiots, and they are doing what is necessary, but the goals have moved from outward-facing market and customer concerns to inward-facing organisational-political concerns.

Every coder in a large organisation, sooner or later, gets given a task that makes no engineering sense but achieves some internal political goal.

It is fair to say that when you have R&D headcount in thousands it better be something really impressive.

Boeing 747 design team was 4500 engineers and they were done in 29 months.

Software is a little strange sometimes.

It’s a different type of systems engineering, that’s more akin to sorcery, than to actual physics.

The hardware is, of course, limited by physics, like the speed of electrons and heat dissipation. But once you get past that, your computer system is a collection of bytes that you reassemble to form whatever it is your imagination desires.

And while the 747 is a magnificent plane, it did have the benefit of over 50 years of aviation and aeronautical science behind it, all figured out.

Whereas software is still somewhat in its infancy right now. And every few decades, it seems to revolutionize itself, as newer and faster hardware become available.

A taxi company over here had a contractor build them their own hailing app. Cost them less than €600k, it's pretty slick.

fair, but also a lot of Uber's business model is leveraged on top of the basic platform.

What does that mean?

It means that the functionality of the app that immediately comes to mind is only one of the many ways in which they lose an extraordinary amount of money.

The functionality may be similar, but the scale is completely different and shouldn’t be ignored. Building a machine that cooks 1 pancake every 5 minutes is a fun project for precocious college students; designing a system capable of cooking 10,000 uniform pancakes per minute is a feat of incredible industrial engineering.

I'm talking about R&D specifically, not infra folks.

But let's say designing for scale is 100 times harder than a taxi app for just one national taxi service. Based on my 23 year career in trenches I seriously doubt that's the case, but let's be generous.

That would net us into ballpark of €60M expense. Non-negligible, but it translates into a hundred developers pulling €100k a year for 6 years, still at least an order of magnitude below Uber.

Very obviously it does not scale anywhere near linear. There has to be a severe case of diminishing returns in technical hiring.

So did early Uber. Famously, the contractors were Spanish-speaking, so FTE engineers #1-5 had Spanish-English dictionaries on their desks to understand the code.

I can't find you a source right now but back in '94 when I was learning IBM mainframe Assembler the history was mentioned and they claimed thousands of engineers, mathematicians and physicist worked for 3.5 years on the first mainframe (or was that the 360?).

The IBM of that era was literally inventing the modern computing systems. The complexity of the tasks at hand were essential, not incidental. I think the general trend of thought here is a critique of the latter day NIH syndrome and the consequent inflation of the technical teams.

>> You have man-years worth of work going nowhere as projects get scrapped mid-development.

That's better than seeing man-years go into features that get deployed. I witnessed one successful control system company spin up a team of new hotshot UI engineers, all right out of the best schools. After months of work the team "updated" the product. Withing hours the call center was hit with hundreds of "You changed the g-dam menus!! Put them back NOW!!".

The trick is to squeeze your long-term UI project in the same update as some routine security fixes. Then the clients are forced to learn the new system.

Or do A/B testing and incremental rollouts?

A/B tested products are terrible. Please don’t do that to an actual product people pay money for.

“On this page, 12% more users clicked save if the save button was green instead of blue”

Bad A/B testing is bad. Incremental rollouts are not bad.

That's more of a failure for the ux and product vision than it is a failure of project management. The trick isn't to just roll out new changes slowly, but to involve the users and figure out what needs to change and how.

>> roll out new changes slowly.

That really depends on your customers and the product. In this case, control systems, incremental changes are definitely not the way to go. Imagine if your car made an incremental change to the position of the brake pedal every time you turned it on. In such situations you don't babystep. You announce the change ahead of time, provide your customers with transition training, and make the change as scheduled.

In the case of "adaptive" menus in control systems, imagine if the elevator in your building rearranged its floor buttons so that the most requested floors were always at the top. Total chaos. People learn where their button is on day one. After that ANY change is going to go badly no matter what the UI engineers say. That UI should be carved in stone for the life of the building.

Were there any dedicated UX people involved? To me, a UI engineer is someone who works exclusively on the front end, and would implement the work of the UX designer. They wouldn't actually do the designing and research themselves.

This is one of those cases where that Ford quote about a faster horse is relevant. Many people will be happy to just have what they have now without thinking how it could be better.

> The trick is to squeeze your long-term UI project in the same update as some routine security fixes. Then the clients are forced to learn the new system.

Is this sarcasm? Seems like a really user-hostile attitude if not. Admittedly, my personal point-of-view is that the majority of UI updates are make-work for engineers and PMs with at best no value added, and at worst negative value created (as you experienced).

It's a funny joke because everyone from Microsoft, IBM, and Apple to Canonical and the GNOME Project has been guilty of this in the past.

Oh god you're right

> The trick is to squeeze your long-term UI project in the same update as some routine security fixes. Then the clients are forced to learn the new system.

Fuck people doing this. Especially extremely radical design changes such as Twitter's and Facebook's "redesigns" that they force down the throats of their users. Or Microsoft.

> The trick is to squeeze your long-term UI project in the same update as some routine security fixes. Then the clients are forced to learn the new system.

No, the trick is to iteratively change the UI in a way that's easy for the user to get used to.

For what it's worth, this isn't unique to unicorns. Almost company successful enough to get to the scale of tens of thousands of employees likely has huge amounts of waste within it. The situation you describe of niche projects being scrapped after several person-years of effort had been invested, incomprehensible management hierarchy, and no coherent strategy perfectly matches my first job out of college, a year at Sprint in 1999. Me and two other new hires spent a year doing next to nothing on a team of 20+ building a complicated Java service with a purpose I never understood, but which had been going on for about a year, with half the team made up of contractors. Pretty sure I never wrote any actual code, though I did have to fill out time sheets every week. I had two managers, neither of whom I ever talked to. After a year, I got a 15% raise and a promotion. Then soon after I found another job and quit, our project got cancelled, but everyone on the team just got moved to other teams doing similar "work".

"...you can tell from the inside when a company loses its way."

Those example pathologies you listed are SOP at all the mature orgs I've worked at.

Do you think the degree to which a company succumbs to one of these "pathologies" matters? Is it a matter of timing (i.e. becoming successful faster than they become moribund)?

What do you think differentiates a company that becomes "mature" vs one that flames out?

Mature companies, notably, make a profit. Not a future profit, not an expected profit, not profit-discounting-advertising, a right-now end-of-year no-accounting-tricks more revenue from sale of goods than expenditures from all sources. I'm going to take a page out of the IRS handbook, and say that they have made a profit for the last three out of five years, in fact. Let's take that a step further and say that they've existed for five years. Their primary method of financing operations is not private equity, but public equity or loans. Once a company looks like this, it's pretty well set.

Sound like what happened at Intel

Well said. That's exactly what happened to many unicorns, including Uber.