Growing is hard and unicorns are expected to grow really, really fast. In the end many companies with a completely viable product end up going under just because investors thought that a million dollar company should be a billion dollar company.
At scale it tends to be worthwhile, even necessary, to engage fully with the complexity of all those "niche" aspects.
When you don't understand how something could possibly take so much effort, it's possible all the people working on it are idiots and you could do it better in a weekend. (When you spot situations like that, think of them as startup opportunities...)
It's also possible they're doing a good job hiding the complexity from you.
Everyone underestimates the complexity of systems they aren't personally familiar with. Ask the average person how many parts are in a modern automobile and they'd probably guess too low by an order of magnitude.
But it is also true that large organizations get weird when money is easily available. You get a Cambrian explosion where without selection pressure to ensure people and teams do real, useful work, everything starts to seem like a good idea.
Determining which organizational complexity is essential and which is accidental is likely the quintessential hard problem of business.
Three questions.
(1) "What would the impact on our business be if we built the best possible, state of the art feature here?"
(2) "What would the impact on our business be if we half-assed a solution in a quarter of the time?"
(3) "What would the impact on our business be if we utterly failed?"
(Usually, the different between 1 & 2 is negligible. Sometimes, between 1 & 3. Don't ask people to justify success. Ask them to justify why we should spend any more time than slightly-above-failure.)
You end up with team A who wants to add the feature, team B who thinks it's an anti-feature, team C who thinks the entire application where feature lives should be scrapped, PM D who says users can already accomplish what the feature does using existing features, UX design E who says the fact that users are still asking for the feature means all those other features must be poorly designed and we should fix those...
Ergo, standing in the present, one should do ones best to estimate future costs and decide what percent of perfection seems optimal.
In reality, when asked to self-estimate, too many teams (and especially naive PMs) select perfection as the optimal approach.
When large companies have such excess engineering power to blow, they tend to say yes to all requirements and scope creep takes over everything. So no, the engineers building the abstract framework factory are not stupid, but they should have just built the report dashboard instead of the tool to generate tool generators.
The decision to agree to a feature request starts to consider Who the feature request came from before any of the practicalities of the feature itself.
They're not idiots, and they are doing what is necessary, but the goals have moved from outward-facing market and customer concerns to inward-facing organisational-political concerns.
Every coder in a large organisation, sooner or later, gets given a task that makes no engineering sense but achieves some internal political goal.
Boeing 747 design team was 4500 engineers and they were done in 29 months.
It’s a different type of systems engineering, that’s more akin to sorcery, than to actual physics.
The hardware is, of course, limited by physics, like the speed of electrons and heat dissipation. But once you get past that, your computer system is a collection of bytes that you reassemble to form whatever it is your imagination desires.
And while the 747 is a magnificent plane, it did have the benefit of over 50 years of aviation and aeronautical science behind it, all figured out.
Whereas software is still somewhat in its infancy right now. And every few decades, it seems to revolutionize itself, as newer and faster hardware become available.
But let's say designing for scale is 100 times harder than a taxi app for just one national taxi service. Based on my 23 year career in trenches I seriously doubt that's the case, but let's be generous.
That would net us into ballpark of €60M expense. Non-negligible, but it translates into a hundred developers pulling €100k a year for 6 years, still at least an order of magnitude below Uber.
Very obviously it does not scale anywhere near linear. There has to be a severe case of diminishing returns in technical hiring.
That's better than seeing man-years go into features that get deployed. I witnessed one successful control system company spin up a team of new hotshot UI engineers, all right out of the best schools. After months of work the team "updated" the product. Withing hours the call center was hit with hundreds of "You changed the g-dam menus!! Put them back NOW!!".
The trick is to squeeze your long-term UI project in the same update as some routine security fixes. Then the clients are forced to learn the new system.
Is this sarcasm? Seems like a really user-hostile attitude if not. Admittedly, my personal point-of-view is that the majority of UI updates are make-work for engineers and PMs with at best no value added, and at worst negative value created (as you experienced).
That really depends on your customers and the product. In this case, control systems, incremental changes are definitely not the way to go. Imagine if your car made an incremental change to the position of the brake pedal every time you turned it on. In such situations you don't babystep. You announce the change ahead of time, provide your customers with transition training, and make the change as scheduled.
In the case of "adaptive" menus in control systems, imagine if the elevator in your building rearranged its floor buttons so that the most requested floors were always at the top. Total chaos. People learn where their button is on day one. After that ANY change is going to go badly no matter what the UI engineers say. That UI should be carved in stone for the life of the building.
Fuck people doing this. Especially extremely radical design changes such as Twitter's and Facebook's "redesigns" that they force down the throats of their users. Or Microsoft.
“On this page, 12% more users clicked save if the save button was green instead of blue”
No, the trick is to iteratively change the UI in a way that's easy for the user to get used to.
Those example pathologies you listed are SOP at all the mature orgs I've worked at.
What do you think differentiates a company that becomes "mature" vs one that flames out?
You know the line "don't mistake motion for progress"? There's a whole lot of motion up in those SoMa offices... but very little progress. Some of these teams are very good at doing a lot of jiggling around while never accomplishing anything. They manage to snow their manager, which snows the next-level up, and if nobody calls BS, it just goes on like this.
R&D? When the response to building something new is an _immune system flare-up_ from the people who benefit from things remaining exactly as they are, there can be no R&D.
Just like the server situation, the employee situation is bloated beyond belief. You have people making messes and others cleaning it up, instead of just not making the mess in the first place (and then needing neither of those people).
If a million monkeys at a million typewriters would eventually produce the works of Shakespeare, some of these companies would likewise boil down to "three monkeys, ten minutes" (not my line but I love it).
As the userbase scales, infrastructure scales, and what was once a simple "ALTER TABLE" is now a system with a team managing it because a single Postgres instance didn't scale.
As the engineering headcount scales, code changes, builds, even understanding the system become slower. You overcome that with better engineering, but mostly more engineers.
As the product matures, there's still a drive for growth, but the low-hanging product fruit has been picked, so you get lots of product teams trying to either optimize their little part because, at scale, it makes a big difference, or product teams working on new, crazy ideas--the startup within a unicorn-startup--that will most likely fail.
Uber is on another different plane altogether... they have more than double the number of engineers and from talking w/ colleagues that worked for years at Uber it's my impression that even on infra there's a lot of bloat. Plenty of high level ICs that couldn't produce quality IC work to save their lives and plenty of pet projects.
These are complete guesses for the pre-covid era, but it gives an idea of scale difference. The average Airbnb user probably books 2-3 times per year. The average Uber user probably books 2-3 rides per week. At 50x the volume, the infrastructure is different, and probably more complicated to handle the volume.
There are programmers and senior programmers, data designers and senior etc. There are also team leads and program managers. There's basically two tracks, you either create stuff or you manage the creation of it.
They have different titles now, and there's a lot of Agile fluff (how is a scrum master not just a secretarial service and the function of a team leader not to keep their meetings focused?)
Software has way gone out of the stage where one person keep the recipe for the secret sauce, and could change the game.
No one is irreplacable.
This seems so strange to me. As a lead, most of my work is management, organization, and planning. To scratch the development itch I usually have 1 pet project going on that I either use when I'm taking a break from actual work or work on in my spare time. But even that pet project has pretty clear long or short term value and the only reason why it's not being worked on by my team is because it hasn't been prioritized yet. I can't imagine working on a pet project that has no clear value to the company.
At the end of the day, it's just a reminder that careful screening is important. Could be a cultural issue that impacts certain unicorns, but it's also on the hiring company to thoroughly fact-check and test a candidate (and no, leetcode and simple arch questions aren't enough).
That said - to the question about unicorns: You're always expendable. Same as any job - same politics - same nonsense. No matter how important they make you think you are to a products success - they'll fire you regardless. It's amazing how fake management will be about urgency. "This is the most critical business function! We must have people working on it!11!!1!1!" Fires key employee because they didn't like them - "critical business function" gets shoved into the backlog to never be seen again. People are fake and think they need to show urgency to get the most value out of their employees.
I call this "artificial panic". That term came to mind when I joined a FMCG around Y2K, and on my first weeks I noticed people were running around stressed, like headless chicken. When I asked "what's on fire" and the answer was "nothing".
https://dilbert.com/strip/2012-07-15
The "Minimalists" say on their podcast that "most emergencies, aren't". When I see panic setting camp in a company's mentality, I make myself scarce for a while until things calm down. If I see that the artificial state of panic is a perpetual feeding machine, I try to dance around it. I've read my share of Dilbert to know to avoid these toxic environments.
Sometimes you need to be a Wally to survive.
Ideally in a company there would be one single order backlog, showing the relative value of each project to everyone. However getting such an ordering would involve huge co-ordination across vastly differing internal companies / cultures and would drag up every buried political hand grenade of the company's lifetime.
So instead you do what someone shouts loudly about. If the shouter is right more than 50/50 they are doing pretty well.
Never worked for a unicorn, but everything from big corp to startup to government. "Urgent" provokes no reaction from me anymore because of this.
,"Critically Important", etc. is the most recent whatever caught the attention of a typical S/E/VP with the attention span of a ferret long enough for him/her to be able to utter it in the words. If you're able to highly visibly jump and demonstrate activity during exact that moment - a great talent leading to advancements/promotions - there is no point to react as the next new "critical" directive/change of course/etc. is coming very soon. Of course if your manager is one of those aspiring "talented jumpers", he/she will try to make you jump with him/her - it is a real PITA to have such a manager who instead of filtering and protecting the team from would amplify all that stuff flowing down.
On the other hand, artificial deadlines that are invented by your company's management are usually not urgent.
The whole point of "Agile" was supposed to break down these divisions. Management is "generally charged" with getting the greatest productivity they can out of their resources. How that productivity is measured may be by "the priorities as set by Product", but that doesn't take into account ROI, sunk costs, COGS, Cap/Opex etc etc.
- Building a high-quality product is a longterm investment and most technology companies know that. As a result, in my experience companies who expect growth would always prefer to keep engineers and PMs as investments.
- Companies generally find it hard to hire Engineers and PMs that meet their standards, especially in competitive markets like the SFBA, NYC, Boulder/Denver, Seattle, Austin etc. (You can argue whether that's caused by overly stringent hiring standards - that's a much more complex question). As a result, companies are somewhat more likely to "hoard" engineers if they expect that they'll need them to grow.
- Because engineers in these markets are expensive, it makes economic sense to spend to improve their productivity. If I have 100 engineers on my team and I can make them all just 1% more productive by hiring an additional engineer who focuses solely on internal tools or open source libraries that improve developer QoL, that's arguably money well spent.
- Many products at scale are more complex than one might expect from the outside, which demands a lot of ongoing product/engineering effort to maintain. If you're working on something that has a credible path to revenue or clearly makes/saves money now, your job is probably fairly safe.
This all assumes that 1) the company wants to keep you, and 2) the company is growing, even modestly. Once the financials go downhill companies will cut directly to the bone in order to survive, and at that point nobody in any industry is safe.
(edited for formatting, thanks @mkl)
"""
- Building a high-quality product is a longterm investment and most technology companies know that. As a result, in my experience companies who expect growth would always prefer to keep engineers and PMs as investments.
- Companies generally find it hard to hire Engineers and PMs that meet their standards, especially in competitive markets like the SFBA, NYC, Boulder/Denver, Seattle, Austin etc. (You can argue whether that's caused by overly stringent hiring standards - that's a much more complex question). As a result, companies are somewhat more likely to "hoard" engineers if they expect that they'll need them to grow.
- Because engineers in these markets are expensive, it makes economic sense to spend to improve their productivity. If I have 100 engineers on my team and I can make them all just 1% more productive by hiring an additional engineer who focuses solely on internal tools or open source libraries that improve developer QoL, that's arguably money well spent.
- Many products at scale are more complex than one might expect from the outside, which demands a lot of ongoing product/engineering effort to maintain. If you're working on something that has a credible path to revenue or clearly makes/saves money now, your job is probably fairly safe.
"""
Please don't use code formatting for text. If you want bullet points, put a blank line between them to make separate paragraphs: https://news.ycombinator.com/formatdoc
Following this type of logic, joining Uber in 2015 or later is a bad idea. Joining airbnb in 2015 is a bad idea. Joining Google now is likely a bad idea. Joining new orgs in AWS is probably a good idea.
A few heuristics that I find useful: 1. Revenue per employee 2. Moving average of number of substantial launches in the past X months 3. Actionable technical blogs that address real challenges directly related to specific business needs. So, no, Uber's why they switched from MySQL to Postgres and then later another article by the same person on why they switched from Postgres to MySQL do not count.
It’s super important to note that business impact is not even close to uniformly distributed throughout a company, even though we sometimes like to pretend that it is.
This is all to make a subtle refinement of your point, that it’s worthwhile to join teams that are high value per engineer, even if the company is more bloated. (Ad serving infra at Google, ec2 at AWS, payments processing at Uber).
However, because those teams are so essential, they tend to have low turnover, higher bars for entry, and a higher than usual rate of internal transfers (as high performers from less essential teams are shifted into more critical roles). So you’re less likely to be placed into those teams from the outside, unless you have a history of working for those kinds of teams.
I think it's the same with big companies. You can be a big part of your individual team, and it doesn't particularly feel much different than working at a startup day-to-day. And as a bonus you get to be a part of the large-scale wins and loses. Much like how it's exciting when the Warriors win, it's exciting when something big comes out of your company. Even if you did nothing but cheer.
Everyone is expendable. Travis, the founder and CEO of Uber, was expendable. So are people at large companies or small startups. But within your team, you get to do good work, and it doesn't feel that way.
I didn’t like the way he conducted himself or ran his company, but the jury is still out on that. TFA is about how that ball is still currently in play.
I work at company with less than 100 people, I know where many of the bodies are buried, etc. But if I get hit by a meteor tomorrow, they would send flowers to my wife and open up a req and keep moving.
You don't have a group life insurance policy through your company? Sudden unexpected death is exactly what that supposed to cover. If not, that's unfortunate, especially since it's fairly common among white collar jobs at least.
If you do, it should cover at least your funeral costs, which should be pretty minimal if your cause of death is meteor strike. It would likely cover a percentage of the income you would have earned for your remaining years of work.
But opening a req seems totally reasonable when you are demonstrably not coming back.
But the point is everyone is expendable. I worked for a company around 2009. They laid a lot of people off that had more seniority than I did. I wondered why they kept me around.
Three months later I found out. The founder had written a bespoke development tool chain in C++ using MVC and assembly. The board push him out and the CTO told me that I was now responsible for maintaining the tool chain because I was the only one who knew C++ and assembly.
In the extremum, everyone is expendable, but not everyone is equally expendable, as your own anecdote bears out.
When you expend enough people, especially increasingly less expendable people who carry your basic institutional knowledge and skills, you do so by inflicting damage on your company or organization.
The economic winds may dictate that, as they did frequently in the 2008 economic crisis, but it's not like there is no institutional cost to expending people. This is the basically why countries like Germany and the UK are paying employers to keep idled people on payroll.
1) Are you suggesting that developers should go join a smaller company so they are less expendible?
2) Are you suggesting that developers, now that they are laid off, go work on R&D projects, tooling and open source startups?
I am so confused. What is the take away?
Orthogonally, typically large companies have a maslow hierarchy of sorts: for every infrastructural endeavor, there may be a number of others that are more "nice-to-have" niche projects that aren't really critical to anyone else's ability to deliver results.
The most vulnerable employees are those on niche projects, despite these projects being internally focused (as opposed to open sourced). Infra folks whose work may be open source are typically less vulnerable because they do in fact work on critical, well, infrastructure.
I've always worked in manufacturing so people aren't that easily expendible. Takes a long time to train someone and without them, one of the line shuts down or gets less productive / slow.
You're wrong, and not just about the Pareto principle.
Effort spent on building specialized libraries, tooling, secondary infrastructure as well as meetings, coordinating with stakeholders, dealing with outside research firms, etc. explodes while effort spent building those things that are clearly on chain delivering value to the customer grows rather more slowly.
It's really hard to decide if this is a good or bad thing. Certainly huge amounts of it can be seen as non-value-adding wankery, but it may all be in some way intrinsically necessary in order to scale up.
Of course any large company with staff in the thousands or tens of thousands will have bloat. But I’d guess a lot of the engineering headcount that is not in “core stuff” are adding value. There is a lot of value to be added by making relatively minor tweaks when your revenue and costs are in the billions. Reducing a few percent server load or getting a few basis points of additional revenue / members etc can be worth multiple teams of engineers.
The company will survive if those teams are gone, but it’ll probably hurt long term growth trajectories to some extent.
Through this pandemic experience, companies that were not previously forced to run more efficiently are now having to learn. Once they have figured this out, and IMO that does not take long, then there are few reasons to again carry the same amount of overhead, including employee compensation and benefits, office space, etc.
These BS jobs are not responsible for increasing revenues or decreasing costs. Eliminating them to reduce overhead is a no-brainer.
Having read David Graeber's original article and his follow up book, I believe he went to some lengths to explain why those jobs are so difficult to remove.
Jordon Peterson has also touched on this subject in one of his less controversial lectures. He uses Price's law to explain the proliferation of these jobs, the subsequent demise of the organisation when the handful of useful workers leave, and also that the remaining employees are unable to step up when this happens.
I prefer Peterson's interpretation, mainly because Graeber is quite fundamentalist in his view that a job is either bullshit or it isn't. Peterson's view is that large numbers of employees do productive work but that it is minimal in comparison to the few very productive workers.
I also think more people than Graeber have been aware of the existence of BS jobs for a long time.
What is nice about the Wikipedia page is that it acknowledges that BS jobs "is a thing".
Perhaps it facilitates more open discussion of the phenomenon.
SARS-CoV-2/Covid-19 may be facilitating the discussion even more as it has forced people to consider what jobs are "essential" versus what logically can be referred to as "non-essential".
I have seen one commenter on HN raise the issue of why pay does appear to correlate with whether a job is essential or not. Perhaps this is something more people will start to think about.
I've never been at a unicorn, but I have been at many startups. The story is the same everywhere. Almost no one is irreplaceable. When times get tight, cutting overhead means letting people go.