Since I first used Airbnb 7 years ago, the only real change I can think of is when they added experiences (which is awesome don’t get me wrong), surely at some point you begin to wonder if we are utilising this talent effectively.
Since I first used Airbnb 7 years ago, the only real change I can think of is when they added experiences (which is awesome don’t get me wrong), surely at some point you begin to wonder if we are utilising this talent effectively.
Engineers with 5 YOE are making $500k+ a year at these companies because they are generating much more than that in revenues and profits for the companies. Airbnb is worth $110B.
It's not all blinking cursors, there are thousands of little experiments going on all the time. Increasing the number of bookings by 2% here, 3% there. Increasing the average price of bookings. Random extra features here and there to close that one whale of a client. Expanding into corporate accounts. Improving performance by 5% to take a big chunk out of the tens of millions of dollars infrastructure bill.
From a 10,000 foot view looking at the product, it always looks like it's 90% done in the first couple of years and then things just coast along lazily. But from a profit point of view, that last 10% can generate hugely outsized returns. It's the power of exponential growth, all those extra late nights and tiny improvements that move the needle just a little bit all compound over time.
Another reason to have in-house expertise in various areas is that they easily pay for themselves, which is a special case of the generic argument that large companies should be larger than most people expect because tiny percentage gains are worth a large amount in absolute dollars. If, in the lifetime of the specialist team like the kernel team, a single person found something that persistently reduced TCO by 0.5%, that would pay for the team in perpetuity, and Twitter’s kernel team has found many such changes.
If removing a blinking cursor has a 1% chance to increase the booking rate by even 0.1%, , then it's worth it for AirBNB to pay an engineer to implement the change.
That is probably true. But we shouldn’t forget that these people can do these things often because they are supported by a large number of people who do the mundane tasks that need to be done.
For example, lets say you give a project to a competent engineer, he writes a clean and maintainable solution quickly alone. But if you schedule it to him together with another engineer of similar status but much less competent, then the other engineer will take a part of the project and basically block it since the competent engineer is unlikely to to take that fight, and instead just lets the project stall.
You don't get promoted for pointing out incompetence, there is a reason managers hires consultants to do that for them. This made me wonder, are there software consultants who act like management consultants and mostly go in and fire a lot of people? I don't think management consultants would do a good job of firing the right software people, they would need to be engineers.
It’s really hard to be surgical about this because who wants to be the good engineer that has to pick up the barely functioning pieces of code left behind. Who wants to reward a solid engineer with a big refactor job on an already late/failing project? The optics aren’t great. I’m not saying it never works, but as a general rule, deferring the project is often a better option.
The keyword "impactful engineering" needs some clarification though.
It does not mean there's a 100x guy walking around the office while everyone is slacking off.
A specific proof of concept hacked together by a guy in a week might eventually become the company's flagship product. That's impact. However, the thing needs to be rewritten from scratch to become production ready or even deployable, and that takes far more work that does not fit the definition of "impactful".
I personally know a principal engineer of a FANG which single-handedly wrote the proof of concepts of more than a few projects that thousands of users use every single day. From his own words following one of his recent presentations, "this needs to be rewritten from scratch as this would get me rejected from our job interviews".
A surgical ten lines of code across 5 services can absolutely create billions of dollars out of thin air. The combination of technical, political and domain expertise required for such changes is relatively rare.
(I mean political in the purest, non-controversial sense, i.e. the communication skills to answer objections and acquire group consensus on the required change.)
Meta-meta-question: assuming that it does, how does the line get described (let alone drawn) dividing "this work contributes to identifying the 1%" from "this is a waste of time"?
(Insert something about gradient descent and local vs global maximums here)
I'll call this a genuine question, I could definitely use some refinement of my own optimization of this problem space (and not wind up in micro-optimized dead ends etc).
If the payoff in the 1% case is high enough, then you don't have to know beforehand! Just do all 100 changes and the one winner pays for all the rest.
It's all a numbers game. The median member of your kernel team, the median fire extinguisher, the median link to a sales web page, the median joist in a floor, all will do nothing of note. But if you distribute the resources properly they will hopefully be where they are needed to generate a positive ROI that pays for the overall system. If you zoom in too much or too little it all looks silly.
You need a bench. Some of the brilliant folks who are great at pushing through problems are awful and maintenance and sustainment. I worked for a bit in a SWAT engineering team tasked with addressing crisis problems or emergency response. If you don’t have people you’re developing on the bench, you won’t be able to respond to those types of things and will get bogged down with tech debt.
I was a faux “10x” person because I had license to break the rules to get shit done - because the value of what we were doing was higher than the cost of cleaning up the mess. (Not because of any brilliance on my or the teams part) We did two years worth of work in a month, but it’s still being refactored 3 years later.
I've always felt that there is a dilemma in this space. Any number of small UI features can claim a small increase that are often just noise and curated data (assuming no malicious baking), the cumulative increase isn't even close what the claims stacks up to.
The only fundamental way for a business to make more money is to invent actual, new feature and business models. Shifting a box here and there, enlarging an icon just creates a generation of glorified engineers and PMs who are placing their attention entirely on the wrong thing. Yet the success reporting often rewards this behavior and eventually drown the business in fruitless 'noise' endeavors that favors short term gains over actual innovation.
So a single data point? I didn't see it claimed that there is not a single small thing that can have a big effect. It's about large numbers. I think individual examples obfuscate more than they help us understand, when it is given in lieu of wider view instead of just as a support for one.
In this context, I would also like to see considerations of a larger picture. Sure, it helps any single company make more money. What about beyond that? Because money is a proximate goal, even if it's the ultimate one for a business, from a human society point of view that introduced this tool for a specific reason and not for its own sake.
For example, what does it mean to have some ultra-optimized large companies dominating their space? Would society really lose something without some micro-optimizations, even if they raise profit? The people who end up not making a purchase on a website because they are not as much caught in the optimized patterns, is it actually bad for them, or maybe they are actually better off and the optimizations tricked them into making an actually unfavorable (even if slightly) purchase? I think there is more to consider than just the view from the respective company or even the department. Are the means used to win actually good from a wider point of view that is not centered on that winner alone?
If someone does not make a purchase because something is slightly off, are they actually worse off, neutral or better in the end? It can't have been all that important to them to make that transaction, no?
I’m not sure this is fair for services like Airbnb where what they sell is conveniences.
Their entire product existed before Airbnb. It even existed before the internet. What Airbnb did so successful was that they branded themselves with a good product that disrupted the market by being easier to use.
The thing about disrupting markets by selling easy of use, is that your competition is already well versed in the market you disrupted. Some of them may do the dinosaur but others will take your ideas and try to make them better. If you don’t continuously improve the ease or use of your product, then people are going to download eurobooking or whatever other options there are now instead.
You know this, because you’ve likely done it yourself a million times with various applications that do the same thing, but some find a way to be easier to use.
Obviously these companies are going to want to find (or invent) new products to get included. But that is typically more of a non-software quest that eventually leads to some of the engineers being tasked with building what the marketing/sales/business process people find.
On your second point, though, if all you focus on is building new features, you are losing out on tons of unrealised gains you could've had by making an old feature work better. At the scale a bigco works in, small percentages add up to quite a bit.
The answer to this is better measurement and statistical analysis, not abandoning the effort entirely. It does take more than "haha, trend line go up" to prove a change had a positive impact, but it can be done.
From a company’s perspective, the PMs and devs good at optimizing existing features already exist. Why not hire them to work alongside and behind the devs and PMs good at developing new capabilities?
AirBnb lost billions of dollars last year, and it has rarely made positive quarterly earnings. Based on your example, the engineers are highly overpaid as there efforts rarely generate a profit. They are being paid by funds from speculators (stockholders and venture capital) that are betting on huge profits in the future.
In reality, these engineers are being paid due to labor market forces and extremely focused recruiting policies (eg only hiring those with degrees from top engineering schools or poaching workers from other companies that have similar policies).
The simple truth that most software engineers don't want to hear is - people are quite conservative in this. If it works once, most folks would be extremely happy to keep using it for next 20 years in exactly same way, as it is. Bug-less has higher priority than shiny (which is highly subjective) and constantly changing.
Signups/purchase flows are meant to trap users and keep them from escaping--not create a smooth, easy to use experience
Unfortunately this tech oligopoly landscape doesn't leave users many ways to express this preference, but I often wonder if the culture will change at some point in the future as the industry matures.
An aging western population might have an influence here as well, in the long term. Older folks have less tolerance for having to relearn how to do the same thing they already knew how to do for the umpteenth time.
In a big company belief that something is worthwhile is better than actual value because market forces, that care about actual value, cannot act on each sub-project out of thousands of projects in the same company independently; but managers, which by definition can only care about their beliefs (idealism rears its ugly head again) do very much act on teams independently. The engineering department will pay $500k+ to engineers who exist only to pervert insufficiently representative metrics just as readily as the market would stop that from happening if they weren't hiding in an autocratic structure hundreds of times larger than the limits of their own abilities to be wasteful.
Indeed, there is no reason to think that the market has a greater ability to eliminate waste within a company worth $100B than it has within a communist country (which exists as an entity in international markets) worth the same.
Also $500k engineers should have as the only purpose in life to do engineering, working on projects that are found by management to be important, not to find themselves something to do. Most engineers are bad at business decisions and it would be a waste of time and skills for them to do it anyway. An engineer with business knowledge is no longer an engineer, is a product owner or a senior architect, etc.
Employees get so aligned with these experiment systems that obvious faults in statistical approaches go unnoticed. At the end of the day it becomes just a way to put something on your promotion case.
In most cases, core product stays the same over the years without significant innovation.
Before you complain how people waste their talent, maybe ask yourself what you did to help them. Or broadly how to truly make society reward people who do God's work. Society cares about getting likes, don't want to pay even a little for quality work, so smart people give them a platform to earn likes and show them ads. Everyone is complicit, the worst among them put all blame on "smart people".
right now these fake tech companies are literally the best we can do to keep everything moving forward on an economic basis.
Just as we tricked each other into bidding up crypto tokens, we could self-hypnotize to bid up biotech patents or something similar, but we have not yet decided to do that.
Leaving a review has become a huge bloated mess. The sight is slow as heck. And driving prices up just makes me more likely to look at alternatives. It’s still good, but it feels increasingly less fast and cheap.
Sure, the major shareholders might see their wealth increase noticeably, but anyone else (even small shareholders through option programmes) will generally not see a big effect.
Now, is that additional shareholder wealth really worth the years of engineer effort going into that?
Don't get me wrong: airbnb is an amazing service that has improved the lives of myself and other regular people.
I'm just arguing that maybe most of the benefit from society happened within the first couple of years plus maintenance and critical innovation as needed, and the rest is suboptimising for the benefit of the few.
Where the engineers are partially compensated in stock, RSUs or options? Absolutely. Its not like they're focusing on optimisations at the expenses of other, more dramatic changes.
That's because capitalism is a winner-takes-all game.
Spending 1000000 manhours to get 0.001% improvement in quality pays off, for exactly this shitty reason.
If you think about it, both are true. If you don't think these are true, you are far too caught up in a hamsterwheel to notice.
You're grossly misrepresenting the problems being faced and what forces engineers to work into the wee hours of the morning.
Just because stuff seems done to you or you notice no change, that does not mean nothing is being worked on.
Let's take basic A/B testing. To you, it's a blinking cursor. For the company, it's a bunch of business metrics being reported from N different components feeding into a data lake, with different versions of the same feature being deployed simultaneously to specific subsets of all customers based on their profile. But the cursor blinks differently depending on the market, and needs to comply with accessibility guidelines, on all X supported browsers regardless of their quirks. Some markets might not even have a cursor at all. Perhaps your cursor is only expected to show in a specific geographical location.
But you see a blinking cursor, and as you are oblivious to everything then you think it's just a tag somewhere. To you that's just a line of HTML, right? How hard could that be?
These basic everyday requirements are oblivious to those who have zero first-hand contact or experience with professional user-facing problems imposed by business requirements.
However, just because you're oblivious or unfamiliar to these requirements, they don't mean they aren't requirements.
Think about it for a second. If it's necessary to target a service to specific geographical markets to comply with legal and/or business requirents, and given it's considerably more profitable to target a shop to a customer based on their personal interests, why would you ignore that and naively presume that the hypothetical "blinking cursor" is straight-forward to implement? And I'm not even touching hard technical probs which most developers aren't experienced or competent in, such as security and reliability.
There is a widespread problem in software development which is this this tendency to be very opinionated over all problems in spite of being totally ignorant and oblivious to the underlying problem domain. Everyone is an idiot except themselves, who always hold the answer in spite of not even knowingwhat the problem is, let alone understanding it.
Could you paint me a realistic scenario for such multifaceted complexity revolving around this blinking cursor in a particular field? I can see how this might depend on a language/writing system but kind of get lost beyond that.
I do have experience with such "professional user-facing problems imposed by business requirements" btw but they tend to be more related to desktop and factory software each used by hundreds which is probably a few zeroes less than what we're talking about here.
>And I'm not even touching hard technical probs which most developers aren't experienced or competent in, such as security and reliability.
I know more than most that software can behave in weird ways but don't you think if there's a reasonable worry that altering this blinking cursor affects security that something is off?
Unless you're reinventing the wheel for each modification.
What leads you to believe in that? I mean, you have zero insight or understanding how things work or were designed. You have zero idea of where that blinking cursor comes from, let alone who owns that particular bit of code.
Let's think things through for a moment. Let's imagine you're talking about a blinking cursor in a random page from Google or Amazon. These are organizations where you have teams owning small widgets that show off only in specific pages, and that the page that you see in your browser come from a lengthy page engine pipeline that has all sorts of tests and failsafes, not to mention regional and localized deployments managed by whatever deployment policy.
This doesn't even take into account the whole workflow from product managers, who often demand data on the impact of touching a button.
You don't just edit a HTML file and hit save, don't you?
This isn't just a personal opinion. I've heard it spontaneously from various friends and acquaintances.
Granted we're mostly a similar demographic in a single market.
But even so.
I can't and won't speak on behalf of AirBnB, but you should keep in mind that in general:
a) all software grows by accretion until a breaking point,
b) paying off technical debt without solving any concrete problem or adding any tangible benefit is not considered a worthwhile investment,
c) a site did not changed for the worse if the data shows its conversion rate increased.
It's a business whose product transcends the site it's presented on, and exists in spite of the quality of its site.
I don't think engineers at AirBnb are overpaid or wasting their time but I do think most engineering leadership doesn't understand the idea of _true_ reliability where a service can run itself for long stretches of time.
As a company hires more and more people, modern development and devops can create as much unnecessary complexity as needed.