Uber's crazy YOLO app rewrite, from the front seat | Hacker News Reader

Uber's crazy YOLO app rewrite, from the front seat

blog.pragmaticengineer.com

261 pointsgregdoesit177 comments

> Ariana Huffington joins Uber's board around the same time and releases her book on the importance of sleep right around this time, and piles of free copies are available in the office. Yes, sleep is important. So why am I at the office past midnight? Right: the deadline that can not be moved.

This feels too familiar. “You have unlimited vacation, just take a week off, but we are not moving that deadline.”

You have free lunch, catering onsite chef or order from practically any takeaway. Right: just don't think you're getting a lunch for anything other than eating

It just goes in the long list of things companies say but clearly don't mean.

"Make sure you do the mandatory training called Safety First. On an unrelated note, we're pulling back on remote working and you're expected to be at your desk 3 days a week. Don't forget to wear your mask at all times."

I was at Uber during this time. Her book was stacked on tables all around the main headquarters offices, free to take.

Took a looooong time for them to disappear and I'm still not convinced more than a handful of people cared.

People didn't really like her internally. She was good friends with Travis apparently and that's about as far as the relationship to the company seemed to go.

There was once a time where employees could request a ride with her directly from the app but to be honest the prospect sounded horrifying.

> There was once a time where employees could request a ride with her directly from the app

You'd share an Uber with her? Or would she be driving the Uber? Sounds fascinating either way.

You'd share an Uber with her, yeah. I guess it only lasted a few minutes.

There was another time I think in Denver (might not be remembering correctly) where an event allowed users to hail a helicopter for a ride. Rumor had it that the event was so busy that the helicopter was cheaper than Uber Select at certain points.

To be fair this happens because there are those who are willing to hurt themselves in order to get ahead in their career. We've all had coworkers that send late emails at 2am in the morning or don't use up their vacation days because they want to appear as hard working and as loyal to the company. Many of these people have no real skill and sacrificing their free time is the only real edge they have. This is fairly common and observable at your standard big enterprise software corp. Not trying to shift the blame from companies but it does take two to tango.

This perspective ignores the power dynamics. The tangoing duo are not equal partners, the company has the power to exert pressure on the employee to burn themselves out, and in fact did in this story.

Shouldn’t management be able to identify that their team is comprised of these otherwise incompetent individuals? I see those 2am emails as a symptom rather than a catalyst.

The managers do it too. I've seen it all the way up to the VP level. There is just too much bloat in these large companies.

Xoogler here. Back at Google, there was an explicit policy that "heroic" efforts result in at most a bump of one level on the internal assessment scale - there was an idea that you should be doing productive work without having to overwork yourself, and too much "willing to work constantly" from reports was a danger sign for the manager's performance review. (Granted, this also could be super stressful for borderline employees, who are essentially told that even if they work hard, we'll still fire you.)

It make sense for a startup to heavily reward employees capable of "heroic" efforts because the company might not be there next week if the MVP doesn't ship.

But for a company like Google, nowhere near an existential threat, it makes sense to prioritize long term players capable of delivering for years.

Can you name some companies that have died from a week's worth of developer shortfalls?

Should probably restrict this to companies that had revenue and paid its employees.

If the sleep actually really matters and if long hours do damage long term productivity the way studies suggest, then the frequent heroic effort is irrational and detrimental to productivity. There is no way around it.

There is no way around it for borderline employees too. Assuming these studies are correct, what the heroic effort for borderline employee do is signaling desperation and emotional investment.

Depends on your view on cycling through resources. Who cares if someone who no longer works here is burnt out? Ergo, you have to tie that into the understanding that good employees are hard to find, and rotating them comes at a cost that's greater than not burning them out.

Even then you'll still get "just this once" people who understrand the problem but can't help themselves.

One of my old bosses literally put an LED countdown clock on the wall for our next major version release date.

He simultaneously refused to pushback features and kept adding new features well in to beta phase.

That product shipped buggy and no fucks were given on our side.

Highly focused development crunch time is "refreshing" from time to time. And since the involved engineers probably got pretty good salaries, it's hard to feel sorry for anyone.

But as I always say: Don't let a management problem become an engineering problem. The deadline was artificial and put stress on everyone. Uber is not an agency which has to deliver a product to some client. They have the freedom to release apps "when they're done" but choose not to do so.

While it's not an agency, Uber has to allocate resources for a finite amount of time for a particular task. If there are other projects predicated on this app rewrite, it's absolutely essential that the rewrite is completed by a certain date.

The only things predicated on the app release are marketing, etc. that they've decided to predicate on it. They also chose to take the launch on in the first place.

Sometimes there are external deadlines (events whose schedule you don't control, things for partners, etc.), but this just clearly isn't one of them. They could've released the update after Christmas - that was 100% under their control.

Indeed. Sounds like TK was a major issue here, but so were the whole tree of managers under him that agreed to ridiculous timelines that would destroy their teams instead of just pushing back on the deadline. Then again ... wouldn’t surprise me if TK very intentionally built such a management team.

Regardless, to be safe, I’ll definitely never work at Uber, or any other company TK has touched. This sounds like an absolute nightmare, and is so different from my experiences in ~a decade at other tech companies that it’s crazy.

The crazy thing is... his bold, risk-takey leadership worked. So who's to blame these executives for continuing to pull these kinds of stunts?

Sure a bunch of people burn out and leave but does that actually move the needle on any metrics besides turnover that these people look at? Assuming that departures get filled I don't know that turnover has that much of an impact on other metrics.

I think you are focusing on the business aspect and ignoring the humanity or lack of it in this situation.

I wonder what the health toll was on the all the hundred of people who worked on this? The reduction in life expectancy, health issues that arise due to 16 hour days...

It's not necessary.

I think their point was that TK&Co do ignore the humanity for the "success" (ie: their money)

Losing fully onboarded people certainly hurts velocity on projects after this one. And being known as a bad place to work is very damaging for recruiting efforts - I’d think a strong majority of devs would agree with either/both of “I’d never work for Uber” or “I’d never work for a Travis Kalanick owned company.” For a B2C tech company, where the product is absolutely crucial to the business’ success, having trouble hiring good people is a major downside.

Personally, about a year ago, a recruiter reached out to me about an interesting sounding role at “CloudKitchens”, that I was legitimately considering, but once I learned it was a Kalanick startup, I turned it down. The product/idea sounded interesting, but I’ve heard too many horror stories to work for the guy. I’m sure I’m far, FAR from the only one.

Did Uber struggle as a result of this?

I know they struggled as a result of Travis in general, but do they have trouble hiring? Has their development pace really slowed down (Kotlin aside)? Is their product suffering as a result?

I’d imagine they do have trouble hiring, and this hurts them via lower quality hires that they have to pay more for. But it’s a guess, I don’t have any hard numbers, likely few people outside of Uber HR do.

I’m sorry, but I don’t buy the theory that burning out your most talented engineers until they quit in an effort to put out an app update 6 months early (or whatever) is a healthy long-term strategy.

> [it] worked.

What worked ?

What important thing have been achieved ? An application that worked fine and did the business well have been rewritten entirely. Nothing wrong would have happened if it was released 1 week or 1 month later.

Even letting appart the human cost, I can’t even imagine all the shitty « dirty but it somehow works » code this kind of management create. The kind of technical debt maintenance teams will have to live with for years.

Until the next bullshit-deadline-rewrite driven by single-KPI managers

TK?

Travis Kalanick.

He says lots of people burned out and quit afterwards. Doesn't sound very refreshing. Sounds like the exact opposite: they spent everything they could and the adrenaline of the situation kept them going, and when that stopped they collapsed.

Well they resigned from the company, they didn't die. They took the money and went elsewhere or on a break

I've heard about this rewrite multiple times now from different perspectives, both public and private. It's interesting how much the narrative changes based on the context of how it's being told. This author seems genuine and honest, but it's also clear that the story is being spun in the most positive way possible.

The previous thread here has a different perspective on the situation: https://news.ycombinator.com/item?id=25373462

If you read to the bottom of the Twitter thread, it's pretty consistent with what we're hearing here.

It sounds like they're both saying there were lots of near-disasters, but these were overcome, and the app successfully shipped - however at the cost of burnout in the team. I wouldn't say either is putting a more positive or negative spin on it than the other.

Is it really a different perspective? This author is trying to do their best to give it a positive spin but the article is still a description of an unforced disaster that miraculously worked out for Uber.

I kinda hate this engineer, he didn't push hard enough, he let the professional bar be crossed, there was seemingly no business value in the whole project either. It would be refreshing if the devs just didn't take this, those that quit after should have quit during the project instead.

I think there's a saying that goes: "you get what you're willing to tolerate".

This is often the reason why employees get screwed over, they're willing to tolerate far too much, for far too little reasons.

Covering up bad management and planning is not something engineers should tolerate. You should say, no sorry, you missed the deadline to fund us and have engineering start early enough, that's that, you screwed up your planning, you failed at being a good manager. We're not going to cover for your screw up.

Edit: Specifically, I don't personally hate them, but I hate that it played out this way, and I know it's a difficult position to be in and it's difficult to get out of, it just makes me mad, and I don't know what solutions there are.. maybe to fight to change the norm back to hourly employment, or have contracts with a per-week maximum hour cap that if breached warrants bonus pay on top of salaried employment.

Not everyone wants to martyr themself to save management from hoisting themselves on their own petards.

Especially when it's just a taxi app.

Well they are getting paid like 500k no?

No, that's how you get exploited. You get paid 100% of your salary for 100% of your agreed upon time.

If you're then expected to work 200% of the time then you should be paid 200%, with the ability to say no if you don't want to. The actual amount is irrelevant.

This is a deluded way of thinking the world works.

Is it how things should be? Probably. But this is nothing close to reality.

This is bad advice and not what it means to be on salary.

If you want to be paid by the hour, be an hourly worker when you clock in and clock out. Then you are paid by the hour.

The whole point of being a "salary" worker is that you are not an hourly worker and you are not paid by the hour. In fact, you have no right to overtime as there is no such thing as "overtime" for a salary worker. There are state by state regulations that can limit how many hours you can legally work in certain professions, for example airline pilots are prevented from working too many consecutive hours, but there are never any regulations requiring you to get more pay if you work longer hours if you are on salary. Work expectations, time off, work-life balance -- these are things you need to negotiate and figure out when you sign that salary contract because it is absolutely not a per-hour contract. So please don't tell people it is -- instead make sure what you are getting into before you sign that contract.

i worked on the backend for this rewrite, and it was even more extreme than the other write-ups suggest. we not only rewrote the apps, we also

  1. moved from email-based auth to phone-based auth
  2. removed all the auth and onboarding logic from the clients and moved it over to server
  3. changed the networking from a hacky OpenAPI/Swagger code to Thrift over HTTP

i am not saying it was the most conservative call to do all this at once but it was necessary. the auth logic was basically untenable and no one really knew how it worked. some team in amsterdam could (and did) lock down uber for android users in russia because they changed some code.

we also had to move to phone as a primary identifier, because honestly there are many countries people dont even use email. but how do you deal with phone numbers that were validated more than 7+ months ago, if you want to avoid things like this? [1] or how do you deal with promotions that required validating phone numbers not being VoIP that you could do out-of-band before but now have to do in-band?

anyway, i am being scatter minded here but it was probably one of the riskiest things uber did, from my point of view, changing the top of the funnel + also moving users from one auth to another with a download, esp given with apple's rollout mechanism we had no way back.

but it all worked! i should probably write this up one day.

1: https://arstechnica.com/information-technology/2016/02/when-...

Can you expand upon the team in Amsterdam locking down Uber for people in Russia?

Uber is in many markets, but has to serve them all from a single distribution. That leads to logic, screens, and workflows specific to each customers situation. I imagine it isn't difficult for one team to inadvertently affect another, given this constraint.

i am being sorta vague here and taking some creative liberties with the markets involved. but at a high level, there was so much logic spread over various versions of the apps + geographies + some permutations of those. so you could add a check somewhere (app/server/both) like if (foo) and assume it'd not break in some markets, but lo-and-behold, if often did.

Yeah, on my team (payments) we completely underestimated the backend complexity, assuming there would be little to no effort, and pulling in a backend engineer “just in case”.

We ended up scrambling as it became apparent how many things changed (from polling to push notifications, thrift changes etc etc) and making the backend ready became a heroic effort itself... especially that we had very few backend engineers in Amsterdam at the time.

You should write up your experience!!

yeah, probably we annoyed the sh*t out of many payments folks as so much of the top of the funnel relied on things they should not have been doing. and the thrift changes, lol, that was something else.

but it all worked out seems like. and honestly, if you consider how fast uber moves compared to its competitors, im willing to bet it was the right call now.

> we also had to move to phone as a primary identifier, because honestly there are many countries people dont even use email.

And I'm over here without a SIM card in my devices, and not using a telephone number at all.

Well, it's a numbers game (pun intended)

  deal with phone numbers that were validated more than 7+ months ago, if you want to avoid things like this? [1]

Can you expand on how apps handle this usecase?

The short answer is, you need to validate phone numbers every 6 months (which is the time phone operators keep the number closed when people change numbers) if you need 100% confidence that they are the real owners.

This is a problem if you are relying only on phone-numbers as a factor, which Lyft was (so was Uber, in some markets). In reality, a lot of the time you verify the phone number and present a password challenge (aka ask for password) so it's not a huge problem.

But then, you'd need to handle the case when Alice signs up in with the phone number 123, then changes their number to 456, and 6 months later Bob signs up with 123 because they are the new owners. Now, Alice has to provide a new number (with some Grace period) and Bob has to be eligible for all the promos / signup goodies that were previously tied to that 123 number again.

Software is hard.

Most operators in Thailand (maybe other high tourism GDP nations in Asia as well) recycles SIM card numbers every 30 days.

Yeah — it's a never ending challenge.

If I'm reading this correctly, the whole fiasco started because the CEO decided they _have_ to release a new version of the app by some arbitrary date? And to meet this arbitrary deadline, the engineers had to roll out a ton of new features all at the same time inevitably causing a ton of fires/bugs/outages for the end users? Sounds like a nightmare and I'd be super pissed as an engineer having to support this mess. Is there more background on why Helix had to be released so aggressively?

At my former employer deadlines were often not even ostensibly about some rational scheduling or roadmap, but simply "the birthday of the long-deceased husband who made the company big and successful".

Slightly better than random tho right? Cause you can plan around predictable things.

I've done the deadline-via-management-boast thing, and it's highly corrosive. Burnout's especially bad when you look back when it's all done and ask "why?" and there's no answer.

What I don't understand is why they did a complete rewrite. If the CEO wants a "new app", you get to work doing a redesign of the UX and a refactor of the code base. You usually end up with a better product and better code, and it gives you the flexibility of not being all or nothing.

Some codebases have just mutated into junk beyond the point at which refactoring could still rescue any value, so a full rewrite actually becomes the better option. I would assume this problem to be especially widespread in apps of hastily growing startups.

Wow, what a nightmare! I remember a project or two like that in my career that I wish I could go advise my younger self to avoid. I have been around long enough to remember when people bore this sort or abuse as a mark of pride almost, like the scars made them who they are despite seeing peers burning out, becoming sick, quitting, or getting divorces as a result of overwork.

I'm surprised that "moral hazard" is rarely mentioned in these types of stories. By making their overtime effort available to the CEO as insurance against bad decisions, the engineers are in a way encouraging more bad decisions, since it's the engineers bearing the costs.

Until the entire team quits at the same time which has happened to many companies that do this stuff.

However, if this update failed nothing significant would have happened to Uber. So, they might not have actually cared.

My takeaway -- even the highest regarded engineering orgs are still shitshows behind the scenes.

In my personal experience, the more the org gives off the impression that it is hot shit, the worse it usually is.

The more the org values external appearance, the more the org justifies projects based on external appearances.

Sometimes i think about trying to get a job at Netflix just so i can find out what it's really like.

I had 6+ years at a FAANG. It was awful. I would never go back. I happily took a 20% pay cut to go elsewhere. This said, I learned a lot (both good and bad) and it has been a good reference point for my career.

So I recommend all engineers to strongly consider a FAANG while they're young and can tolerate a bit of burnout. Learn how the big companies do it, pay attention to what's good and what's bad, and keep that perspective with you in your career.

But be honest with yourself ahead of time about what would make you leave and follow through with that. "Golden handcuffs" of high pay is a real thing.

Not all teams in FAANGs result in burn out or require careful through about joining. I've been at one of them for 6 years and still find my work life balance to be sane. In fact I enjoy what I'm doing quite a bit because of the business space I'm working in and the quality of peers.

what do you consider sane work life balance? I consider anything over 40h excessive

Yes. Aside from on-call week when I carry a pager 24-7 for the week but I don't work outside business hours unless paged.

Yes what? I asked "what do you consider sane work hours?" lol.

Because it was obvious, we know what full time is defined as. Anything more than 40h is more than a full time job.

Another data point: I've never felt pressured to work more than 40 hours a week, and have probably averaged quite close to that over my FAANG career. Recently, with pandemic/WFH/less-to-do-outside-work, I started working some on evenings/weekends, and my manager asked me to stop because he doesn't want to have that kind of team culture.

+1 It can create a really toxic work culture. If I do work more its because I'm interested in a problem that I don't want to let sit till next week or the following day.

In general, I try to do this discretely by not pinging people outside working hours or trying to give impression I'm working late. Working late for me is usually working through code or a design at a time when I'm caught in it and don't want to let it sit until I solve it. If this happens I'll start late next day or reduce hours following week and I make sure I call this out during standup so my team knows why I'm signing off early.

It can become a snowball when one teammate sees another working and feels compelled to do same. Kudos to your manager for calling this out.

what was wrong with it?

It likely depends on the team and area of company. Any company of reasonable size will have a distribution similar to bell curve of teams ranging from excellent to complete shit shows. Most fall somewhere in the middle with diverse pros/cons depending on the tech stack, connectedness to customer and where their software falls in its lifecycle from shiny new to dusty needs replaced.

You get lucky rarely with an awesome team and usually fall in middle with some gripes. If you get into a shit show head for the door as fast as you can or stick around to learn a thing or two about what NOT to do.

Yeah, but if you're not already in the job, the distribution of "teams that are currently hiring external candidates" can be quite skewed towards the worse side of things, and if you don't know this going in, you'll have a very bad time at most large organizations.

A successful strategy is quite frequently to treat your first team at a company as just the second phase of an interview to get into one of the better teams in the company. Depends on the company, to be sure, but I've definitely seen many instances of "we're a good team, we know it, we'll happily poach good engineers from any team we like". It helps that the good engineers are going to be unhappy with their current team and looking to make the switch, anyways.

If you don't switch teams within your first year or so, it usually means you a) got very lucky at random, b) got into a good team because you knew someone internally with good info and were able to short-circuit the process, or c) you failed to network/interview well enough internally to move up, and you should probably either be okay with a dumpster fire or move to another company.

Funny, that was my first thought as well. Recently read their engineering blog post from 2012 on the API gateway + UI system [1] and thought it was really cool and made a lot of sense (in their specific, truly massive use case).

From recent memory, I can't recall Netflix really reinventing the wheel a-la-OP/Uber.

[1] https://netflixtechblog.com/embracing-the-differences-inside...

> I can't recall Netflix really reinventing the wheel

Netflix has many private and public projects that have alternatives that could have been adopted. I'm glad they didn't, because Netflix releases quality product and I'm super appreciative of their contributions.

https://github.com/netflix

It took me a second to correctly analyze the sentiment of “hot shit.” I realized it means “really good,” as opposed to, for example, “steaming pile of shit” which means the opposite.

It also generally has the connotation of someone having an inflated opinion of themselves.

You'll enjoy this exploration on this precise topic:

https://www.youtube.com/watch?v=ifNEgwGACEQ

Here's a link to the original video from the actual standup comedian in the video

https://www.youtube.com/watch?v=igh9iO5BxBo

The expression “shit hot” is a superlatively good thing. I don’t ever recall hearing “hot shit”, although perhaps it is a US term or a modern usage. Urban dictionary defines both.

"Hot shit" is a US term, and it is old and was around in the 80s at least.

Longer than that. I saw it in a book from the 1950's.

I expect it's a variation of "hot stuff" which I've heard in songs from the 1930's.

What about Netflix?

Yep, In my experience :

Very high standards = Slow shipping = Boring

You can bet that working on the Google Ads teams is far more 'boring' than Stadia or Glasses. Similarly, at my FANGAM company, the most boring teams are also the most stable.

> Very high standards = Slow shipping = Boring

I don't think this is quite right.

Slow shipping is tied to coordination more than standards (although if the standards are enforced by testing from another team, there's your coordination). If the standards are self enforced, you can still ship frequently, but maybe not a lot of big releases. Small teams and decoupled work where possible helps.

I can only quibble about boring though. The code changes may not be exciting, but the environment can be. Some external event that spikes traffic can test your high standards, and may need remediations (in which case, I hope you can ship fast). Of course, boring is actually nice.

Are you referring to Uber as "highest regarded engineering org"? :grimmace:

IMO, I would not put Uber engineering in the "highly regarded" bucket.

I've personally never thought of Uber engineering as the pinnacle of contemporary engineering teams, is that a common sentiment for others? I know they're popular, but I thought it was mostly due to the pay scale / name recognition, not technical quality.

Uber's engineering isn't well regarded. They have a reputation for over-engineering everything and having a misogynistic, cowboy culture.

Why is it legal for businesses in the US to ask employees to work around the clock like this? It seems like an insane and unethical practice

At-will employment. If you don't like, go do something else.

> If you don't like, go do something else.

Many people don't have many options and companies abuse the shit out of that because people don't want to be homeless, get kicked out of the country or dead when they can't pay medical bills. Etc. Thus the need for legislation. And the lack of such legislation partially explains fabulous areas like skid row.

Skid Row would be better if employers were prevented from having their highly paid salaried employees from doing crunch time?

You're pretending to misunderstand and you know it.

I'm actually not. I have no idea what the connection between Skid Row and this type of employment law would be.

that doesn't explain the _why_, that only explains the how this keeps persisting

Because large companies have a huge amount of influence over the political parties.

Good luck being a large US corporation without giving money to politicians. They'll shake you down with threats of regulation and other tactics.

They're incredibly well compensated and for the most part have the financial freedom to quit at any time as a result. They can choose to work in terrible conditions for a lot of money - that seems totally reasonable.

Now on the other hand, consider people who are stuck in part time jobs that treat them terribly for minimum wage. They don't have the freedom to quit because they can't afford to miss a paycheck.

In general, I don't think people who freely opt into anything should get sympathy for it.

How about if no one would get treated like this?

What is the issue with people willingly subjecting themselves to poor treatment in exchange for a lot of compensation?

It happens all over the place... medical students and residents are run ragged and brutally overworked. Young investment bankers are in the office constantly and always on call. Same thing with young lawyers in biglaw.

These are not people being abused or taken advantage of - they're making their own choices. You're suggesting that we should not allow people free choice in this context, and I think that's a much worse thing to do to them than to allow them to pick a job that's unpleasant.

Also violence happens all over the place, but it does not make it ok. Extremes and provocative-examples aside, the problem I see is not with exchanging life-obliterating work for money, but with signing for normal-ish work and getting brutal 24/7 instead.

We can think that they are free to go elsewhere like we can think that those in abusive situations can simply go elsewhere, but there are financial considerations, gas-lighting, just-another-month syndrome that leads to burn-out. Life is just in theory pack-your-bags-and-go easy.

Your violence comment is totally irrelevant.

If you thought going to work at Uber was going to be normal-ish, then you just didn't do your homework. That's on you. No reason we should make policy to protect people who won't do basic research.

You cite financial considerations, but again, these people are getting paid huge amounts of money. Unless you consider golden handcuffs to be a financial consideration that would prevent someone from leaving, that just doesn't apply here. That's the point - the tradeoff they're making enables them to leave whenever they want.

First, take a deep breath. This knee-jerk reactions do not help the conversation. I wrote "Extremes and provocative-examples aside". People do not leave abusive situations for a variety of reasons, some of which I cite in my comment.

I did not work for Uber, despite receiving a decent offer (a 4-yr package that given the current Uber valuation would have been worth between 1.5 and 2 million USD) 3 years ago.

because they pay through the nose for it

Money.

I have a lot of friends who worked at Uber around 2015. I didn't accept an Uber offer because they were all insanely burned out.

Props to the author for a great read!

Just recently I had to pull a couple 12 hour days resolving a critical issue for an enterprise customer; I'd never been so ready to hang up my keyboard and move into a cabin in the woods afterwards. I can't imagine extending that crunch for _months_.

I worked at Uber and this whole shit show was typical and unnecessary. From the time, one of the things that TK was so proud of was the logo redesign and the design of one of the floors in the SF HQ that had a bunch of scifi elements.

What was the benefit customers got out of any of this?

Not everything has to benefit customers to stroke a founders ego.

Or to get engineers promotions.

POA: Promotion oriented architecture is a frustrating form of castle building.

Theoretically, in a different company, great office design would make employees happy and more productive, for example. But I'm yet to see a better office design than a remote-first company.

Building the company is just as important as building the product. The company is more than the product.

I hope this is sarcastic because if not I'm going to frame this very characteristic Hacker News comment.

"Please don't sneer, including at the rest of the community."

https://news.ycombinator.com/newsguidelines.html

>Ariana Huffington joins Uber's board around the same time and releases her book on the importance of sleep right around this time, and piles of free copies are available in the office

So basically, work 80 hours a week and have Ariana Huffington shill her useless book to you? Uber sounds like a shithole if you ask me. Those people that quit made the right choice

> Tooling is a major issue: compiling the app from scratch takes 25-30 minutes on iOS and around 10 on Android

As a relatively naive to massive apps iPhone developer (I’ve made apps with millions of downloads, but nothing on Ubers scale) I find it confusing how a relatively straight forward iPhone app like Uber can be so complex. It’s basically a map and 3 buttons? And an ‘in the car’ state. They weren’t doing pool or bill splitting at this point I don’t think. I’d have thought everything else is back-end complexity?

Can someone do an explain like I’m 5 on this?

From memory, when this came up before there was a lot more information about how they have to jam in a load of stuff to deal with international travel so that everything works as soon as you arrive in another country.

This exact question was answered before here: https://news.ycombinator.com/item?id=25376346

Even reading all of that I am still surprised to find that the Uber Rider app is a over 1M LOC per platform. 20% of the functionality used by most users takes how many lines? It seems that they’re chasing the long tail of covering every global trip-taking use case. It might be fun to build, but how many of those features have a positive ROI?

It's the ancient Microsoft Word vs. Google Docs debate. Without long tail features Uber would be less valuable to each customer.

You land in Madrid. You hail an uber using the app. You stand there for 2 hours, hail multiple drivers. The uber never comes. For some reason the uber on the map drives in circles around the airport - then cancels. You eventually get in a cab. The driver says 1) uber cars can't be that close to the airport - and 2) that he doesn't take cards. You no longer use Uber in Spain. You tell the rest of your family to not use Uber when they get to Spain.

I asked this question to another iOS dev a while ago and they mentioned the way that uber is a single app that functions in all countries it’s available, you never have to download a country specific version by design. I think for that reason it needs a whole lot of regional specific functionality and localisation, I believe a lot of work is done client side too. Someone from uber could chime in and correct me if I’m wrong.

Edit: looks like it was also brought up in a previous thread.

Here‘s a pretty comprehensive video about it: https://youtu.be/zmeCYiD0hnE

TL;DW (IIRC): Uber is in a ton of markets and each one has significant differences, for example in payment providers - the app needs to support all of them and each comes with its own SDK.

Where to start? A wild guess:

- A/B testing capabilities

- Different features per country/region

- Different payment methods per country

- ...

I mean it's like e-commerce, when you stay in a single country, things are easy. When you go international, wild sh*t happens to cover all the spectrum of scenarios and respect the laws in place, while keeping it simple enough to maintain for devs.

https://mobile.twitter.com/StanTwinB/status/1336890442768547...

It's way more complex than it seems

Celebrity board members in a nutshell.

I'm curious, what would have been the repercussions of an engineer saying "I'm comfortable with at most 40 hours of work per week". Would they have been fired?

Also, were the people working longer hours getting paid overtime?

That was the year I sold my cars and switched to bike+rideshare. Even though my cc gave me an Uber benefit the new app experience was so flakey that I dumped uber for over a year, until I traveled to a place with poor Lyft service and found that the uber app was usable again.

I’ve never seen why the app needed rewriting.

From a cost/opportunity cost standpoint, it’s an enormous decision. I wonder what those numbers look like.

In general though I’m pretty biased towards boring things that work correctly as long as possible.

"The art of the rewrite" by Uber Leadership could be an interesting book. In 2019 When AB5 was passed another rewrite was done in order to make the case that drivers were not employees but independent contractors and teams were assembled into one "zone" working like crazy to make some major changes for one market (the fear was that AB5 would be the start and all other states would follow with similar laws). The funny thing is during multiple meetings people would ask if Lyft was doing anything as radical as Uber and AFAIK the answer was no. FFWD to 2021 and with the passage of Prop 22 most if not all of the features implemented as part of the rewrite including DSTOP (Drivers Set Their Own Price), Price range instead of Fixed price etc are being stripped out.

Burnout was an obvious cost to this mess. A less obvious cost must surely have been all the people that read about it and decided to never work for Uber. But then it's always been easy to ignore the costs that can't be measured.

My team and I were often the target for insane rewrite requests like this; I know this type of ridiculousness only too well. Sure you can succeed even under the most insane circumstances but that cost is not worth it for the people who are forced to do the impossible; in the end the people who suffered are mostly ignored while the top execs revel in the end result as if it was them personally doing it.

I finally said enough of this madness and retired this year. No matter how many times you manage to pull off miracles the people demanding them don't give a shit about you or your team.

This. If you're willing to routinely go "above and beyond" your normal work hours, then in management's mind that "above and beyond" will become your normal work hours and it will factor into their planning.

There are truly great things that can be accomplished in life with crunch time. I'd consider good examples as the early days of SpaceX or some other game changing endeavor.

A rewrite of a mobile app doesn't seem to fit the bill. I'm also reminded of the time that during an Uber ride the app took me into this secret screen where it attempted to get me to do some kind of coding assessment for recruiting. The engineering on this feature was absolutely nuts. But also completely over the top and laughably irrelevant for me and my geographic location.

I love the author's writing in general, and it obvious that he learned the lesson from this experience. What's insane is that their management has not learned that lesson, from, say, the last 30years of people writing about software project management.

I guess it will take such a caricatural death march to kill a unicorn before people get a clue. (And even that would be surprising. Not the killing - the getting a clue. )

Still, that's almost as great an advertisement for _not_ working at Uber as, say... everything else coming out of Uber.

> And as per Apple's restrictions, if your bundle size is over 100MB, users cannot download the app over the air

I wish stores implemented more such hard limits, or other strong incentives to reduce space usage.

I bet Uber could make its app significantly smaller than 234 MB (Google Maps: 137 MB, Chrome: 36 MB), but that would require effort, and it's cheaper to consume an extra 100 MB on 100 million other people's phones than to put in that effort.

Just curious…why?

For the vast majority of Uber’s users, space is basically free.

Because I can store either Uber's (and 10 other companies') bloat, or an extra 10 album's worth of music.

I'd rather have the music.

Well, in practice, it's not either-or, the vast majority of users get to have both 10+ companies "bloat", as well as 10 album's worth of music.

The base model of an Apple iPhone comes with 64GB, which is probably more than enough for the vast majority of Apple users — all the bloaty apps one might use on a day to day basis, all their pictures, and all their music. For an extra $50 you get 128GB. This is more than enough for the vast vast majority of Apple users (yes I'm aware that there probably exists dozens of idiosyncratic power users who might need even more). In fact, if you're the kind of user that's going to blow past that, it's probably because you're storing more-than-average 4K or 8K video and RAW images, where shaving down app size is probably not going to help anyway.

If you'd read the article you would learn the many reasons why the app blew past the bundle size limit. None of which particularly being their fault.

Here's a Twitter thread about the same story, that dives a bit more into the internal politics --

https://twitter.com/StanTwinB/status/1336890442768547845

I like this quote: “A bunch of folks burned out along the way. A ton of money was spent, hard lessons were learned, but still to this day most people insist the rewrite was all worth it. New engineers who joined up loved the architectural consistency and never knew the pain it took to get there.”

I'm surprised to read that most people left after the nightmare instead of during.

I would have taken this shit for a week, maybe 2, but past that, I'm out of here. If you're gonna stick around during the whole thing to see it through, why leave once it's over?

A lot of people want to stick around to complete a project as it looks good on your CV. Particularly at a junior level.

From my experience (in games) it’s also quite intoxicating at the time and you get quite emotionally invested in supporting the team and fixing issues. In particular crunch is partly enabled because you get close through the adversity. You usually don’t get much opportunity to take stock until afterwards being so caught up in the moment. It’s pretty common to get a rash of departures right after it all calms down.

They didn't have a choice. Imagine their reference check for future jobs where Uber HR tells the new potential employer "X left during the start of crunch time for our new app". You have to suffer through, then leave.

Fair enough, but honestly, I would probably just be open about why I left Uber. I would want to make sure I would not end up in the same kind of toxic environment.

HR is not going to snipe an ex-employee in solidarity with the new employer. As a rule, they only confirm the person worked there when they said they did.

> As a rule

Rules are there to be broken, and even those mandated by law will get broken (see for example the non-compete cartel drama from a couple years back).

Burn out.

This first time I did a game jam many years ago, I stayed up for nearly 72 hours to get it finished. I slept maybe less than 8 hours that entire time, I felt like absolute shit towards the end. Nausea, headache, exhaustion.

The game was completed, but to this day, if I even look at a screenshot of that game or hear one of the sound effects, I start to feel nauseous almost immediately.

I'd imagine some people on this project had a similar (although maybe not as extreme) reaction.

I wonder what the effective hourly rate was that these developers were paid for all of this work.

Another example of out-of-touch upper management dictating an unrealistic development schedule for no particular good reason (according to the article), and making everyone's life hell for months.

This whole blog post reads like The phoenix project book. I actually chuckled at the part where the legal got involved for the client-side fix like it so used to happen in the book.

Shit like this is why Uber should have failed. Total disregard for humans. TK is a monster.

> I have no idea how, but two and a half-hours later, the fix is live. I'm assuming the release team used all the goodwill accumulated over the years to make the fastest app update I've ever heard of happen. Either way, it's done, and done fast. Let's just hope it works.

It's sort of amazing how we as an industry have all just adapted to the App Store industrywide censorship and gatekeeping.

It's astounding that Apple has been able to pull this off, and even more astounding that so many companies are willing to work as sharecroppers for them.

Find it on the Apple App Store under AOL keyword Uber!

my question is, what was the business value of doing a full write rather than incremental shift? same question on the specific date - was their any business value delivered by specifically launching the new app on New Year's?

If not, this rewrite basically cost huge amounts of burnout across the mobile and server teams for no business value. At best it seems like an intense learning experience that didn't actually unlock $$.

>Two weeks before my starting date, my manager emails me to ask if there's any way I can do Android.

I've solo made and published over 15 Android apps but I went full Dunning-Kruger and now I think I know nothing about Android development. It is too complex. If manager asks me, I don't know Android. I can't imagine Android development can be known.

If you ask someone if they know Android development and they say yes, be skeptical.

To be fair, this was in 2016 :)

A hundred engineers to rebuild the Uber app....

That is insane! To me it seams like something that could be done with 2-3 guys over 6 months with room to spare.

But yeah I get it they have the money and everyone wants to work with their friends.

I have no problem with leaders encouraging overworking, but the article could have detailed the compensation that people got during the work. My guess is that they were getting not much more hourly wage than a McDonald's worker for correcting for the bad leadership.

> My guess is that they were getting not much more hourly wage than a McDonald's worker for correcting for the bad leadership.

I agree that this crunch time seems unnecessary, but it's mathematically impossible for Uber engineers to work enough to reduce their wages to anything near fast food wages.

Even someone working 80-hour weeks, 52 weeks per year at $15/hour would earn $62K. That's less than half the starting salary for entry level engineers at Uber.

Entry-level compensation at Uber is around $160K, with senior software engineers earning $400K or more: https://www.levels.fyi/company/Uber/salaries/Software-Engine...

>80-hour weeks, 52 weeks per year at $15/hour would earn $62K

Someone not salaried would be eligible for overtime and double time if working 7day weeks. That would significantly change the amount earned at $15/hour

Doubling the wage for the 40 hours of overtime still doesn't get you to $100k, and the people working on this were all making much more than that.

OP here. We did not get any additional compensation for the long hours beyond you could expense food at late hours and on weekends. And at the end of the project, my director told everyone to take their SO to the most expensive restaurant, hire a babysitter if they have kids and expense the whole thing. I submitted a bill of €300.

When I joined Uber, my compensation effectively doubled from my last - already pretty well-paying gig - so even if calculating the per-hour wages for those 3 months, I made good money, and nothing comparable to the hospitality industry.

That sounds ignorant of reality and abusive at best. In my former org when we go into q4 things get a bit crazy and chaotic. Once the dust settles my manager would let us take a week off and do whatever we wanted. No vacation needed, just don't come to office and do something fun.

Its sad how many orgs still abuse salaried employees time.

Let me guess, some kind of ad, marketing or e-government agency? The "clients have leftover budget they need to burn or next year's budget shrinks" problem is real.

I wonder why no large company has ever discovered the absurd waste this budget policy causes.

Add to that the problems associated with tenders (= sometimes absurd amounts of money wasted for pitching) and the overhead of way too many stakeholders and feedback loops on client side... in many a project the single thing that makes the most savings is a competent (!) PO on client side that gets "dictatorship powers" from their C-level that can override petty politics and internal squabbles if they threaten progress too much.

Sorry, to be clear I don't and never have worked at Uber. My former org was a reference to a team I was on prior to current team at another company.

So you work (at least) double hours, a bunch of employees quit, "extreme stress", everyone "giving 110%", and you get nothing but a dinner out of it and the salary you'd have got if you'd worked a normal role. Uber's stock, meanwhile, got as big as Travis's ego...

What a fucked up world where this is deemed reasonable. You're even able to justify it as "hey it was tough and ridiculous and half the people didn't make it but look, I wrote some books and was able to become a manager."

There is always gamedev, where after a year of 80 hour weeks, the game is launched and… half the people get fired because they are no longer needed.

> world where this is deemed reasonable

I mean... the world where uber engineers are getting paid 300k a year, is probably that world where people might think that this is reasonable, given the compensation at companies like uber.

Sure, I understand, it's a relationship that is still quite good, so if you are young, healthy and can manage the stress, it's probably not a problem.

I have another question though that's somewhat unrelated: I use Uber quite often in latin america when I'm travelling, and my main issue is driver cancellations after 10-30 minutes of waiting for the Uber to start coming to my place (outside the city). It's quite frustrating to me to read about all the UI improvements over the year at Uber, and it seems that software engineers at Uber don't care about the real user experience (waiting a lot for nothing). I would happily pay more to the drivers if this behaviour would stop (they cancel because I'm outside the city usually), but nothing has changed in the last 5 years about it. I don't care about the nice icons or dark mode or other UI changes BTW, because that's not what I remember after a ride.

Reply on news.ycombinator.com