Swift architecture at Uber
skilled.io
skilled.io
The big picture observation is that for value types with storage larger than a few words, several instructions must be emitted per call and per storage word because they cannot be passed-by-value in registers. And Swift often emits more calls than you can see (e.g. thunks, protocol witnesses, weak sentinels, etc). This is not new to Swift—the same requirement exists when passing large C values around—but we use value types a lot more in Swift for various reasons, so the issue becomes more salient.
In terms of remediation, I audited our app for all structs larger than a few words. I just did this manually; there were something like 120 structs to look at. For each, I converted it to a class then evaluated the impact on generated code size. Only four structs had meaningful impact on generated code size (to the tune of ~13MB), and happily, they were fully immutable, so they retained their value semantics even when converted to classes. If they had not already been fully immutable, I would have had to spend some time either adapting the classes to achieve value semantics, or adapting their clients to tolerate reference semantics.
Then I audited our app for all enums larger than a few words. These can be made pass-by-reference by using Swift’s `indirect` feature, which implicitly boxes associated storage. We had one enum for which this made a substantial difference, to the tune of several MB.
Then I had to make sure runtime performance hadn’t been too badly damaged by all the new dynamic allocations and indirections. In the end, I observed nothing noticeable. We don’t have formal repeatable performance tests, though—it would have been interesting to see the impact on those.
C++ has this issue, too; it largely handles it by using reference arguments when consuming large stack values. In the future, it’s possible for Swift to optimize many cases (especially intramodule) where this occurs by allowing deeper stack frames to reference values types stored in parent stack frames when it can prove it’s safe. Rust has a lot of fanciness here you might find interesting!
Another recent improvement is copy-on-write existentials. There's no code size win here, but it improves runtime performance by avoiding copying the payload when passing existentials by value.
I think they have some more infrastructure that can audit stuff automatically in the same way you did, but I'm basing that off of a conversation I had and I'm not familiar enough with the codebase to find it; maybe it's just this set of tests.
I'm a bit baffled by that. Is the Swift compiler that naive?
Surely you know how to assess the number of processors/cores on your system and you spawn threads in a way that doesn't lead to diminishing returns. You use a bounded thread pool and you stay within these bounds.
Seeing such a speed up from simply merging text files is really puzzling to me. You have to type check the code anyway, surely the overhead of opening a new file is completely negligible compared to running this type checker on the same file? Especially since all these instances of the type checkers have to share a lot of data anyway.
Swift is very cool and I'm excited to see it and Kotlin become our next generation languages for mobile, front end and back end alike, but it seems to me the Swift compiler is still very immature.
As usual, compile times depend on what you're used to. It is something we're working on improving though; we want it to be very fast! We expect incremental recompilation to move out of nightly soonish; that will help quite a bit.
I don't think that implies those processes all were spawned at the same time (or even that Apple's compilation process did the spawning. Given the effort they spent on their build processes, I do not rule out it is their tooling that does the spawning)
The thing is: all these processes are independent of each other. Uber likely has tons of files that are imported into many, many files. Each compilation process that imports such a file has to individually parse it.
I can see two ways around this: reusing intermediate results between processes and using fewer processes.
The first is like precompiled headers in C/C++. The second could mean having a queue of to be compiled files and a fixed number of compilation processes that pick up files from it.
Risk in both cases is that unintended state may leak between compilations. For example, compilation flags may be different for files f and g. Does that mean g can't use preprocessed module m generated while compiling f? Because of that, I would go for something like precompiled headers, because it makes it easier to reason about what information flows between compilations of f and g.
I don't know whether that would be sufficient, though. The biggest concern I have w.r.t. Swift is that all its cool features, combined, make it essentially impossible to build a fast compiler (Yes, languages such as C# have most of them and aren't that slow, but adding protocols and overflow detection (which you may want to optimize away a lot, even in unoptimized builds) may just tip the balance)
I am screaming out loud. Please open source this!
I love Swift, and I love Kotlin.
To any young programmer out there (I'm in my fifties), learn these two languages and you will be highly employable for the next decade, on top of the wave.
1) Learning new concepts in computer science. For example Swift uses a lot of modern concepts around type safety, closures, parallelism, synchronization, etc. These are a big learning curve but they are not specific to Swift and you'll notice other modern languages are adopting the same concepts.
2) The UI frameworks are fundamentally different. If you're an option expert at XCode constraints, storyboards, etc, none of the applies to android and vice versa.
I can tell you this, if you do bite the bullet and learn both very well you'll be surprised how much it helps you learn other systems more quickly because you've see all the concepts before.
For example, node.js, many people say its popular because lots or webdevs knew JS already, but that's not even close to true, most people went to node.js because they found a couple of usable libraries on npm that gave the trust necessary to start the project.
None of those things seem particularly modern to me. Does swift actually have any concepts that weren't already implemented in other languages by say, 1980?
It might seem like I'm being pedantic, but the flip side is - why not learn older, simpler, more mature languages that already have those concepts?
TIL that ML is ooold.
Modern for mainstream languages. Non-mainstream languages are irrelevant to the discussion, since nobody cares about theme except niche industries, hobbyists and academics...
>but the flip side is - why not learn older, simpler, more mature languages that already have those concepts?
Because those languages are not tied to a $50 billion app industry or have major adoption and increased support.
Ada. [0]
It's been around since before '83 (when it became an ANSI standard), and was developed for the DOD, to replace the hodge podge of languages they were using, with an emphasis on safety.
It has all the above, whilst focusing on being plain English.
Ada also has a few things that are considered to be fairly modern, and has had them for a long time. Such as:
* No Primitive Data Types
* Type Contracts
* Subtyping, operator overloading
The more languages change... The more they stay the same.
These concepts aren't new.
They wanted something safe for embedded, and the original specification in '83 included generics, so you could handle data structures in a nice, safe, performant manner.
Ada 2012 probably even has more generic data containers on their standard library than Swift.
ADTs can be done via tagged records.
That's interesting, I did not know that.
From a quick glance it appears they do not have bounded polymorphism ("protocol-constrained generic parameters"), associated types, or existential types ("values of protocol type"). So for example if you have a generic Set data type you would have to pass in an equality and hash function to each operation instead of saying that the element type is Hashable, etc.
The explicit instantiation looks quaint, but it reminds me of ML functors for some reason:
procedure Instance_Swap is new Swap (Float);You can use abstract classes and interfaces for that.
But, if you insist:
> Ada does not have parametric polymorphism.
Yes, it does. [1][2] It's supported through the use of generic units.
> Ada does not have ... algebraic data types
Ada does have tagged records, and other variant types, and has had for quite some time. [0] They aren't quite Sum Types, but are incredibly close.
> Ada does not have... Objective-C interoperability
GNAT does. It's part of the GNU Compiler Collection, and as such, can be linked against other languages supported by the toolchain. GCC also supports Objective-C.
[0] http://archive.adaic.com/standards/83rat/html/ratl-04-07.htm...
[1] https://rosettacode.org/wiki/Parametric_polymorphism#Ada
[2] https://en.wikibooks.org/wiki/Ada_Programming/Generics#Param...
Yay! :-)
One thing that's probably almost unknown these days is that Objective-<X> was always supposed to be something you can easily add to any <X>, and in fact there were quite a few of these, including Objective-Assembler.
Ada was a beautiful design for its time. Maybe its most fundamental flaw was government oversight. So much about designing a widely successful language is non-technical, choosing the right features, trends, hardware, that builds enough momentum to support a self-sustaining ecosystem. To do so often requires an agility that the DOD just doesn't have. Ada is one of so many examples that reminds us technical superiority commonly loses out to pragmatism.
I do like the name, it would have been fascinating to know Ada Lovelace.
What does Swift have in terms of parallelism/concurrency?
To be honest, I would've said going deep on JavaScript right now (React, React Native, etc) would give you a shelf life of at least five years.
Kotlin seems like a bit of a random recommendation - it's not in the TIOBE top 50, and a quick search for Kotlin jobs on indeed.com (the first job search site I found from Google) reveals several orders of magnitude fewer jobs than for anything mainstream (hell, even Haskell beats it comprehensively). Perhaps it's the next big thing though!
Already there are large apps that are built around React Native. But more importantly, js as a language is becoming indispensable.
A few days ago, there was a discussion about Netflix - who specifically re-engineered it's back end infrastructure to let it's app engineers (who are js) write API services in nodejs. We are talking 40% of The Internet here ?
There is a large boost in hive mind productivity with one programming language . If Java 9 Truffle/Graal.js is as good as everyone says, then it is pretty much js all the way down.
And for those who think js is a shitty language (god knows I did), please try out ES2017 or Typescript - you will be pleasantly surprised.
Because "they already knew js". I know that Hystrix is still core, but IMHO the shift has started.
I don't get this. Is it saying that structs can increase your binary size and as a separate issue they are created on the stack. Or is it saying that because structs are created on the stack they can increase your binary size? How would that work if stack allocation is something that happens at runtime and affects your memory footprint rather than binary size? (I don't use swift so I might be missing something here)
He did mention running a tool to add explicit types everywhere, but it's very often a matter of just writing the most generic ones. Maybe not in uber case, but for everybody else you should try it.
Apart from that dictionaries and arrays should be typed.
Also some overloads are really hard to process, especially the '+' operators are pretty expensive when used on non-integers and not (previously) strictly typed values.
Never do something like "This is a " + birdName + " on it's nest" but use the "This is a \(birdName) on it's nest" method instead. But sadly it also applies to libraries like Cartography that rely on such overloads to create more elegant declarations.
Uber Everything is hiring iOS developer for logistic service? I used to work in German logistic which uses Java technology and does having lot of issues everyday. Swift would prove logistic industry in a new way and probably use Grand Central Dispatch (GCD) which is powerful. If Swift will supported Corountines after 4.0, useful for server side swift.
Sorry, I find UberEats UI is distracting with animations. It does not feel native to me however I tried to, the font is awkward to read when Sans Francisco font looks much cleaner on native iOS app.
Good to know
The frontend jobs do not share state, so each one parses all the files in the module.
In general the parser is very fast, and the type checker tries to only type check declarations in files other than the primary file when absolutely necessary, so it's not always O(n^2). But there are pathological cases you can construct today where the type checker ends up doing too much work.
Seems easily avoidable though through basic dependency analysis, though.
However, with LTO (link-time optimization), the compiler doesn't finish compiling and writes out .o/.obj files with partially processed outputs. The linker is modified to re-invoke the compiler to finish compiling all the files at once. In GCC this is available with -flto.
Actual amalgamation, where you combine many C files into one, is often not possible without modifying the files. It works for SQLite because they've made sure that their code works with amalgamation.
On a large enough codebase, being able to rebuild only what you touch and relink is the only way to get acceptable build times in dev.
https://h4labs.wordpress.com/2016/02/09/should-i-use-objecti...
But the old Uber app had hit its age limit. It was build on top of technology chosen for a small number of features (ex: a single global DI component, lack of typing, a small MVC hierarchy). So we either needed lots of large migrations or a rewrite. The incremental migrations required to fix these issues would have been extremely disruptive, they wouldn't have gotten us an entirely refreshed UI and they wouldn't have given us a number of other benefits.
So we decided to do a rewrite. We had lots of engineers that knew the issues to watch for from the first time we wrote the app. And a handful of us spent months researching/building different architectures, static analysis and tooling that would ensure the rewrites success. We weren't going to repeat the same mistakes twice.
On the very first day the app launched it was more reliable and performant than the version of our app that we had been maintaining for years.
We have also taken advantage of the rewrites to rebuild in Go and Java. Most of our older systems were NodeJS and Python. Many of those have been rewritten in the two languages we've now mostly standardized on.
Pretty much every rewrite I have witnessed has gone well, so I can only guess there's something we're doing right in this respect.
One of the best policies we have at Uber is our internal transfer policy. So long as you're in good standing perf-wise, you can transfer to other teams and projects within 1-2 months. For such a policy to work, it needs to be easy to work in unfamiliar codebases. I can't speak for Java, but it's much easier to drop into an unfamiliar golang codebase and be productive and not introduce bugs than it is to do that with NodeJS or Python.
I do exclusively golang work now but when I joined I was doing exclusively NodeJS. While the first two to three months with golang were a bit frustrating, I can't imagine writing a backend in NodeJS ever again. That said, I would still use NodeJS to build frontend developer tools, but that's about it.
Yeah, when a team goes from 2 to 100, a serious business/engineering manager got to ask a few questions. However, if money honey flow is not a problem, it could be in everyone's interest just to ride along
Why would I as a user want to download a variety of separate apps to hail cars, etc in different locations? People got really pissed off when Messenger was split out of Facebook (it's still mentioned in app store reviews to this day) I'd imagine they would not be wild about Uber doing something similar. I could see this getting really annoying really quickly.
Imagine landing at a new airport and having to download a new app just to hail a car or a tuk tuk or whatever.
Different locale, different customs, regulations may dictate entire different screens.
How an "app" is designed and packaged internally to enable the business to respond quickly does not have to impact the use experience. There are plenty of design patterns and engineering experience to draw from. It's a somewhat boring topic.
I find it more interesting to find answers to question such as:
1. A native person in India will be presented with an "Indian" app, satisfying the locale, regulations, laws, culture, etc.
2. A tourist landing in India, well, er, what version should that person use? Laws, regulations still apply (well, Uber may have a different take :) ) but how about the user experience and colloquial details? Would the tourist prefer something from home or something that more accurately, and perhaps more apt for India?
"There are plenty of design patterns and engineering experience to draw from. It's a somewhat boring topic."
I disagree, I think its a fascinating topic but then again I might be biased as I'm a mobile developer ;) Very few organizations have single app's that are worked on by > 50 people. I worked on the FB iOS app for several years. When an app gets that big you run into all sorts of problems that are not obvious both from an engineering and product perspective.
So to me it's pretty interesting to read about how Uber tackled these problems. Especially as it pertains to Swift which has had quite a number of performance issues and language changes.
"1. A native person in India will be presented with an "Indian" app, satisfying the locale, regulations, laws, culture, etc."
Why not have the app detect that and adapt to that user rather than having a separate app?
"2. A tourist landing in India, well, er, what version should that person use? Laws, regulations still apply (well, Uber may have a different take :) ) but how about the user experience and colloquial details? Would the tourist prefer something from home or something that more accurately, and perhaps more apt for India?"
When you land at a different airport currently uber provides a customized experience for that airport/locale. It knows that you are in a different local.
"From a user perspective, you may not have to download a new app."
Point is that even if it isn't a separate download, in different locale's, it may be essentially a separate app with different screens and UIs.
What you describe is what uber currently does?
> This application has served us very well for the past four years. But as we've expanded and exponentially grown our mobile engineering team, we started seeing breakages of the architecture, we started seeing problems where feature development would become pretty hard. We had to test multiple code-paths because we shared a lot of view controllers amongst different teams. We really started to hurt with the old architecture because it was written by two engineers and we had grown the team to over one hundred, at that point.
The actual quality of the software isn't as important, since Uber drivers and riders don't have a choice anyway.
Terrible networking? how many engineers does it take to implement a few flavors of retries? :)
Uber growth in term of getting drivers, that's what I want to know. It's not just ride subsidies, but the flow of investor honey helps, of course, but they do have some alpha growth managers.
At scale, across so many different markets, with so many different features, things get complex fast. We want the user experience for both riders and drivers to be magical. More magic requires more engineering. The goal is transportation as reliable and available as running water everywhere all the time.
Here's an example of telematics engineering used to improve safety by detecting harsh braking: https://eng.uber.com/telematics/
We also use the IMU measurements to detect drivers who aren't using a phone mount and are instead holding their iPhones while driving.
Features like these allow us to advise drivers on how to be safer and get better ratings since not using a phone mount and harsh braking both correlate with negative user ratings.
A lot more goes into the Uber app to make things magical. I used to read the front page of HN daily to read about all the cool things people around are working on. Since joining Uber, I check far less often now because there are so many cool problems being worked on internally that many of the front page stories sometimes seem quaint in comparison (not that many HNers aren't working on awesome things but the quantity and quality of cool problems my colleagues work on easily captures most of my attention these days).
The best way I can describe working at Uber: it's like building that transportation component of sim-city for every city, everywhere, but it's not a simulation. Despite all the click bait negative press, its bar none the best place I've ever worked.
It's easy to drop in and criticize the architecture of some older systems, but it's also instructive if you were around when they were built and knew the trade offs and constraints that existed when they were first conceived. The growth we've experienced has been astronomical and it would have been non-trivial exercise to build systems 2-3 years ago that account for the scale and business needs today. Even today, it's hard to plan farther that 2-3 years out with the growth we are seeing. I work on a system doing millions of QPS and in 2-3 years, it will be handling an order of magnitude more QPS and more business needs. We might scale horizontally or decide a different solution is needed. I don't think there is a silver bullet and the microservice architecture means that rewrites and re-architectures are tractable problems.
Your sentiment is valid, but consider this: whenever you're working with a project with a ton of money on the table, it becomes possible to add small bits of functionality which more than pay for themselves in terms of return on investment for the engineering effort. This is why you can often write a basic version of a more established application in "a weekend". We've had extensive discussion and meditation on this topic on HN before[1]. Basically apps will grow and add features as long as the engineering effort will pay for itself (I suppose I use that term loosely in this case).
Also consider that the UX is different in different markets, and the reason for that is related to the first issue.
I sympathize with what you're saying, though, and couldn't help but think about Alan Kay's talk that we discussed the other day, about natural complexity vs artificial complication.[2]
[1] https://news.ycombinator.com/item?id=12626314 [2] https://news.ycombinator.com/item?id=14188759