LightSpeed: Rewriting Messenger’s codebase for a faster, smaller, simpler app
engineering.fb.com
engineering.fb.com
Munch Munch Code Munch WTF Munch AGILE! $$$ Munch.
Designers: 1. We need a new design system. 2. We need a new brand! 3. We need a new font!
Users: ¯\_(ツ)_/¯
/s <3
These cycles pop up all the time in development as priorities shift (or as developers find more excuses to “fix” what isn’t broken). Real world coding always involves compromise - leadership is recognizing which compromises to make without continually backpedaling.
And in the case of Messenger, I highly doubt it was the developers who said, "Hey, let's add this completely unrelated thing to our codebase - payments!". More likely it was marketing or "product" that told the devs what to add.
And the amount of time spend writing a bit of glue code, specially on Android, is still less than debugging integration issues in all those Hybrid/Abstration frameworks.
Because they haven't been burned by that cycle yet.
Sounds like not only did they make the app quicker and lighter, they improved code maintenance too. If the android app is as the same size as the new ios messenger app, they'd still have less code to maintain than the previous ios messenger app.
See also: this other comment - https://news.ycombinator.com/item?id=22468036
Wonder if part of this is custom video / audio codecs? That I’d imagine would contribute a good amount...
Simple: they added tons of crap nobody care. On this list, I only use video. Messenger is valuable because of the network effect, but feature wise 2003’s MSN was good enough.
With that many features, it doesn't surprise me that the previous codebase was 1.7M LoC. Sure if you count only the messaging part, I'm fairly certain that the LoC is way lower than this.
It doesn't make sense to compare the number of LoC of messenger to the number of LoC of software like Postgres because Postgres tries to be good at 1 thing and messenger tries to have a ton of features with good UI and UX.
I don't like many of their features (I'm not using the stories or the payment), but some of my friends do and you have to recognize the work that the messenger team did to make them work that well.
> _I_ only use
(emphasis added)
Just because you don't use those features, doesn't mean nobody cares about them
Another plausible explanation is lack of code re-use and reinventing the wheel because you don't know someone else solved the same problem. Also correlates with number of engineers.
Is there anywhere that suggets it was ever non native?
Rather than reinventing the wheel, we used the UI framework available on the device’s native OS to support a wider variety of application feature needs. This reduced not only size, by avoiding the need to cache/load large custom-built frameworks, but also complexity. The native frameworks don’t have to be translated into sub-frameworks. We also used quite a few of the OS libraries, including the JSON processing library, rather than building and storing our own libraries in the codebase.
This is awesome. It shows the benefits of building an app leveraging the OS, instead of fighting or abstracting it.
These are supposed to be the smartest people in SV, but from the outside, it strikes me more as a "No shit, Sherlock" moment.
Things like this are why I'm glad I'm not in the bubble anymore.
When I got ahold of the source and started diving into it, discovered the client had 2 copies of the data, which was kept in memory. The server had 3 copies of the data in memory! Worse yet, the primary cache would be updated, then a multicast message was broadcast to update all the clients and the server's secondary and tertiary caches...thats right: the server would update the 2 other caches by broadcasting a multicast message, receiving it, then updating the other 2 after message receipt. A re-architecture was definitely in order. Took 3 months. Took another 3 months to get the interface to be backwards compatible with the original version.
Mind, the server was responsible for serving up relatively static market data for finance. But, considering we traded globally, we had +400 objects we cached, more than a few had 100ks of record. Memory usage for the server was many GB.
The rewrite involved moving to a single cache, instead of multiple for both client & server. It was also architected to handle large volumes of updates. Original implementation could even handle 100 updates per hour without bringing the server's and all clients to their knees. The write could handle +10k updates per second amd clients would barely notice a slowdown. Write locks were only held for a pointer swap. Memory usage for the server went down by close to 2/3. Client memory usage by 1/2.
During that time, also moved from static libs to dynamic on both Linux and Windows, so I could make nonbreaking changes without a firm wide recompile/relink, which typically took 3 days. HUGE productivity boost. A small bugfix didn't require a firmwide recompile/relock for the smallest bugfix.
They might just be. Everyone else is cargo culting on React Native and the like.
“They’re not confessing, they’re bragging”
I’m guessing the oldest code in Messenger stems from a time when the OS-provided libraries were not up to par or simply didn’t exist yet. Partly because it’s not until now that the OS manufacturers finally see what there is demand for in apps.
> For any cross-platform logic, we used an operating extension built in native C code, which is highly portable, efficient, and fast. We use this extension for anything OS-like that’s globally suboptimal, or anything that’s not covered by the OS. For example, all the Facebook-specific networking is done in C on our extension.
C was the universal donor 30 years ago, and it's the universal donor today!
If you look further down the article, it describes their extensive use of sqlite, and a layer around it for managing synchronization with FB servers, also written in C.
I write a lot of Python and a lot of Go, and I love C. So I enjoy imagining the subgroup of HN commenters who wail that Go is ignorant of the last 30 years of CS technology, or that no one should be allowed to write C these days, reading this.
RN isn’t the right tradeoff for Messenger — whose new core is written in plain C — but its use at FB in general is growing, with 750+ screens in RN. So the rumours of its death are greatly exaggerated.
I think it’s great to have different options with different tradeoffs available to choose from. But hey, that’s just me.
- NumPy is 360K lines mostly C
- Postgres is around 2.1M of C
- Go 1.13 is around 1.5M of Go code
- Rust 1.37 is around 1.2M of Rust code iirc
- core llvm is around 3M of C++ iirc
...
A chat app is 1.7M. A chat app...
Makes me feel sorry for all the time spent/wasted by all those devs, many of whom are unquestionably brilliant and could've worked together on something truly awesome, big and useful. But we all know that doesn't pay the bills and so here we are.
THAT code, my "all encompassing, personal repository for every VR app I'd ever make", is only 100K lines of code.
I once downloaded all of my dependencies and counted all of those lines and still only got to 200K. Well, not counting Unity. But at that point, you might as well start counting the LoC of the OS.
I wouldn't mind if other people started using the code, but I've been burnt too many times during attempts to push my OSS projects over the years that I just don't want to put any actual effort into attracting people anymore.
If it having 1.7M lines of code indicates anything it is that creating a "simple app that provides simple functionality over a network well" is hard.
Look at the garbage generated by Google - text messages and images on Google hangout are not guaranteed to arrive in order ( or even be sent in order ) unless you are on a great network.
FB messenger exists and works on crappy connections. Google hangout exists and messes up on crappy connections. Telegram exists and messes up on crappy connections ( see the issues it has in Africa ). Signal exists and does not work well even on non-crappy connections. Whatsapp exists and sort of works on crappy connections. Matrix exists and sucks on crappy connections.
What we need is something that exists and works on crappy connections that has less LOC than FB messenger to demonstrate that it is possible to handle crappy connections and modern messaging in much fewer lines of code.
That's why your comment is nonsensical, if not downright contradictory noise.
> Ordering messages and providing reliability does not require 1.7M lines of code.
This has been demonstrated ad nauseum a priori with loads of toy chat apps that reconnect and retain state. It's not really a challenge. EVERYTHING else crammed into it, should be the reasoning for 1.7M lines of code, but as they demonstrated TO THEMSELVES AND EVERYONE, it wasn't (360k now)
Yes and they figured out what was needed after first writing a 1.7M line of code thing that worked. That's the process of reiteration and a process of initial LOC bloat before LOC stripping.
Without that process by thinking it has all been demonstrated ad nauseum a priori with loads of toy chat apps that reconnect and retain state you end up with deciding that if Friday was Feb 28th then Monday is certainly March 3rd because date/time handling is simple and end up being Robinhood, down for a day.
Does it? I routinely use Telegram, Line and Messenger. Messenger is without contest the one that gives the most trouble.
I've been using chat apps at least since ICQ (1996) and sometimes I feel the user experience has declined.
That being said, I'm sure that a lot of progress have been made and that there's a lot going on behind Messenger in terms of security and scalability. I'd be curious to know what's hidden between the millions LOC.
You never used ICQ over a spotty 3G connection, though, did you?
$ tc qdisc change dev eth0 root netem loss 0.3% 25%
$ tc qdisc change dev eth0 root netem delay 100ms 20ms distribution normal
Speaking of garbage, looks like most of the 1.7m LOC on Messenger was actually garbage, now that they’ve literally thrown most of away.
Also known as:
You can do dropbox with FTP.
If Friday is Feb 28th, then Monday is March 3rd. There's no need for those bloated date libraries.
> Speaking of garbage, looks like most of the 1.7m LOC on Messenger was actually garbage, now that they’ve literally thrown most of away.
Certainly. It worked before the change. So it is given that 1.7m of "garbage" provided a working product.
How many lines of code it took Google to make Hangout not get confused by image uploads on crappy connections? Unknown, because it is still confused.
How many lines of code it took Signal not to delay messages when a recipient is on a crappy connection? Unknown, because it still does that.
That's a joke, right? On a crappy internet connection (e.g. using O2 as carrier on the train cologne-hamburg you get ~12s ping, 30kbps, >90% packets dropped, via edge) most chat apps stop working, but Facebook Messenger is (except for Riot.im and Discord) probably the one that works worst.
Telegram works better, even IRC via Quassel/Quasseldroid works significantly better, and especially WhatsApp works so well you don't even notice the connection issues.
Facebook Messenger doesn't even properly start on such crappy connections.
> We also used quite a few of the OS libraries, including the JSON processing library, rather than building and storing our own libraries in the codebase.
Messenger may not have "HN clout" but the devs who work on it are having some of the greatest software impact in the world.
To be fair, this is part of the problem ^
In fact people who like stories and camera effects likely way outnumber the people that don't.
Summarizing it as a chat app is probably the first miscommunication here.
The number of people that use camera effects as anything other than a gimmick is even smaller.
Maybe a different age group?
I think that most of people of HN have friend groups which are wildly unrepresentative of world population at large. We're talking not even an average US or European citizen here - we're talking 1+ billion. People from developing countries who very often access internet only from a smartphone (they don't own a computer) and who almost don't visit plain http websites outside of the few social apps.
It's a completely different world out there.
Some of the problems outlined, like having 40 different views of contacts, with unique code, seem like fairly clear organizational problems.
I wonder if they could make apps more modular, and only load the binaries for e.g. payments on demand. I know Apple offers a system that does that for content chunks (e.g. levels and assets in games), and in a web-based application (e.g. react native) they could use bundling to remove it from the initial download.
I am not really defending it, but number of code lines alone is not a good way to evaluate code.
Having a GUI doesn't justify 1 million+ LOCs, but just for a sanity check, pgAdmin 4 has ~260k LOCs, without counting its dependencies.
Spotify had about 500k, lines of code in 2015, (not including external libraries).
We did make it in a feature based architecture, where there was a main container app (and a major core playback library) and for every major feature they had their own sub-project.... and there were 10+ teams contributing into it.
There were about 13 features at the beginning, (think, playlists, radio, shared ui libraries, etc...).
Each feature was about 10-20k lines of code in total. So, about 2-3 features equal a small standalone iOS app.
When I left, there were more than 80 features in the app. Some were essential, many were a/b tests, and some were probably just legacy/waiting to be removed.
80 features * 15k lines of code = 1.2mil loc, easily, and they are the equivalent of shipping 20 small apps standalone....
What are these features and why is just a stupid music app so large? : video, podcasts/shows, genius, etc.. etc.. etc.... To users just looks like another tab but almost every one of these major features could be its own standalone app for size and complexity.
I currently work on a project that is much larger than I would expect it to be, both in terms of codebase and people, and while I don't know who to jank, I also don't understand why it has to be so big.
And then your bathroom. And then convert the garage. And then add an addition. And in 10 years you have this cobbled-together monstrosity
Now your entire house is built with those screws, but they’re not produced anymore so you’ve set up a small smelter next to your home where you’re making them yourself.
Unfortunately the quality is a good bit worse, so parts of your house start degrading faster and faster, but meanwhile you’re still building all new parts with your shitty screws.
Nobody wants to replace the screws because “It’s a proven method”, ignoring all signs to the contrary.
All new parts of the house are only as good as the worst part, the screws, and the moment anyone steps on the wrong plank the whole room comes crashing down. But nobody worries, next time they’ll just use two screws.
Because nobody ever got promoted for not hiring people or for not writing code.
The problem is 10x as bad when you're an "IT company", i.e., writing code is your primary product.
It can oftentimes be worthwhile to have individual teams each fully "own" a piece of the product, and that also comes with the autonomy to choose to duplicate stuff that other teams might have already built rather than working with them and tweaking it to match their own specific new requirement.
This significantly reduces communication & coordination overhead for delivering new features, at the expense of consistency and code reuse, which can often be a worthwhile tradeoff at scale.
So, each team runs its own servers, for the features it is developing/maintaining, and most of the api goes throw a common gateway, etc.. etc... don't want to give out more than this.
Some of the simple front end web pages (i.e. marketing websites, analytics) might even be on php/python/ruby. Some teams wanted to use javascript (Node), but those are not the critical main app, as java is pretty much required to be used for anything that affects the main app.
Also, if you are doing machine learning, then whatever it gets the job done (usually Python).
btw, I left few years ago, so things might have changed.... I know at least one team that wanted to use GO for some of their services....
I can't shake the feeling that the dominating factor in growth of LOC isn't the number of features per se, but the number of features worked on in parallel by separate teams.
Most people (including myself) will probably never see a >500K loc application or a project with >100 developers working on it. Biggest ones I've worked with had around 30-50 odd developers?
No.
> incompetence
Yes.
> or self interest
Yes.
That said, I’m happy with React.
True though, if you want more features you need more code. Sure, loads of people on HN could build a 'chat app' in a lot less but if they kept it going for ten years (same age as our product) and kept on adding new features the lines would start to add up...
> [...] We accomplished this by using the native OS wherever possible [...]
I honestly think it is. Compliers, DBMS, a library such as NumPy— they're all complex, but Messenger, even considering the features removed, is a very full-featured application. We're not talking IRC here. It has activity status, payments, stories, photo filters, audio messages, location sharing, video-calling, etc. It's actually pretty impressive that that fits in 360K LoC IMO.
I'm really not a fan of this virtue shaming that people often tend to do as developers, where if you're not working to solve world peace then you're a waste of a developer.
Sometimes people just like getting paid? Sometimes people just don't really care, they just want a job or want to write code?
NumPy is a library. The other three are compilers. Why are you comparing these with a GUI-based chat application?
> We accomplished this by using the native OS wherever possible
Does this mean this application does not use React Native?
It would seem that way.
Mind = blown!
One can only hope that Slack and other Electron apps reach this point eventually...
Given the headaches caused by Apple finally barring UIWebView (the basis of frameworks like Cordova) from new App Store submissions beginning in April, I'm not surprised FB would do this. There is plenty of rumor-mongering that Apple is going to make it increasingly difficult for apps based on hybrid frameworks through the submission process.
Apple legitimately doesn't seem to care about hybrid frameworks, they've been there since day one. Any time it's seemed like it got difficult Apple has backed down. The lack of JIT in WKWebView is probably the only thing that you could stretch here.
Cordova (and derivatives, such as Ionic) have had WKWebView available for a long time now, I think since iOS 9.
The Cordova team added a compile-time flag to force WKWebView in 5.1.0 (out since last November), and will be removing UIWebView altogether in 6.0.
Fortunately, nothing quite takes the wind out of a platform's sails like watching the creator back away from the platform.
-step 2 : rewrite it in native so you greatly improve metrics while reducing code base size.
Seems like a good way to achieve great improvements in your product
There is also an inherent cost to rewriting all your codebase at once that should not be under estimated.
For something so big and integrated as messaging, I am not sure that this was a good match.
Admittedly turning messaging into a do all à la WeChat was very probably part of the initial plan.
That doesn't mean React Native is not a good fit for any kind of application - for many kinds of apps, start up time and app size are a much smaller issue - but seems Facebook made the right decision for an app like Messenger which primarily needs to start fast and drain as few resources as possible.
It does not mean it runs JS for anything critical, and very likely it does not.
My car entertainment system (GUI) run JS, and I'm fine with it. I would not if it would run on the emergency breaking system...
I guess many native developers feel negatively towards it because a lot of companies will be asking "why aren't we using React Native instead of building two separate apps?" which I can imagine feeling somewhat threatening towards your area of specialisation
If you want to go deeper or have more specialised requirements, then you might need a deeper understanding of the native platforms though.
Beyond "Hello World" all of them required me to get my hands dirty, at which point I was better off just doing C++ with native views, standard out of the box tooling available across all SDKs.
Nowadays I rather go for PWAs, or plain mobile Web, unless it is something very special that requires OS APIs or hardware access.
There seems to be a much more obvious explanation as to why people are assuming it used to be React Native. The article says:
> > Compared with the previous iOS version, this new Messenger is [mentions improvements].
> > We accomplished this by using the native OS wherever possible […]
That seems to imply that it wasn't native before and they got the gains by switching to native. And since Facebook are well-known as the creators and biggest users of React Native, it's not a big leap to make the assumption that that's where they were coming from.
[Also, in case there's any confusion please note the distinction between "native", meaning it is compiled to native code; and "React Native", which is a cross-platform framework that is not compiled to native code. "React Native" is a horrible name for something that is not native.]
This paragraph just restored my faith in developer sanity.
Every program can be (memory)optimized by at least one byte and has at least one bug in it.
It therefore follows that every program can be reduced to a single byte that doesn't work.
I wish the article was a little more fleshed out, I'd be interested in finding out more about how and why they went down this path
By moving all state into SQLite they have implemented the "store state in relations" layer, and then written (as far as possible) functional code on top.
I've been thinking about how to make my software more like this so it's interesting to see Facebook are thinking along the same lines.
There's a lot of really good ideas in Out of the Tar Pit and I'd really recommend reading it.
Size, not necessarily, but yes it was in the case (the person was looking at why the binary itself, not any of the assets was above 100 meg).
> or complexity of an app
I'm not sure how 18,000 classes isn't a guaranteed extremely complex app.
[1] https://reasonml.github.io/blog/2017/09/08/messenger-50-reas...
I'm also aware of how iMessages and SMS are used in the US, yet this is very US-specific, and I don't believe any other country uses SMS.
For Chinese people, WeChat is the standard, and what pretty much anyone uses.
In Argentina and the Netherlands (and I believe much of the EU), WhatsApp is ubiquitous.
Yet, I don't know a single person who uses either SMS or Messenger, or iMessages.
So what truly fascinates me, is how different companies put so much work into developing such sophisticated messaging apps, but they're all reinventing the wheel and each consumed in different regions -- I think nobody would have guesses years ago that we'd have so many different apps used in different countries, and not at all inter-operating with each other.
My favorite is using Apple Messages since I can send/receive from any of my Apple devices. I use macOS for other reasons, but being able to send a text from my computer is awesome.
My fascination is that we've very equivalent things (eg: WhatsApp and Telegram aren't that far apart), duplicated even in the same generation.
I did use ICQ, MSN, IRC then WhatsApp, but they didn't belong to the same periods.
So it's entirely conceivable that in some places, someone with a limited income only has SMS bundled as a free service on their plan, with data either not enabled, or expensive pay-as-you-go. Therefore in that demographic SMS messaging will be the primary method.
In the UK, almost all tariffs come with a bundle of data, so SMS has all but died out (except for automated parcel delivery updates, and 2FA authentication et al.).
Data was always way cheaper in Argentina, which is why WhatsApp quickly killed SMS, to the point where it's never really used anymore.
it's a freakin chat app, and it's freaking facebook ! come on, people were doing full rewrites in entirely different languages of software insanely more complex than fb messenger back in the 90s and with near-zero tooling, eg from cobol to pascal to C or similar horrors.
Meh. It displays text, pictures, and videos. That's about it. This is just Messenger we're talking about here, not the full-blown Facebook app. The only state to keep track of is "Have you received message UUID?".
> And they also had their own UI framework.
This is where I suspect the bloat occurred.
Writing your own JSON parser and so on doesn't add up to any 1.7 million lines of code.
Image editors are the simplest — they just let you paint with colored pixels.
For android you have to ship basically ffmpeg for correct mp4 encoding, on all platforms you have to carry some library on top of classical ui framework to make it just not suck at performance, you have to get your own json parser, yes, since built-in one will be 10x slower, you may be want to carry even low level QUIC library for networking and websockets, etc, etc.
All this also not increasing complexity since built in one stuff makes your code much harder to read/write.
hopefully they aren't counting ffmpeg in their LOC count, right ? else they may as well count the lines of code of the C and C++ standard library, V8, etc - this would be a very useless metric
Good work Facebook!
I have to go years back to think of someone who told me they just couldn't take the bad UX of the clients so they deleted it.
Like I've heard from peace corps folk about how a large part of the third world thinks of messenger as "the internet", which is absolutely terrifying.
Spotify lite, for example, has all the features, a better (!) UI (with material ripples) and everything loads about twice as fast.
Every app should be a lite app. Devs, please start giving a shit about performance.
re app size : Google has already published stats urging apps to optimize their size as much as possible since it has direct effects on acquisition.
I suspect that long download times lead to more errors + more wait time before you can use the app = more likely that the user will abandon its current task and
Same with slow app or ui patterns that don't fit with the platform = easier to lose users in the first crucial minutes.
I didn’t pull the trigger on canceling the account as from time to time I find news and event from pages that I follow, but there’s no way I’m installing an app that uses so many shady tactics.
Would be interesting to see how many seemingly harmless apps start to break, I would expect similar results to surfing the web with a JS blocker.
Isn't the screen off in nearly all of the legitimate uses of a microphone, ie. "I'm replicating the ux of a phone call in my chat app"?
Aren't those using Electron?
tl;dr web clients are probably more performant than their electron counterpart
It may have been slow and bulky before that, but that's when I started to notice.
It's unreal, our industry has lost its mind.
I'm not quite sure why I need two apps on my phone for communications through one company when 12 years ago I had one app on my desktop for 5 or 6 different company's messaging services.
That said I find it a bit heart warming that all the major apps are now between 1/2 and 1/4 the size of what they were just a few years ago. I didn't expect the tech companies to ever realize the problem with insane sizes. Is a similar debloating happening to Electron apps?
It's pretty sad that a Messaging app can't be run on a perfectly fine nine year old device. Bring back IRC :)
12 buttons. And that’s apparently stripped down. Yikes.
FB named a open source project the same name as a company.
I really wonder if in their meetings they sold the speed and lighter weight as a bonus for the user or for the collection of data on the user. This must reduce their overhead and be kinder on their servers.
[1] to access messenger because fb stubbornly thinks I want to install their crap/spyware on my phone
Yikes. Okay, I know I'm maybe being a bit unfair here, but I'm sorry to say that this is one of the weakest engineering blog posts I've ever read. This seems to be a massive "let's end the craziness and finally pay our tech debt" effort. Good for them, but not something that excites me about working at Facebook.
Yes you could. Note however 'simple chat system'.
To build a global messaging app used by hundreds of millions of people is a whole different ball game.
By comparison, before acquisition by Facebook, Whatsapp had a team of ~50 people, and that was considered lean. Based on that, it would seem you're about an order of magnitude off.
The issue here is only partly inherent complexity of the feature set. The real problem Facebook had is one I don't see discussed in either the blog post or this thread, which was organizational. The way they structured their mobile apps was as a huge shared codebase with many disparate teams just checking code into it whenever they wanted. To the extent there was any architectural planning at all it came from some small shared library teams who were in no position, managerially, to enforce their will on the others. The blog post makes clear that a huge portion of their problem was multiple teams inventing their own ways of doing things and creating stuff that want necessary, along with a lot of pointless duplication because there was so little coordination and thus so little code reuse.
The new app doesn't sound like anything special design wise, and that's the point: for the first time they've done things more conventionally. They now have someone who can dictate to every feature team "thou shalt use sqlite with these schemas" and other rules.
At first look, I thought it is 36k, that would be truly great.
>One of our main goals was to minimize code complexity and eliminate redundancies. ... To build this unified architecture, we established four principles: Use the OS, reuse the UI, leverage the SQLite database, and push to the server.
>We accomplished this by using the native OS wherever possible, reusing the UI with dynamic templates powered by SQLite, using SQLite as a universal system, and building a server broker to operate as a universal gateway between Messenger and its server features.
>... the existing OS often does much of what’s needed. Actions like rendering, transcoding, threading, and logging [and JSON processing] can all be handled by the OS.
>To simplify and remove redundancies, we constrained the design to force the reuse of the same [UI] structure for different [UI] views. So we needed only a few categories of basic [UI] views, and those could be driven by different SQLite tables.
>Now, ... All the caching, filtering, transactions, and queries are all done in SQLite. The UI merely reflects the tables in the database.
>We developed a single integrated schema for all features. We extended SQLite with the capability of stored procedures, allowing Messenger feature developers to write portable, database-oriented business logic, and finally, we built a platform (MSYS) to orchestrate all access to the database, including queued changes, deferred or retriable tasks, and for data sync support.
>MSYS is a cross-platform library built in C that operates all the primitives we need. ... With MSYS, we have a global view. We’re able to prioritize workloads. Say the task to load your message list should be a higher priority than the task to update whether somebody read a message in a thread from a few days ago; we can move the priority task up in the queue.
>With MSYS, it’s easier to track performance, spot regressions, and fix bugs across all these features at once. In addition, we made this important part of the system exceptionally robust by investing in automated tests, resulting in a (very rare in the industry) 100 percent line code coverage of MSYS logic.
>For anything that doesn’t fit into one of the categories above, we push it to the server instead. We had to build new server infrastructure to support the presence of MSYS’s single integrated data and sync layer on the client.
>Coordinating logic between client and server is very complex and can be error-prone — even more so as the number of features grows. ...
>Similar to MSYS on the client, we built a server broker to support all these scenarios while the actual server back-end infrastructure supports the features.
>[To minimize code-base growth] We also built a system that allows us to understand how much binary weight each feature is bringing in. We hold engineers accountable for hitting their budgets as part of feature acceptance criteria. Completing features on time is important, but hitting quality targets (including but not limited to binary size budgets) is even more important.
Windows Messenger was about 900k. Sure, it only did text, but Skype wasn't that big either. I'd accept 20mb for stickers, givs and stuff but 80? And 300k lines of code? For a messaging app? It just seems... unreasonable to me.