Improving Facebook's Performance on Android with FlatBuffers | Hacker News Reader

Improving Facebook's Performance on Android with FlatBuffers

code.facebook.com

129 pointsnumo1640 comments

The FlatBuffers repo on github continues a couple of "meta-trends" I've noticed in recent Google projects. https://github.com/google/flatbuffers

First, it uses CMake to build - for a long time Google projects had seemed pretty anti-CMake (for example using gyp, plain Makefiles or autotools) so it's nice to see them using CMake. IMO it's the best build tool, though all build tools generate various levels of hate :-)

Second it's another Google project that generates good developer docs from source code using doxygen and markdown. These docs look good on github directly (https://github.com/google/flatbuffers/tree/master/docs/sourc...) as they are markdown, and even better on the dedicated site where they have custom css.

If I were to write a C++ library, I'd definitely copy these 2 approaches.

Thanks! I guess it's because we're a game development group inside Google, who are more externally focused than most Google engineers. We wanted to ensure the library is attractive to outside developers, hence CMake and other choices (like not having any dependencies).

In short: They first used JSON, if I understood correctly. Then

"In last six months, we have transitioned most of Facebook on Android to use FlatBuffers as the storage format. Some performance improvement numbers include:

Story load time from disk cache is reduced from 35 ms to 4 ms per story.

Transient memory allocations are reduced by 75 percent.

Cold start time is improved by 10-15 percent.

We have reduced storage size by 15 percent."

This is indeed a very interesting encoding. One part of the optimization comes from the use of binary encoding and the other from using offset tables to optimize random access. Offset tables also allows to modify random data in the structure without having to reencode the whole information. This comes at the price of less compact data.

It looks like I could update my YABE encoding which is a straight JSON binary encoding.

They didn't need to mutate the JSON hierarchy. That's the big observation, the take-away.

So they took advantage of that fact by not inflating ("flattening") the data structure using a stream.

For those are are totally new to Flatbuffers, here is a good video by Colt from Google. It was primarily created for game developers but technically any app can use it - those who are not relying on a library that handles networking and has implementation for JSON.

He also mentioned Flatbuffer use in his recent at Android meetup in San Francisco and we debated it's benefits again. It has a learning curve but well worth it: https://www.youtube.com/watch?v=iQTxMkSJ1dQ

OK, so if I understand this correctly, a binary format where you can seek to arbitrary positions to read only the data you need to currently display is a huge performance win versus a big blob of text you have to read and fully parse. As someone who has had to uninstall FaceBook from my Android phone due to its poor performance, excuse me if I'm bewildered that they didn't realize this years ago (FlatBuffers isn't the first binary serialization format) and write it that way in the first place!

Facebook(the company) values putting any functioning software out there over putting optimized software out. This is intentional, but it leads to bad experiences for people like you.

There's a lot more to it than just being binary. The big deal is that it can be accessed in-place while still being portable and forwards/backwards compatible. For example, Protocol Buffers is binary too, but requires unpacking, causing lots of object allocation, etc.

I'm sure your billion person social app is killing it due to awesome optimizations baked in from day one. Nicely done.

I liked this writeup. I'm curious how much this improved battery life for a typical user.

FWIW, I help maintain FlatBuffers for Go and Python. I'm happy to answer any questions I can.

I didn't like the writeup, because I can't tell how the layout works from his bad description of it. The author makes that classic mistake of technical writing in which the writing assumes the reader already knows what they are talking about.

I ran across Cap'n Proto (https://capnproto.org/) a while back and at a glance it seems to do something similar. Is that true? Are you familiar with the tradeoffs between the two?

Relevant thread from the designers of Cap'n Proto and Flatbuffers from around the date of the Flatbuffers release: https://news.ycombinator.com/item?id=7901991

And from that thread, here's the Cap'n Proto author comparing Cap'n Proto, FlatBuffers, and SBE -- https://capnproto.org/news/2014-06-17-capnproto-flatbuffers-...

Not only can JSON be slow, SQLite has been shown be very problematic as well: http://0b4af6cdc2f0c5998459-c0245c5c937c5dedcca3f1764ecc9b2f...

The filesystem is a fantastic way to persist data for 80-90% of mobile applications.

> The filesystem is a fantastic way to persist data for 80-90% of mobile applications.

Well Sqlite provides an extensive query language. I don't think Android's file system does.

Does that make your application more responsive to the user?

They are missing the second major reason why flatbuffers is awesome, cache locality.

It's incredibly hard to layout objects in memory with Java but if you don't mind the lookup hit on bytebuffers flatbuffers is a great way to structure data in the patterns you access it.

All this stuff is pretty old-hat to game dev people bit it's nice to see mainstream dev start caring a bit more about performance.

Good point. Lookup in Java is a bit slower than in the language FlatBuffers was originally designed for (C++), but being able to bypass Java's object allocation may well make up for that.

In general, objects allocated in sequence will be consecutive in Eden. After GC moves stuff around, objects referred to by object X often follow object X itself. YMMV.

Nice to see Facebook acknowledge contributions from Google. It's a departure from typical not-invented-here mentality prevalent at large tech companies.

What does the iOS Facebook app uses? Couldn't be core data based on the fact that they claim they can not normalize the data ahead of time. The iOS app is nice and snappy, at least on my phone, though it does take some time to load.

How much binary size does it consume? Have an app with 100 objects with a few thousand fields total and you have several megabytes consumed by protobuf alone. With 3mb source files if you put it all in one proto file.

I'm a bit surprised they decoded json on the main thread...

me as well. My hypothesis is that they need it in order to to know how tall a list item needs to be. Still, it seems really suboptimal.

I'm curious is Facebook working on a version for Obj-C / Swift or do the iOS and Android clients now get back different types of results?

The article specifies Android

Funny how similar this is to just a simple filesystem.

That actually makes me wonder if storing things as such might be a great way to go.... like just reading 58283/photo.jpg or 58283/name.txt or whatever... of course you have you then pay the price on decoding whatever you get from the network into these files.

JSON parsing is a major problem on Android. When I also first ran into the 35ms parsing I thought for sure I was doing something wrong. Nope.. the latest phones can get down to 4-5ms JIT'd but still iOS is an order of magnitude or two faster which completely changes your architecture decisions.

Why this can't be integrated in to be fast & native I've never understood. It should have landed in Android a long time ago.

I'm curious (knowing next to nothing about iOS development) - what makes iOS orders of magnitude faster at JSON parsing?

Seems to be native versus JVM. My guess is it's a combo of string processing, many temp objects, etc with too many interpreted versus JIT'd code paths.

If Facebook really cares about performance so much, I wonder why they don't just implement that app engine native.

A bit surprising but I'm guessing it's the multi-architecture fat binary problem they don't want to solve? Until recently debugging native plus Java was very difficult, and most of the UI stuff is in Java not native.

As far as I'm aware Dropbox uses c++ between iOS and android shared for all the model/network bits so other biggies do it.

Why is this only for Android? Is that because of JIT compiler or is it an issue even on JVM and hence can we use flatbuffers even on web?

Flatbuffers targets C++ and Go[1], so it does not appear to require JIT. It would appear the necessary code is generated statically even for Java and C#, to take advantage of strong typing.

[1]: https://google.github.io/flatbuffers/

Makes sense but do people use it for conventional web. If there is such a great deal of efficiency why isn't there a great deal of adoption? Edit: Added some more text

Because JSON is the "default" (everybody knows it, everything has tooling for it) and most people don't care about efficiency on that level until they have to.

FlatBuffers isn't available natively for JS (yet), though you could use it through asm.js if you really wanted to.

So they're telling me that they have all this multi-gazillion-CPU-core Android devices, but they parse JSON on the main thread?

Reply on news.ycombinator.com