A Closer Look at Android RunTime (ART) in Android L | Hacker News Reader

A Closer Look at Android RunTime (ART) in Android L

anandtech.com

95 pointscpeterso54 comments

It's kind of funny to hear AOT compilation being heralded as the next step after JIT that will bring better performance because you only have to compile once, since we spend so much time hearing exactly the opposite, especially in the context of Javascript.

(Notwithstanding that, as other comments have pointed out, JIT vs AOT is at best a gross oversimplification of the difference between Dalvik and ART.)

Apps and web pages are different use cases. An app you expect to spend a moment "installing" and are likely to use regularly, so AOT compiling is useful. Web pages on the other hand are often visited briefly and a few seconds delay can make a huge difference difference, and often contain lots of script executed only once or not at all - so AOT compilation would actually hurt performance.

Not really true with all the HTML5 apps based on Phonegap and WebView.

Not really, it is deja vu for anyone that has been around in this industry for long time.

The idea of universal VMs go back to UNCOL project (1958).

A few mainframes made use of VMs with some kind of AOT compilation at installation time. The most notorious of it OS/400, still being used.

The first sucessful JIT compilers were designed as part of Smalltalk systems, namely SELF, which JavaScript is a descent of. So of course you hear a lot about it in the context of JavaScript.

Where JIT compilers bring a major win is when compiling dynamic languages.

With strong typed languages, the type of optimization algorithms you can make use of usually win over the limited options JIT compilers have available due to responsiveness limitations.

In the early 80s there was a fad of VM craziness around P-Code virtual machines, which eventually died out when everyone went back to proper compilers.

So this going back to AOT compilation trend, is just the IT circle going around one more time.

Not in the Android performance picture so far: language runtimes (e.g., jruby and groovy) that dynamically generate bytecode. A JIT can optimize it (and for jruby and grovvy on the jvm, it typically does).

That hasn't been an issue on Android so far because those language runtimes would have had to be retooled to generate Dalvik bytecode, and none of them were. (Ruboto, jruby on Android, runs in pure interpreted mode, with a significant performance hit.) But I believe that with Dalvik, it was technically possible. Does ART just close this possibility off?

I do not think dynamic DEX generation and loading was ever possible with Dalvik. So ART makes this no more or less difficult. An Android runtime environment, by definition, includes all the code to interpret DEX, even if it isn't made to be capable of using dynamically generated DEX.

But you have other problems to solve: How are you going to generate DEX? Generate java bytecode and bring dx to the runtime? Make jruby emit DEX directly?

EDIT: As per the comment below, that is how it's done. You generate Java bytecode and run dx on it.

It is entirely possible to dynamically generate and load DEX files in Dalvik, and it should be possible in ART.

To the best of my knowledge, there are too projects for generating Dex bytecode directly:

1. dexmaker https://code.google.com/p/dexmaker/ 2. ASMDEX http://asm.ow2.org/asmdex-index.html

A different approach, which I used for Clojure/Android, is to generate JVM bytecode and then use a bundled dx tool to generate Dalvik bytecode from that.

AOT + JIT is heralded more than AOT itself. This is because you can do both. Literally nothing stops you from doing both AOT and JIT'ing whatever you want (C/C++/Java/etc).

I wonder if it would make sense to rewrite some native components of Android in Java now. Presumably, whole-program optimization means that the ART compiler is more effective if more of the code is in Java (or technically, Dex bytecode) rather than having opaque native parts with JNI overhead.

A lot of Android is written in Java already. What would be interesting, however, is Android transitioning to a message passing micro-kernel (instead of Linux) and writing as much as possible of the rest of the kernel in Java.

That is already the case. When you look at the NDK, you can only have access to game related API (audio, graphics and sensors) almost everything else involves JNI.

Actually quite a bit of the view code is native still - start reading the source for View etc and you hit a level where it's mostly calls out to native libraries. e.g https://github.com/android/platform_frameworks_base/blob/mas...

Well, graphics code eventually needs to speak to the graphics card.

I imagine outside graphics, audio and hardware integration APIs, there is very little done at C level.

"This also means that Android is finally able to compete with iOS in terms of application fluidity and performance, a big win for the consumer."

Does anyone actually have any real comparisons between Android and iOS? Was there an acknowledged 2x performance benefit for iOS on the same hardware? For games and Ui, for example, wouldn't the GPU be more important?

Interesting that the new on-device output is a Linux ELF binary. I suppose this opens the door for a more flexible cross-compiler system that could in the future better support languages that are less Java-like. E.g. - something like Swift that doesn't use a GC at all, for example. Or at least an option to develop in Go?

It will be interesting to see what sort of extensions Google adds to the dex format in the next few years. Hopefully, there will be some efficient lower level language bindings for the UI calls, perhaps?

This architecture is insane from an overall system point of view.

Why ship DEX proprietary bytecode to be compiled by the same compiler in the same way on hundreds of millions of devices to produce the same output?

It would seem far more logical to compile all the versions needed, one for each device architecture, just once. As other aspects of the apk need to be customised for each device class already, there is not really new overhead.

Going native is a great idea, doing it this way is plain bonkers.

Backwards compatibility is extremely important--existing APKs will still work with ART. They also avoid separate binaries for each supported architecture, MIPS, x86, and ARM.

Plus this system also allows apps to take advantage of updates to the compiler without requiring that developers recompile and resubmit. Imagine in 2 years a new ARM version drastically increases performance by using some clever new instructions. If you have pure native code every developer has to recompile their apps to take advantage of the new architecture--with ART, Google handles the update, and every app on the appstore benefits automatically.

System-on-chip vendors could also tune the compiler, just as some do now with the Dalvik VM. For example, this AnandTech article mentions that the Moto X has a Qualcomm-modified version of Dalvik: http://www.anandtech.com/show/7517/google-nexus-5-review/2

It's how Java works. It is, indeed, how HTML, CSS, JavaScript, and many, many other technologies work (millions of us are all using the same Chrome browser that's doing exactly the same work to make sense of this markup, build and transform DOM trees, compile and JIT the scripts, etc. Why don't sites just serve up Chrome native code?).

It isn't really "insane". There are benefits and detriments. Not least that the same APK can run on many Android versions, across many architectures (x86, AMD64, ARM, ARMv8, MIPS), with many different bits of system hardware.

Further the whole "ART goes native" thing is largely wrong -- Dalvik already compiled to native, storing the JITted code for subsequent uses. ART is just a better performing, more considered rewrite.

Well, what makes it kind of strange (IMHO) is that it's intermixed with native code via the NDK (which is heavily used by games, an enormous segment of mobile apps). So, in practice, the platform independence of Android isn't the same as that of the Web.

is that it's intermixed with native code via the NDK

Idiomatic use of the NDK is that you always provide Java surrogates for NDK functionality, but of course we know that isn't often the case. Instead developers can and do cross compile into a single APK. That is exactly what I do -- for my NDK components I build to x86 / ARM / MIPS (the last one I can never really justify) and it all goes into one APK. Alternately, and as a middle-grounds, I can make separate APKs for the platforms.

> Instead developers can and do cross compile into a single APK.

So you lose the Web's ability to be retargeted to arbitrary architectures. You can only be retargeted to architectures that you compiled for. It's the exact same situation as in native code (because it is native code), and has very little in common with the Web's model (which can be retargeted to any architecture, even ones that Web authors didn't "cross-compile to").

By way of example, Nintendo could ship a Wii Browser (PowerPC), because JavaScript is platform independent. But Nintendo couldn't ship a Wii Android app runner and have it play games, because nobody compiles games for PowerPC.

It's the exact same situation as in native code

Just to be clear, Android applications can be, and often are, 100% DEX (the vast majority of productivity, video, etc). Because, you hold, there are exception applications that break this model, the whole model is broken? It's saying that the web is broken because ActiveX is used somewhere.

That is horribly specious. It is a dead-end argument by exception.

But let's look at those uncommon apps that use some native code.

a) The majority of the code in most cases will still be in DEX. It will still benefit from device specific compilation.

b) The majority of the NDK code will or should have Java equivalents. When the native code can't be used, DEX code will be used. Again, this is idiomatic.

c) And when those two aren't true, on x86 for instance if you only include ARM, it simply transcodes the ARM native code to x86 native code. It obviously isn't optimized, but it's a pretty decent exception case.

In every case you benefit from the device specific compilation. So what doesn't hold again?

> But let's look at those uncommon apps that use some native code.

Games are not exactly uncommon apps: https://play.google.com/store/apps/collection/topgrossing?hl...

> a) The majority of the code in most cases will still be in DEX. It will still benefit from device specific compilation.

Do games really bother to ship Java equivalents? I would be really surprised if they did.

> c) And when those two aren't true, on x86 for instance if you only include ARM, it simply transcodes the ARM native code to x86 native code. It obviously isn't optimized, but it's a pretty decent exception case.

Yes, you can use emulators for native code too, but that's hardly a good user experience unless the games are old.

It's a solution for a different time: a time when the code was distributed, where it could theoretically run anywhere, and be installed from anywhere. In that world, distributing one "intermediate-VM blob" which was then compiled to native on the client made a lot of sense.

Today, in the world of centralized app stores, there's an argument to be made for directly distributing compiled binaries as an optimization. It still makes sense to use an intermediate format as it allows the most compatibility (i.e. new hardware), but the app store itself could distribute binaries for known/supported handsets as an optimization (it wouldn't surprise me if it didn't already do this).

By what possible perspective is that from "a different time"? Because Microsoft decided to distribute compiled .NET apps to Windows Phone?

Is there some great advantage that Windows Phone gains from this? It certainly doesn't seem so.

It's quite remarkable to read people talking about how insane this is, how it's a solution for a different time, etc, when it is working remarkably well (number of people complaining about on phone AOT/JIT time -- about 0. Number of people sure it must be some great issue -- too many). It seems to be a solution for this time.

> By what possible perspective is that from > "a different time"? Because Microsoft decided > to distribute compiled .NET apps to Windows Phone?

When Dalvik would have been designed, there wouldn't have been an App Store. The model for mobile development wasn't clear like it is today.

I agree that it's a great solution: the AOT cost is, in practice, tiny. The flexibility (and, I expect, reliability) provided is with the cost.

No more insane than all the linux distributions which compile programs from source.

It allows new architectures to be introduced without requiring that the developer compile them into the APK - and with all the architectures Android supports, that could become large anyway. Compiling on the device allows the manufacturer of that device to ship a compiler optimized for it.

It is possible though that Google could do some compilation steps on their servers, and deliver those through the play store - but the main advantage would be a decrease in installation time.

Much saner than source-based distros, in fact, since in that case, the compilation step is slowed down by the notoriously slow compile times of C and C++, whereas the ART compiler is dealing with Dex bytecode.

I haven't yet studied ART in detail, but I think one reason to compile at installation time is so the compiler can analyze the whole program, including the libraries provided by the OS. If just the application code were AOT-compiled to native code as part of packaging, the compiler would miss out on a lot of optimizations that are available to a whole-program-optimizing compiler. And note that JIT compilers basically do whole-program optimizations, such as devirtualizing and inlining methods, eliminating null pointer and exception checks, and others. So a unit-at-a-time AOT compiler would be worse than a good JIT compiler. But a whole-program-optimizing AOT compilation step at packaging time would require every application to statically link all the OS libraries. Compiling to native code on the device at installation time seems like a good solution to me.

Whole program optimisation would require a recompile at every system update wouldn't it? With lots of apps that would be pretty costly.

Every Android system update already has stage where it prepares every app.

Every time there is a system update, it recompiles all the apps.

I still have a Nexus S. With early versions of ART in Android 4.4, booting the system and precompiling everything took more than half an hour! Current hardware is much faster, though.

This is also what the .NET CLR does (ngen.exe), and it's maddening.

On the other hand, "managed" code is truly safe from the whims of untrusted developers--if your bytecode format has no way to express some unsafe/privileged native instruction, then the developer can't do that thing on your device, period.

It'd certainly be better if there was a middle step where, say, the Play Store took your Dex-APK and generated a native APK from it. But if devices can't do that step themselves, then sideloading stops working.

True, Although Android's security sandbox enforced using the kernel and user accounts, not JVM limitations - you can ship native code using the NDK and run any instructions you like, but you are still restricted by the permissions given to the application.

That means that either the app-provider, or the app store, has to compile a separate version of every architecture.

Which then means that if you introduce a new architecture then you're reliant on one of those two people re-compiling for it.

This way they can introduce a new architecture, or indeed an improvement to the compiler, and you don't need to do anything other than ship it out to the phone.

You are assuming that it is more desirable to optimize for CPU than it is to optimize for bandwidth. The significant size difference between byte code and compiled code has to be downloaded hundreds of millions of times too and bandwidth is way more expensive than CPU cycles.

It took less time to recompile 200+ apps on my phone when I switched from dalvik to ART than it does to download 2 or 3 more new apps. CPUs are fast, even on mobile.

Also the largest part of the packages are probably not code, but assets which presumably don't get compiled further ;-)

That's a great point. It would be interesting to see a list of largest mobile apps by code size instead of just code + resources.

Worth noting is that the mobile CPU is only doing a fraction of the job, most of the compiling (from source to bytecode) was performed beforehand, and only bytecode to native remains to be done locally.

It's only half bonkers, and not the bad half:

It's not just architecture that determines what kind of code you want your compiler to emit.

It is possible, even likely, that the compiler's strategy for a TV plugged into AC power is going to be very different from the compilation for a mobile device with a small battery. And within the worlds of small battery mobile and AC-power or big-battery (e.g. car) devices there will be parameters like heap size, CPU speed, etc. that will vary how the compiler emits code for that device.

Expecting a bytecode designed to be interpreted to be the ideal intermediate representation is a little "bonkers" but it may be good enough that it's not a serious problem, either.

DEX is not a bytecode as such (more of a halfwordcode if you will) and it was designed with the idea of being both efficiently interpretable (look at lua's snappy performance with their wordcode format) but also a more reasonable IR for a JIT or AOT backend.

> also a more reasonable IR for a JIT or AOT backend.

It might have turned out that way but I see no evidence that DEX was designed that way from the start. The claims about DEX performance when Android first came out were that it was twice as fast as interpreted Java bytecode and half the size.

The Dalvik JIT compiler is a nifty design, but there is no evidence DEX was designed to make it easier or better. The JIT compiler was added years after Android shipped.

One can make the general claim that there is no big impediment to producing optimal compiled code from any reasonable bytecode. But I have yet to see anything indicating DEX was designed for compilation or confers any advantages on any compiler design.

One side effect of supporting .dex as the intermediate form is the entire existing infrastructure (including side-loaded apps, third party app stores, etc, etc) continues to Just Work(tm).

It also allows for devices running different architectures (including those not currently supported) or with workarounds for weird architectural bugs to work without having to coordinate that with a central authority.

DEX remains the canonical intermediate and wire form of "native" (meaning compiled-from-java-source) Android apps.

CPUs often come with many variants and optional features, and compiling on the target device allows the compiler to identify the feature set of the given CPU and generate optimal code. For example x86 is not "just" x86 - it can have a number of extensions, like MMX, 3DNow!, SSE, SSE2, SSE3, SSE4, AVX, CMPXCHG16B, AES, and so on. Some of those even have sub-types themselves (SSE4.1, SSE4.2, etc), and there are flags to indicate the presence of extra single instructions. That's only the 32-bit type; there's also the 64-bit type. Other architectures like ARM have their own extensions like NEON. So in fact the APKs are likely not all the same, and will vary depending on the specific features available with the CPU model.

That is what Microsoft has done with Windows Phone 8.

AOT compilation is done at the store. The OS only does the linking at installation time.

"Why ship DEX proprietary bytecode to be compiled by the same compiler in the same way on hundreds of millions of devices to produce the same output? "

Because it isn't, or maybe it won't be?

Think very hard about this. You use your device in different ways than the next guy. If someone did profile collection on the device, and then fed that back into the compiler, you'd get a different executable than the other guy.

Also, building and shipping N hundred binaries from the play store seems like a recipe for disaster.

In fact, i'd strongly argue doing it any other way is plain bonkers.

Another advantage is that you will not need to recompile your app for 64 bits architectures or whatever new chip arrives in the future. It is already compatible. You also don't have to worry about the SOC when sending an apk, the system will make sure to compile it.

This seems familiar:

https://news.ycombinator.com/item?id=7738223

ART is a good news for Android users, but it's really optimize the performance? I watched someone use Android L to run some popular social app(like WeChat), you can't open it normally. Some big bug sure be fixed.

I am testing the L build on a Nexus 5. Our app has a feed full of very rich views. We already implement a lot of optimizations (not all of those that are possible though, namely flat custom views, simply because we don't have the engineering ressources to do it right now) but it scrolling is not perfectly smooth on 4.4. We lose a couple of frames each second when scrolling rapidly (interestingly, in addition to its native code, iOS also limits scrolling speed in order to deliver a smooth experience). This is no longer the case with Android L. Without changing anything in our code, this feed is now smooth as hell.

About the app you mentioned, L is right now a preview build, there are indeed a lot of bugs everywhere. It is not really surprising since we are months away from the release. It does not have anything to do with perfs though.

I think you are right. I'm not a pure Android developer, just do some works with Android. Your test give us a good news, we happy to see Android more smooth.

Reply on news.ycombinator.com