A Closer Look at Android RunTime in Android L | Hacker News Reader

A Closer Look at Android RunTime in Android L

anandtech.com

119 pointsrohan40494 comments

It's interesting that an asm.js Javascript application running in a web browser yields better performance on Android then a Java application running on Dalvik:

https://blog.mozilla.org/javascript/2013/08/01/staring-at-th...

So ART and AOT compilation is a step in the right direction, but I'm secretly hoping that Google will enable Portable Native Client 'natively' on Android outside the browser for proper C/C++ development. Distribution format is a frozen LLVM IR subset, final compilation to the CPU architecture happens on the client side, and performance is close enough to pre-compiled that the difference doesn't matter much.

I would just be happy if little things like SKIA and SQLLite were exposed in the NDK, instead of forcing us to bundle our own versions.

And having a gdb setup that works properly.

Here's a question - has Google made the move to ART for performance reasons or is it because it makes it possible to move to ... say.. Golang for Android ?

Let me explain - One of the big reasons that Google cannot switch over to Golang (or Dart, or any other) is that the core of Android SDK is written in Java and therefore every app needs to work with Java to be able to tap into it.

However, ART can make it possible for SDK components to compiled down to object code, thus making it possible for Golang/Dart/etc. to link against it, while MAINTAINING backward compatibility with old java apps.

Will it allow new apps to be written in pure native code, without relying on the JNI bridge ?

ART is still Java runtime which conforms to Java memory model, GC requirements and everything that comes with it.

So no, you cannot just link random other languages without going through JNI. That is a common misconception. Also most of Android itself is managed Java code, so making other languages first-party citizens would require pretty much full OS rewrite. I very much doubt rewriting whole Android just to support non-JVM languages is worth it.

Making ART VM fast, efficient and adding Java 8 bytecode compatiblity is I feel a much better use of development time. Wasting precious development time to placate devs that don't want to switch languagaes is - I feel - a hugely wasted effort.

I almost wish Google loses against Oracle at the Supreme Court, and is forced to pay Oracle billions every year as long as it continues to use Java, just so it's forced to switch away from Java. Almost, because I know Google losing that lawsuit could mean dire consequences across the industry. But it's still annoying that Android has to use Java and can't change away from it.

Clearly a win for Oracle would be a horrible disaster for the industry and freedom of speech. It would also be a huge relief and benefit to Android programmers once it forced Google to switch the platform to a less awful language.

What stops you from using Kotlin, Scala, or other languages that are available for Java VMs?

This gets asked pretty often around here, and the answer (IIRC) is that in order to use any non-Java JVM language on Android, you have to link in a lot of libraries, and you start running into limits that Android puts on how much stuff you can pack into an executable.

While mostly true, use of Kotlin has been rather nice with IntelliJ. Standard library is very small and out-of-the-box IDE support with Gradle plugin helps alot.

do you mean the 64k method limit ? It does no longer exist (solved in 21, and a multi dex format has been introduced for earlier releases).

Google made ART as Dalvik 2.0. The ecosystem evolves, the hardware evolves, the OS and its requirements evolve as well, so the runtime must follow. I don't think it changes anything to the JNI bridge. I can be wrong, but I don't think that it is a main area of focus for Google. This being said, Google is moving to a more pluggable build system (Gradle and its plugins) and runtime. Is it a nice benefit and only a side effet ? maybe, maybe not. Time will tell if Google want to move to Kotlin/Go/Dart/Scala/Rust/insert your pet language here.

The official line is that moving the core libraries to something else would be an enormous amount of work and at the moment, they don't see the benefits outweighing the cost.

It is possible that Google is secretly working on some other language, it is also possible that they intend to stick with Java and make it better instead (cue Dagger 2, Guava, AutoValue, ...).

You still need an common ABI and Java has a few features Go cannot understand and vice-versa.

Android team stated multiple times that Java is the language of the platform.

No, ART still works with dex bytecode, and native code still requires JNI bridge.

For Google, ART seems bigger than just Android. Consider the degree to which their infrastructure depends on the Oracle JVM and the associated strategic risk. As one datapoint, recall the Oracle vs. Google Java lawsuit. How much additional ART development effort is required for correct execution of non-AWT (Abstract Windowing Toolkit) Java applications (essentially, headless server processes)? I know the Java/JVM ecosystem well, but I have not done any Android development. Surely, Google wants control over the destiny of its core software stack.

The article doesn't mention one seemingly huge benefit of JIT compilation: profile-guided optimization:

http://www.slideshare.net/ZeroTurnaround/vladimir-ivanovjvmj...

Perhaps the baby has been thrown out with the bathwater?

Little is mentioned about how ART compares to the JVM. For example, does ART perform escape analysis? Not all object allocations are equally bad. The Sun JVM can figure out which objects may be allocated on TLABs (Thread Local Allocation Buffers) - an optimization which reduces the burden placed on the garbage collector because TLAB-resident objects may be deallocated as the stack is popped. [Please fact-check me as I'm merely a long-time Java developer vs. an expert on JVM internals]

I agree that this article is rather superficial, but you have to take into account, that Android was running Dalvik VM, not standard JVM.

Dalvik was rather primitive and extremely slow in comparison to whichever JVM you choose on the desktop today.

Well, OpenJDK is getting AOT compilation as well (planned for 9). In addition to JIT.

The SubstrateVM is the AOT compiler for Graal.

And many commercial JVMs do offer AOT compilation.

Also, .NET has had AOT/JIT since the very beginning. And now static compilation is coming as well.

> Well, OpenJDK is getting AOT compilation as well (planned for 9). In addition to JIT.

On ARM? Last time I checked, OpenJDK doesn't even have JIT compilation on ARM, it just interprets the bytecode.

> On ARM?

Well, at least the closed source JDK has it. As many other commercial JVMs.

As for AOT, it was referenced at Java ONE, Java Language Summit and Øredev 2014.

"...an application’s first start-up will be much increased compared to an equivalent Dalvik system."

Any thoughts on why they they are compiling to native on first run, rather than at install-time?

Often a user will want to run immediately after install, but my feeling is that people are less likely to be frustrated by a longer install time than having to wait for an app to start-up.

Edit: Izacus and InclinedPlane say that AOT compilation is at install-time and the article is wrong.

Actually Android 5.0 runs optimization pass as part of install, not first run. The article is incorrect in that aspect.

Thanks for confirming this.

They are compiling to native at install-time (or on first boot if you're upgrading to ART on an existing device). I'm not sure exactly what overhead is happening on first run that would cause a slow down compared to Dalvik.

It seems like the AOT compilation could be scheduled asynchronously after install. You don't need to prolong install wait for the user. If they don't open the app right away, then the first run will be fast because AOT should have completed by then. If the user tries to open the app before AOT has completed, then the first run will have to block for AOT to complete.

> Any thoughts on why they they are compiling to native on first run, rather than at install-time?

Or why they don't do it on the server so that you get the specialized binary directly from Google Play? Why should my phone be compiling anything?

Because Android ecosystem isn't locked down to a single app store that would prepare binaries for all 8000+ devices with several hundred different SoCs.

The device itself is by far most capable of choosing optimal compiler optimizations for it's own SoC.

Because you can also sideload apps. Or because Android != Google Experience (i.e. Play Store). Why do you want to be dependent on middleman?

This is Microsoft's approach.

Maybe auto-updating apps? It could be that the compilation is resource-intensive enough that it would slow down whatever user is currently doing when an app is updated in the background.

Not an Android Developer here. Is there no "pauseForGC" function in Java/Android? I know AS3 has it, and you can use it to suggest GC at a point where there aren't any animation going on.

There is a System.gc() call which is essentially what you described, but it doesn't help with allocations freezing the UI for memory intensive applications

You can use the System class to do this. "System.gc()" will request a GC run. Of course, its only a request. There's no guarantee when, and if the GC will fulfill that request.

This article is a great reminder that garbage collection should not be the default strategy as it is hard to get right, Dalvik is almost a decade old, and that it introduces performance issues which are not transparent to the programmer. Indeed these second long delays have rendered Android hardware second rate.

betweeen having GC pauses and having memory leak continuously (and buffer overflow issues which become security issues) I'll take GC pauses.

Time has proven again and again that humans are really bad at memory management and having raw access to memory is something that needs to be justified on a case by case basis.

> Time has proven again and again that humans are really bad at memory management

Well until AI write garbage collectors I'm not sure I see how your comment doesn't also apply to them. And lots of great software manages memory quite fine, like that browser you used to write your comment and the operating system it runs on, so not sure the use of "proven" is appropriate in your comment.

Really?! Mozilla even has a memory leaks tooling tracking page:

https://developer.mozilla.org/en-US/docs/Mozilla/Performance

And are making Servo in Rust, which uses compiler dataflows form memory management.

Apple has added ARC to Objective-C and Swift.

Microsoft uses GC in .NET, ARC in C++/CX.

Modern C++ favors RC.

They all seem to think manual memory management is to be avoided.

Rust shows that "between having GC pauses and having memory leak continuously (and buffer overflow issues which become security issues) I'll take GC pauses" is a false dichotomy, however.

Rust is one of the coolest things in programming at the moment, but if you have to invent your own programming language to cut this gordian knot, then it's not really a false dichotomy for most practical purposes. And it'll stay a genuine dichomoty until Rust (or something like it) is a mainstream choice.

I'm glad for the advances in Rust. My comment was meant in the context of the current state of the art (Java/C++ like). If we move forward enough the GC questions could be a thing of the past.

Yes, I am following it quite closely.

However it does place a bit of cognitive burden, but last version already improved it.

> Modern C++ favors RC.

Does it? I was under the impression that the order of preference (from most preferred to least) is something like:

  - plain values (i.e. no pointers)
  - references
  - unique_ptr
  - shared_ptr
  - raw pointers

Raw pointers is the last item on your list after RC pointers.

From this response, I guess that maybe you were meaning C++ favours RC compared to manual memory management, and were not saying that it is the globally preferred option, but this differs to everything else in your list: the others are the main/preferred strategy in those languages/libraries.

> I guess that maybe you were meaning C++ favours RC compared to manual memory management,

Yes.

> the others are the main/preferred strategy in those languages/libraries.

While true, manual memory management is still possible, but should be left for the 1% cases that really benefit from it.

I think we've all used software that suffers from some pretty intense memory leaks ( including older versions of firefox!), and garbage collectors can be "good enough" for a huge amount of applications.

Most applications are _not_ games or web browsers, but enterprise-y CRUD applications.

I'm fine with having the ability to turn that off for specific things, but it should be made as inconvenient as possible to avoid the "Can't have GC pauses in my todo list" usecase.

> And lots of great software manages memory quite fine, like that browser you used to write your comment

Which bit of the browser are you talking about? Garbage collection is used extensively during the execution of JavaScript in a lot of browsers. For example, Chrome [1].

[1] https://developer.chrome.com/devtools/docs/javascript-memory...

Presumably the humans writing popular garbage collectors are significantly better at memory management than the average programmer. You wouldn't say that humans shouldn't go to doctors until there are AI doctors.

Memory management is obscure. `malloc` isn't free either, and gc has been shown to allow some 'out-of-order' benefits. It's a different way of thinking altogether. Right now libart's AOT makes launching overhead very small and tight loops faster but sometimes it introduces lags that I didn't have under Dalvik.

> `malloc` isn't free

I see what you did there. :)

Ha. I wish, but it was too early in the morning for me to generate such puns.

What's the pun ?

free() is the "counterpart" to malloc(). malloc() lets you allocate memory for your use; free() returns that memory back to the system.

The original posters meaning was that "malloc() has non-trivial execution costs", but the other meaning is, tautologically, malloc is the "complement" of free (think true is not false).

> This article is a great reminder that garbage collection should not be the default strategy as it is hard to get right,

You are forgetting that Dalvik hasn't been touched since Android 2.3.

Of course, not much could be expected from it.

Even when it was created, Dalvik had a sub-par GC. A bad GC implementation doesn't say much about GC in general.

It does, actually. The fact that good GC implementations are rare means that GC in general is necessarily subject to poor GC implementations and all the problems that go with them. That may change over the coming years but the fact that it's taken nearly three decades to get the state-of-the-industry in GCs to an acceptable level shows what a struggle it's been.

It's a bit like the folks who say that dynamic languages can be just as fast as compiled languages. Perhaps eventually, but in the here and now in the typical case that claim misses the mark by orders of magnitude.

Not using the state of the art will give a bad impression, yes.

There are plenty of very fast dynamic language runtimes: LuaJIT, PyPy, Spidermonkey, V8, SBCL, etc. They're all competitive with JVMs and some are consistently faster.

Dalvik avoided the state of the art on purpose, for some reason. It didn't even have a JIT by default until Android 2!

There are quite a few decent GCs out there, Dalvik's merely had almost no work put into it.

I think you misunderstood the point I was trying to make.

Sure, there are good GCs out there, but good GCs are harder than many GC proponents make out.

Take the top 100 most heavily used applications or platforms that rely on garbage collection. What fraction of them actually use decent GC implementations?

Also, you seemingly misread my other point about dynamic language runtimes, I wasn't comparing them to JVMs, I was comparing them to compiled code written in lower level systems languages (like C/C++). V8 may be fast, but it's not as fast as compiled code, yet. There's still a huge gap between theory and practice there.

I read somewhere that the Dalvik original team is no longer at Google. Not sure if this is true.

It was pretty ridiculous, even on some of the most powerful hardware out there I still found animations to lag from time to time due to the GC_FOR_ALLOC calls! Ended up with a bunch of hacks to try to work around the issue, but none fully resolved it.

If animations lag because of allocations, then you are allocating too many objects.. The most common cause is object allocation in a critical part like a draw call (or view binding).

This is true, but for operations like loading images from disk for example, this is unavoidable. The best you can do is reuse memory instead of allocating whenever possible

On Android you should definitely load the images Async and not load them within the draw call.

Even loaded asynchronously, the calls for allocation still cause lag. Unless I am doing something incorrectly?

You can use inBitmap in order to reuse bitmap instances instead of of allocating new objects. If it fits your use-case, Glide is a very modular lib that implements inBitmap. It is true that you will always have to create some objects though. For example with Glide, a DataFetcher is created with each new image request. You should do your best to keep it at a minimum though, with a focus on creating heavy objects only when it is absolutely necessary.

Other than that, lag can come from many things, so if you lag without GC, the only solution is to profile that part of your app.

No matter what language, allocations are expensive. Your stuttering would appear in C++, ObjC or Java. Don't mistake slow allocations for a GC issue.

The URL points to the second page of the article, can someone fix it?

That's because the original submitter wanted to highlight the garbage-collection discussion, not the whole article. Of course, now that the title has been changed...

Thanks! Fixed.

It was about time Android catched up with how iOS and Windows Phone work (e.g. everything is AOT compiled to native code).

Now it just needs to provide a similar developer experience to the other mobile platforms for the C and C++ developers.

In terms of scaling to more architectures, this solution looks superior to iOS's and on-par with Windows Phone's CIL-MDIL system. Since the original Android phones, they've added ARM processors with VFP, Thumb, Thumb-2, NEON, and now ARMv8-A; MIPS processors; and various Intel instruction sets. Developers targeting Dalvik bytecode can ignore all that complexity going on underneath. The same APK they built years ago will work everywhere.

I agree that they should have made the transition to AOT a long time ago. Their technical excuse is that devices didn't have enough space. That's only because they allowed devices to not have enough space. Even the original iPhone had 4 GB flash minimum.

Android's model for 64-bit support also seems superior to all other operating systems, in terms of not adding the "64-bit memory bloat" (which is roughly 30 percent more memory required for 64-bit apps), so in terms of RAM needed 64-bit Android apps should actually require less than 64-bit iOS apps.

However, I'm not completely sure whether Google just restricts the addresses length to 32-bit, or they are keeping the apps 32-bit even on the 64-bit architecture. It sounds like the former, but I really hope it's not the latter. So far I've not seen ARMv8 supported in the SDK and Nvidia's 64-bit Denver CPU still comes out as 32-bit in benchmark tests, even on Android Lollipop.

I don't know whether that's related in any way, or it's some other Google screw-up (not being ready on time with Aarach64 support), or it's just the benchmarks who don't support ARMv8 yet.

Oh and I agree with your comment on storage. Google should impose more "reasonable" requirements. In 2010, even high-end HTC flagships came with like less than 200 MB free storage. Absolutely unacceptable, even at the time. I've hated my HTC phone for so long because it, and it kind of made me not want to get HTC ever again now, since the brand is tainted in my mind.

Today, even $50 Android phones shouldn't have less than 4GB internal storage (which is like 1GB free storage), but those over $100-$150 should all have at least 8GB. Most people should get at least 16GB. When I'll buy a flagship phone a year or so from now I intend to get one with 64GB internal and a UHS-I 128GB microSD to shoot 4k video and RAW pictures.

Android's model for 64-bit support also seems superior to all other operating systems, in terms of not adding the "64-bit memory bloat" (which is roughly 30 percent more memory required for 64-bit apps), so in terms of RAM needed 64-bit Android apps should actually require less than 64-bit iOS apps.

Is this actually going to be the case? 64-bit addresses allow the use of tagged pointers, which I understand ameliorate any additional cost introduced.

How does Android's support for 64-bit avoid the memory bloat?

As I understand it, the major speed improvements with ART come less from the native compilation than from the improved GC and the ability to do whole-program optimizations during the AOT compilation step.

EDIT: Also, ART takes a different approach to AOT compilation than either iOS and Windows Phone. iOS apps are shipped as compiled binaries from the start, and apparently Windows Phone apps are compiled in the cloud by MS. ART does the native translation step on the device itself, which has only really recently become feasible with the speed and storage capacity of recent Android devices.

I think his main point was that it has caught up with the performance of AOT, not the nitty gritty details behind the process, which is kind of irrelevant to the user.

It is relevant to the user for two reasons: portability and speed. In contrast to iOS, Android in the wild runs on everything from ARM to x86, in many different generations. Doing the compilation on the device allows the AOT compiler to optimize for the specific CPU that the device uses. Also, it increases compatibility, because what is shipped is platform-independent bytecode, not a binary that may only target one specific architecture.

This is a solved problem since the early 80's, known as fat binaries or delivering multiple binaries in a package.

I code mainly in C++ (NDK) and don't have any issues delivering code.

This is a solved problem since the early 80's, known as fat binaries or delivering multiple binaries in a package.

No, they are not the solution: you cannot provide new optimized versions for generations that did not exist yet when you compiled the binary. Also, providing an optimized version for every ARM, x86, etc. generation will make the binaries very fat.

Fat binaries worked for relatively static platforms, such as Macs. For devices which are (still) iterating quickly, it's a suboptimal solution. Sure, it works for delivering code, but it is suboptimal.

(Not that I believe that ART is currently optimal, e.g. it should not be necessary to do the compilation of an app on 1 million devices that are identical.)

They are only fat at the store, the devices only see the .so they understand.

And it isn't that hard to have "APP_ABI := all" in Application.mk. The build just takes a little longer on the CI system.

There's currently more than 100 SoCs out there and several major generations of ARM architectures which show clear benefits of GCC compiler tuning for their platform.

Tracking all those devices and architectures is dumb, when the manufacturer/device can provide proper tuning for the device itself.

Yes, but my experience tells me that "can" != "he/she will".

> known as fat binaries or delivering multiple binaries in a package.

That's not a solution, that's band aid. Fat binaries or multiple binaries cannot support future architectures or architectures that the original developer doesn't support. Platform independent bytecode can.

That is the theory, which I preached for a long time as well.

The real life looks a bit different.

An a simple example, I can recompile my application and target any device while using 100% of all CPU features in all Android generations.

Whereas Dalvik and ART are stuck to the versions that were burned into the silicon.

So gcc and clang can easily outperform Dalvik in < 4.4 generations, given that it was hardly changed since 2.3.

An a simple example, I can recompile my application and target any device while using 100% of all CPU features in all Android generations.

For one application. And again, your binaries will be very fat if you want to optimize for every possible CPU generation.

Whereas Dalvik and ART are stuck to the versions that were burned into the silicon.

Which is a problem with how Android updates are distributed, not the principle of doing AOT compilation on byte code. I think pretty much everyone agrees that the push of updates to Android devices sucks, compared to iOS or, to some extend, Windows Phone.

Also, for optimization for a particular CPU it should not matter if ART is not upgraded, as long ART was optimized for the CPU at the time the phone was released. (Of course, you would miss out on newer optimizations in ART.)

> Which is a problem with how Android updates are distributed, not the principle of doing AOT compilation on byte code. I think pretty much everyone agrees that the push of updates to Android devices sucks, compared to iOS or, to some extend, Windows Phone.

That is why I said:

-- quote

That is the theory, which I preached for a long time as well.

The real life looks a bit different.

-- quote

So one is bound to use the approach that works best, regardless of how it could be.

It still depends on you, as a developer, to recompile the app for the target cpu.

I have a collection of M68K and PPC Mac binaries, that nobody is going to recompile for me. I would take sub optimally running app over not-running-at-all app any day.

> I have a collection of M68K and PPC Mac binaries, that nobody is going to recompile for me.

Apple did.

http://en.wikipedia.org/wiki/Rosetta_%28software%29

Rosetta is not supported in current systems. Since 10.7., I think.

Classic, which provided the ability to run Classic M68K and PPC binaries, had it's last release in 10.4. It wasn't supported in 10.5 and it never run on Intel Macs.

Not to say, that both these technologies are limited hacks compared to proper platform-independent bytecode ala Dalvik, ART, Java or CIL.

"Whereas Dalvik and ART are stuck to the versions that were burned into the silicon."

I don't understand this comment? What hardware is the Dalvik and ART stuff wedded to?

The ROM containing the firmware.

Except for the top OEMs, most Android devices don't get any updates, so you are stuck with the JIT that was burnt into the firmware.

What do you mean by 'ROM' or 'burnt'? Android is normally on flash storage. Even if a device does not get updates anymore, you could still replace it if the boot loader is unlocked (e.g. Cyanogenmod).

Of course, you know this, so I can only assume that you are using such words for dramatic effect ;).

Which normal users are unlocking their devices?

It doesn't matter if it is flash, eprom, rom, whatever.

The practical result is that most devices die with the Android version they were bought with.

Normal users do not care about firmware versions or cpu optimizations.

An app either runs, or not. Dalvik allows the apps to run, without the user having to know about cpu architecture in their phone or whatever.

But I as developer care to deliver the best experience in terms of performance from Android 2.3 all the way up Android 5.0.

Sure, Dalvik apparently was left to gather dust for a few Android releases.

Yes, having the AOT compilation done on the device is an handicap, given how OEMs barely update Android versions.

So while Apple and Microsoft approaches mean you can target older devices while enjoying improvements in code generation, with Google's approach you're stuck with whatever the device supports, unless you use the NDK instead.

Reply on news.ycombinator.com