A performance comparison between Java and C on the Nexus 5
learnopengles.com
learnopengles.com
I think it is one of the things Apple gets right; they know what matters. As an outsider to the iOS/Mac ecosystem I've often felt that Apple makes trade-offs in productivity to achieve a better experience for the end user. While these tests aren't indicative of typical app performance (most apps aren't doing math in a loop), an order of magnitude performance difference is enough to show up on many apps.
I wonder if Google made a mistake by not choosing native code for Android apps. We have heard forever about Java (and other run-times) getting close to C in performance, but outside of specific test cases native code still runs circles around everything else out there.
http://forum.xda-developers.com/showthread.php?t=2706200
I have had an S3, iPhone 5, and HTC One (M7) and the iPhone 5 definitely felt the smoothest, but now I'm interested in seeing if the M8 really has leveled the playing field.
UI thread priority has been debunked time and time again: https://plus.google.com/105051985738280261832/posts/XAZ4CeVP...
Even if it hadn't been, what evidence is there that it would require a ground up redesign?
In the meantime it does seem likely that Java has a non-negligible performance cost. My personal worry is that fast Garbage Collectors usually assume that twice the memory is available than is used. That might put the fact that Android phones typically have more memory than Apple devices in an interesting light.
- namespaces
- memory safe
- you don't have to type @ and [] everywhere to switch between dynamic and static worlds
As for the vm thing, there are also commercial native compilers, even though the general HNer tend not to be aware of them.
Thus giving Android developers all the fun of dealing with GC pauses. There's a lot to be said for ARC, really; I'm quit glad Apple ditched their brief experiment with ObjC GC.
> namespaces
I'll give you that one; lack of namespaces it the worst thing about ObjC.
> ... Dynamic and static worlds...
I'm not sure what you mean here; Java is a statically typed language, and does not have a 'dynamic world'. ObjC has a fo of optional dynamic typing, which is arguably modern, though also only arguably a good thing.
Because they failed to implement a GC that didn't crash all the time, as the Apple forums used to testify before ARC was touted as the best thing since sliced bread.
The wonders of mixing Objective-C libraries compiled in mixed mode.
> I'm not sure what you mean here;
The mixed nature of Objective-C.
With @ and [] one is in Smalltalk land. Without them, one is in C world.
> Java is a statically typed language, and does not have a 'dynamic world'.
Yes it has, via reflection, annotations and dynamic bytecode generation.
This is an odd and not particularly true way of looking at things. [] (for message sending), is merely some syntactic sugar for the objc_msgsnd function call.
A means of gathering the code, dump it into a series of objc_msgsnd, without doing much of real compiler work.
There are languages with Smalltalk semantics without forcing me to type @ and [] multiple times per line, a pain in non US keyboards.
id map = @{ @"my key": @123};ImmutableMap.builder().put("my key", 123).build();
Which hardly seems neater (especially in a more realistic case where you have more than one entry).
First, the overhead of GC depends not only on the heap size, but also on the allocation rate and allocation profile. For reasonable allocation rates found in typical GUI apps, most Java GCs do extremely well having very little additional memory, like only 20% more than the live set size. There is no "twice as much" assumption anywhere in their design.
Second, manual memory managers do have their own overhead too, in form of bookkeeping (e.g. free lists) and internal fragmentation. Fragmentation can be sometimes much worse than GC memory overhead for long running applications, because GC memory overhead is typically bounded, while fragmentation is either unbounded or has extremely high upper bounds (like 10x in "fast" allocators).
There is in that the nursery collector is often a simple Cheney-style copying collector that wants two semispaces. But that's only one of the generations. There's definitely no requirement that total GC memory is double the used space.
Apparently they haven't done any changes since JIT was introduced in 2.3 version. Additionally we have ART without any information whatsoever about future plans.
* Android devices have very laggy touch responsiveness in general, while this is something Apple has spent considerable effort optimizing [1]
* Only in recent revisions (Project Butter [2]) has Android made significant improvements to its general UI lag/smoothness issues. At various times, Google employees have gone into extensive descriptions of why the UI is so janky [3][4]
* Many of the default Android elements were for quite a long time, very unoptimized - this included calls w/ lots of overdraw, or lack of node recycling for lists. This forced app developers to take on the bulk of performance optimization. You can take a guess how often that is done well. [5]
Over time, Android the OS, SDK/API, tools have all improved but I believe your second paragraph gets it right. At the end of the day, people at Apple cared a lot about getting the first versions of iOS to feel great. No one at Google cared enough about about it. Even today, when you compare the iPhone to the latest Nexus devices, there's lots of hardware integration issues that will probably never get fixed (outside of community patches).
[1] http://venturebeat.com/2013/09/19/apples-iphone-5-touchscree...
[2] http://www.androidpolice.com/2012/07/12/getting-to-know-andr...
[3] https://plus.google.com/105051985738280261832/posts/2FXDCz8x...
[4] http://lazytechguys.com/news/google-dev-explains-why-android...
[5] http://www.curious-creature.org/2012/12/01/android-performan...
Quantifying this IPC might make an interesting experiment and blog post...
Man that was a weird read. Guy spends a few months interning and all of a sudden talks about his profound connection to the campus, the fundamental flaws in the product architecture, but watch out! He may be biased since he will be interning on a competing product the next summer!
Regardless of the facts of the matter this seems like a very non-authoritative source.
edit: oops, that was the post that sparked the original specious musings. I meant this followup one: https://plus.google.com/105051985738280261832/posts/XAZ4CeVP...
[4] is total horseshit. He doesn't know squat about any of the platforms. iOS devs also called him out for getting just about everything wrong about how iOS works.
[5] is something iOS devs totally also have to do as well. That is so not an Android-specific thing.
Social problem, basically.
Adapter#getView[1] has been around since API 1, which is used in ListView (as well GridView, etc.) to recycle view nodes.
[1]: http://developer.android.com/reference/android/widget/Adapte..., android.view.View, android.view.ViewGroup)
The choice of Java/Dalvik does seem like a mistake, and was a carry-over of Android coming from classic J2ME roots, which worked well with Google's love of all things Java.
Where did you get the idea that Android had J2ME roots? It definitely did not.
http://www.theverge.com/2012/4/25/2974676/this-was-the-origi...
http://www.scribd.com/doc/217988654/13/Boot-Loader
the Platform will be compatible with Java Platform, Micro Edition (Java ME).
> "Leverage Java for its existing base of developers. Build a useful app framework (not J2ME). Support J2ME apps in compatibility mode. Provide an opTMobileized JVM (Dalvik)," one slide reads.
I think there's a pretty big difference between building a platform on top of something, and planning to support it via a shim layer. At that point Android was already planning on breaking from J2ME by supporting things like JNI (which is listed on that doc but is explicitly not included in a conforming CLDC implementation for security reasons).
Besides, the only microedition APIs that are in the platform are a few khronos libs, so I really don't think any of that thought really affected Android as we know it today (which was the parents original argument). I will give you that there's more JavaME heritage than I thought, but it was thrown out pretty quickly as far as I can tell.
I would say yes.
> We have heard forever about Java (and other run-times) getting close to C in performance, but outside of specific test cases native code still runs circles around everything else out there.
Please don't mix language vs implementation.
Second, the author points out NDK over Java as a solution which is already leveraged by some popular apis transparently from java, such as libgdx.
Finally, the ui is responsive enough. My biggest annoyance whenever I have to help my in laws or my parents with their iphones and there is no back button - I don't notice any significant difference between their iphones and my Galaxy S3, and I'm sure Samsungs phones have only gotten better.
[1] we have three iOS devices and I'm the loveless tech support for them.
I have to strongly disagree here. I'm a mobile dev, I work on Android and iOS. My main phone is an iPhone 5s, recently I tried switching OS just so that I would be more comfortable with the different OS's day to day idioms. I couldn't stand Android, it was sooo laggy feeling compared to a iPhone or even a windows 8 phone. Both Apple and Microsoft have large leads in user experience and speed. Don't get me wrong Android is certainly better than either in it's own way but performance and responsiveness are not it's strong suite. 3 apps that the difference was obvious to me are: Google Maps (the ios version feels way smoother), Evernote, and the facebook app. Also see the yahoo weather app the difference was incredible. Phones I used in testing: HTC One, Samsung S4, Nokia Lumia 1520, and iPhone 5S
Granted all of this was based upon personal experience and isn't a scientific study but to me the difference was absolutely clear. Perhaps if I had only used an Android device I wouldn't have noticed the difference as much but having been a long term iOS user I could not stand it.
Thanks for including that line, because my personal experience is the exact opposite. I have a four year old android phone and I dread any time I need to use my wife's one year old ipod. I find the lag on the ipod slow to the point of almost being unusable. Every time I use it, there's at least one point where I'll put down the iPod to go do something else while I wait for it to catch up with me.
It's amazing how much personal experiences can differ.
Edit: updated possibly relevant details
Honestly, try a Nexus 5 if you get the chance. The hardware in the phone makes a huge difference (and it's often in places that aren't benchmarked, like the touch controller).
The biggest tragedy is that Google had a chance to make a better NDK experience with Android Studio, but instead they chose to ignore it completely. With Eclipse you at least had an official add-on for NDK support, but now you're expected to manage it with your Gradle files by hand. Now, even more than before, you're at the whim of solutions on Stack Overflow.
There's a reason AS hasn't hit 1.0 yet: it isn't ready.
[1]: http://docs.google.com/viewer?a=v&pid=sites&srcid=YW5kcm9pZC...
This is just my personal opinion.
Kinda. I imagine that if these benchmarks were run on idiomatic Objective-C, they'd be an order of magnitude slower than Java. All method calls are un-inlineable and often require a hash table lookup to resolve.
However, the nice thing about Objective-C is that it's ridiculously easy to write performance-critical stuff in C or C++, then expose it with a nice OO interface. Which means that you (or, more likely, the people who wrote the library you're using) are more likely to bother. That and the lack of GC making latency easier to reason about.
So while it is probably easier to write performant UIs in Objective-C than Java, it isn't because of the reasons mentioned here.
In addition, Apple's Accelerate framework makes it easy and natural to take advantage of SIMD for further speedups — you can get powerful vector operations with single function calls. This seems to have no Android equivalent.
Renderscript?
s/idiomatic/pure/
However, implying that "require a hash table lookup" is slow isn't really right. The message send implementation is extremely optimized and takes about a dozen CPU cycles to execute last I measured. To put that in perspective, that means that an Objective-C message send on the fast path (which happens 99.99% of the time, the slow path basically just happens once per message/class combo) costs about as much as two C function calls, or one and a half C++ virtual function calls.
Message send overhead in a typical Objective-C app is small. 10% would be high, for really message-heavy code, and 5% is more typical. Most Objective-C code is not messaging so it's still fast.
Take these benchmarks as an example. To write them in "idiomatic Objective-C" you'd change almost nothing. You'd either get rid of the few C++-isms (templates for min/max functions, parameters passed by reference, no big deal) or compile the file as Objective-C++. You'd never write code like this using Objective-C objects. In addition to be slower, it would be harder to write, harder to read, and much larger.
My point was really that Objective-C isn't always faster than the equivalent Java (a function call, hash lookup and jump is fast in most contexts, but it's still infinitely slower than nothing, which is the overhead of a JIT-inlined method call), but that being a superset of either C or C++ means that it can be faster when it needs to be. Which is ultimately what matters.
However, my point is that not only can you write faster Objective-C code, but that most normal Objective-C code will be faster, because most normal Objective-C code isn't that message heavy. So it's not just that it can be faster, but that it usually will be. Typically. Depending on what you're doing.
You can use native code for Android apps. I do. In the NDK samples there is an app called NativeActivity which is 100% C/C++. You compile an APK without writing any Java/Dalvik code.
From my anecdotal knowledge, most of the popular Android games are written mostly in C++, using the OpenGL ES library, and have a common code base with their iOS counterparts. Maybe they are all native code, maybe they use some Java and JNI stuff, but they are mostly C++. So Android is native on the game side, where it is needed.
A deep look at how all native code runs will show some non-native system code run when it launches, but that becomes pedantry if performance and not purity is the focus.
Android is cleft in two - applications are mostly Java/Dalvik with maybe some JNI access to C/C++ libraries, games from all my discussions with Android game developers are usually mostly the same C++ code and OpenGL ES they are using on iOS.
One example I know a little about is Jigzo. It is an open source game which is C++ and uses OpenGL and SDL libraries. The original developer ported it to iOS. I ported it to Android. Same basic code base, with a few tweaks for each OS. I changed the name from Jigzo to Jigsaw Puzzles on Play though.
At least now the NDK has some sample source code for JNI helper classes to ease the pain.
Qt is the only sane approach for doing full Android applications in C++, but then one is writing multiplatform code anyway.
Seeing as Android runs on a lot more than just ARMv7, no, they didn't appear to make a mistake here at all.
Apple can tell developers to compile their app twice for ARMv7 w/ NEON & ARMv8-A64, but is Google going to go tell everyone to compile for ARMv6 (Android devices running this are still shipping), ARMv7 w/o NEON, ARMv7 w/ NEON, ARMv8 (not here yet on Android, but obviously coming), MIPS, X86, etc...?
> We have heard forever about Java (and other run-times) getting close to C in performance, but outside of specific test cases native code still runs circles around everything else out there.
The vast majority of the Java code in an Android app isn't performance-critical either. The code that actually renders an app is all in native anyway. The code to handle touch events and such isn't, but the difference between .1ms and .2ms isn't going to break the 16ms bank either.
When there are new architectures, you get a warning in XCode and press "Upgrade Project".
Right now my 'default' build is for armv7, armv7s, armv8, x86, x86_64. This setup required no effort on my part. I just created a new project.
Several of Google's own google apps depend on native libraries. Try to install google apps for android 4.2+ on a device without neon like tegra2 and see how many of them actually work and how many crash with SIGILL.
Which means that pretty much computationaly intensive tasks (e.g. image processing, crypto, etc.) are several orders of magnitude slower than what you're used from desktop JVM. That's especially noticable in Android's BouncyCastle crypto implementation, where standard algorithms like PBKDF#2 derivation function can take ages (e.g. 3 seconds for 10.000 iterations on Nexus 4). Simply just moving those algorithms to C library without any optimizations quickly gives you orders of 100-500x speedups just from the GCC compiler (the before mentioned OpenSSL implementation of the derivation algorithm runs about 80-150ms for same parameters and result). So now we do pretty much any crypto or bitmap algorithms in C and just call C library via JNI.
Of course... most apps don't really do anything computationally expensive so C doesn't give much benefits.
Really? I think it's fairly easy to determine what would be expensive vs. what's going to be relatively cheap. I'd love a counterintuitive example to prove me wrong though.
Those are two different languages that just happen to have same roots.
Except for a few things C++ has imported from C like stdint and vararg macros the truth of your statement depends strongly on how far you personally tend to stray beyond C89 in either direction.
Source: Why mobile web apps are slow - http://sealedabstract.com/rants/why-mobile-web-apps-are-slow...
BTW: Your link is about JS, not Java. Performance problems of JS have little to do with GC, there are much bigger problems first.
Yes, the link is about JS. But there are relevant pieces.
Quote from the article: "If you want to process camera images on Android phones for real-time object recognition or content based Augmented Reality you probably heard about the Camera Preview Callback memory Issue. Each time your Java application gets a preview image from the system a new chunk of memory is allocated. When this memory chunk gets freed again by the Garbage Collector the system freezes for 100ms-200ms. This is especially bad if the system is under heavy load (I do object recognition on a phone – hooray it eats as much CPU power as possible). If you browse through Android’s 1.6 source code you realize that this is only because the wrapper (that protects us from the native stuff) allocates a new byte array each time a new frame is available. Build-in native code can, of course, avoid this issue."
I don't know the Java internals on Windows. But someone know why applications with graphical interfaces feels so slow ? My rough guess is that even if Java is fast for mathematical operations, it needs to call native libraries for drawing and this is expensive because of the isolation.
You can draw tens of thousands of things on the fly in a web browser and have it appear instantaneous to a human.
Any shop that ever did a bunch of Swing and cared about quality ended up writing their own layer on top of Swing that made those features easier to use, but they still require quite a bit of discipline.
UIs is the kind of place where the whole reactive model actually works well. Create events, subscribe to events, and make each event handler run asynchronously. Use the dispatch thread for events that actually modify frontend-facing, elements, and everything else on background pools.
So the broadly available toolset could only be used well by programmers that are better than the typical guy that was asked to write the UI, and we never got better tools.
How many times through the benchmark? Did the JIT get a chance to warm up? Caliper provides a framework that can be used to properly assess performance.
There's no benchmark here without seeing a more complete picture.
for my fellow perplexed: ART is a java runtime (dalvik alt) https://source.android.com/devices/tech/dalvik/art.html
impressive improvement, x2 dalvik (on this one case...)
BUT for similar GHz (2.3 vs 2.6), intel is x2 to x8 times faster than ARM...
And side note, we have not seen 54 bit arm cpu performances yet as well
for me... i've been coding on, and browsing with, a dual core 1.2GHz cortex 9.... it's fast enough for what I've been doing. (you need a bluetooth keyboard though)
Is Dalvik actually a JIT these days? I thought it was just a bytecode interpreter with a register based VM.
In any case, ART seems to be heading down the AOT path, which I think still makes sense in a mobile setting.
The reason that the intel processor is so much faster is that with a higher power envelope, you can have extra transistors. There transistors can be used to have better branch prediction, have better prefetching, have better SIMD instructions and have more cache. This is what really impacts performance.
If you compare the performance per watt of both processors you see that they are somewhat similar. Keep in mind that ARM processors are also made to be much cheaper than their intel counterparts. Both ARM and Intel do a stellar job in their respective markets.
Even the slowest native build using clang is not more than 43% slower than the best native build using gcc.
That option results some changes in the floating point math (see http://gcc.gnu.org/onlinedocs/gcc-4.1.1/gcc/Optimize-Options...) which Java is unable to set, but which may be okay for limited domain the DSP filter works in.
Java has an absolutely horrible floating point support. See http://www.cs.berkeley.edu/~wkahan/JAVAhurt.pdf
Java code can thus never (in the current language spec) take advantage of vector instructions etc. While in raw integer performance Java might come near native code whenever you use floats Java is going to always lose.
The sad part here is that the decisions made on Java do not even save the programmer from the evils of floating point math. They just cripple the language.
As for the horrible floating point support, are there other factors than lack of signaling NaNs, being unable to set rounding mode and exception policy, floats using floats for intermediate results?
For many purposes such flaws don't seem that bad.
Why is Java code unable to use vector instructions? I guess there's something in the Java spec that prevent compilers from using them, but what is it? I think that the 80-bit floats the Intel FPU offers are used in Java by default unless a method is defined as strictfp, but this a different issue.
The second is that the a decent JVM i.e. not Dalvik do generate vector instruction streams.
I actually think that java code in the next 4 years will start to be much faster than equivalent C code, due to HSAIL and Graals potential to involve GPUs and specialized floating point hardware in an otherwise normal program.
I'm pretty sure dalvik is NOT generally 17 times slower than native, and this test may just be showing the start-up overhead.
It took Whatsapp two weeks to issue an update that made it work with ART. Nothing else and nothing since has posed any problem. In fact, I forgot I was using it until I saw your post.
The original Android team had a few ex-Sun employees from the Java team, so it might also play a role there. Just guessing.
- Make Go code compile as .so, currently Dalvik/ART are always the app entry points.
- As you say provide a wrapper for all existing APIs, with the problem that many rely on implementation inheritance, not supported by Go. So a 1:1 mapping is not possible.
IOS has these nice loops, more like HMTL.
I hope Google copies Firefox mobile via Gentoo based ChromeOS. Their copying of Java took the worst parts.
'Andorid is the new IE'
Sure, Dalvik may have a JIT but it's difficult to do miracles with a limited amount of memory (and the JIT needs to be fast as well)
This is making the assumption that any latency delta is because the system is CPU bound; I suspect this is one of the least likely cases. In general, I suspect apps are going to be I/O or fillrate bound, with some additional latency because touch controllers aren't benchmarked and are only starting to get more love from device manufacturers.
You don't have to be a computer scientist to see that the fastest way of doing anything is not doing it at all --- and with Java's architecture, there will always be instructions executed which are completely unnecessary in native code. They can make huge sacrifices in memory consumption (e.g garbage collection that doesn't collect at all, if you have enough memory) to try to reduce things like allocation/deallocation cost, but in the end they're still wasting more resources.