Mysterious Android codebase commit | Hacker News Reader

Mysterious Android codebase commit

android.googlesource.com

236 pointsrogerthis158 comments

To explain if you are just joining in: This pretty much means Oracle v Google, a case with major ramifications for the industry has been settled out of court. I don't see how this can be interpreted any other way.

Also, why are the two timestamps so far off? Authored in February 2015 (almost a year ago!), but committed only in Nov 2015.

Maybe because this has been ready for a long time just in case and they just decided to actually switch to that strategy right now?

This is a git repo, so "authored" tries to map to the person who wrote the content, and the "idea" of the commit, if there is such a thing. It's set when the commit is initially created, and for the most part not changed unless explicitly requested. The commit date is the actual time the commit object was made; this can differ if the commit is amended, rebased, cherry-picked, the result of a squash/fixup, etc. If a commit sits on a branch for months, then gets rebased against master, code-reviewed, and merged, those two dates will be significantly different.

This usually works as you might expect. If your coworker Joe authors a commit on a branch, and you cherry-pick it onto another branch, that cherry-picked commit is authored by Joe (and the time reflects when Joe authored it) and commited by you (at the time you performed the cherry-pick).

First, read my response to the other person who provided a technical answer to this question.

https://news.ycombinator.com/item?id=10806642

You have gone a little further and provided some general reasons as to "why this might happen", but you still seem to have missed the essence of the question, which was almost certainly more "can someone come up with some guess why, in this particular circumstance, given the theories about this commit being influenced by a settlement of the Oracle lawsuit, this commit was authored in February but only landed in November?".

This means that a person named Piotr Jastrzebski created a commit in February. This is a different commit based on that one (not in the sense of revision ancestry, but in the sense of rewrite history): this commit is the N-th rewrite of Piotr's Jasterbski's commit, for some N > 0, and the rewriter ("committer") is Narayan Kamath.

The rewrite can have different content: for instance, it can be altered to merge against a different parent.

An example of a rewrite is a simple cherry-pick (e.g. from one branch to another).

All these things are commits. The initial authoring is a commit, and this latest rewrite by a different person is a commit.

You have answered how it is possible, not why it happened: that is like someone pointing out that this "mysterious" commit is actually quite easy to explain, as a commit is an object in a git database, and we have all the metadata to see who made this one: I am extremely well aware of how it works (I give lectures at UCSB about git where I start by explaining the internal file formats and work my way up to the various command line tools), but I still found the question here of "why was this authored in February but only landed in November" fascinating, given the speculation about Oracle.

If this inference is right, I hope this means that Google will enable the web community to use the JVM with or as an alternative to WebAssembly. Hotspot is absolutely incredible technology, and it's competitors are still many years away from coming close to matching it's capabilities.

Embedding the JVM to run code within webpages you say? Hasn't that been tried before? :-S

I'm all for the JVM but it's too powerful to run arbitrary code man. No JVM in my browser please.

The Java plugin's mistake was to embed a rich sandboxing mechanism right in the middle of the language layer. The Java runtime, in the middle of parsing bytecode and registering class hierarchies, is supposed to enforce who can access what things, not simply as an advisory mechanism or safety check (as public/private is just about everywhere else), but as a security mechanism under active assault, with only this single line of defense between untrusted code and full local privileges just like native code. And, like anything with a complicated security policy and a wide attack surface, it had no chance.

Stick a regular, unprivileged JVM, with no secure classloader magic, inside a straightforward non-Java low-level sandbox like NaCl or even just PPAPI + Chrome's renderer sandboxing (like Pepper Flash or PDFium) and it'll probably hold up just fine.

And honestly that's what Android does. Java isn't a security boundary on Android, and the NDK makes this explicit. Each app runs as its own UID, and the kernel is taught to isolate users a bit more than usual for UNIX, and that holds up pretty well -- not perfect, but far better than the Java plugin does.

(To be clear, I'm not advocating the JVM as a platform for web content. Just that, if somehow it turns out that the JVM is in fact the right platform, the sandboxing problem not a blocker.)

The Java plugin's mistake was to have a crap auto update mechanism, and be politically unpopular because it wasn't open source. That's it.

Both Firefox and Chrome ship massive numbers of security fixes with a high degree of regularity, but nobody cares, because:

1) It's the web, and The Web is close to religion for many people.

2) Their auto update mechanisms are pretty good, for Chrome, best in class, so people get the bug fixes.

The JVM sandboxing mechanism itself isn't the issue. The design is sound. All sandboxes have escapes especially where native code is concerned, and JavaScript sandboxes have proven no different in this respect. The issue boils down to auto update (lack of).

And throw out the hard work __ALL__ browser vendors are doing TOGETHER on WebAssembly? In favor of a VM that clearly has restrictions and is owned by a litigious organization? Bloating browsers by adding another VM, which will hurt throughput by having multiple GC's to synchronize?

The JVM ecosystem and Hotspot are certainly modern marvels, but there's hurdles greater than the potential benefit.

The JVM is already open source. If this matter has indeed been settled, then the litigious organization issue might have been too. WebAssembly doesn't have a GC, and probably won't for some time. It also doesn't have ~20 years of performance tuning on different microarchitectures the way the JVM does. Benefits include getting scala, clojure, jython, jruby, and about 30 other languages working client side in the browser for free. Not to mention the languages that have been statically compiled will run way faster than the equivalent JS. That would be great for mobile, and the spread of the open web.

> The JVM is already open source.

I can picture Google's lawyers using that in court against Oracle.

> then the litigious organization issue might have been too

That might is a big MIGHT, the same kind that keeps browser vendors from shipping various media codecs and news companies from using various media streaming protocols.

> WebAssembly doesn't have a GC

The GC all browsers currently have and will not get rid of that I was referring to was the JavaScript VM.

> It also doesn't have ~20 years of performance tuning on different microarchitectures the way the JVM does.

And the JVM doesn't have the start up performance of any JS VM. I would estimate the number of exploits to be significantly higher for the JVM, but worth less.

> Benefits include getting scala, clojure, jython, jruby, and about 30 other languages working client side in the browser for free.

JS is the most compiled to language in existence. Also, my point has been none of this would be free. How do you integrate JVM with DOM? Who's going to rewrite all that? Who's GC cleans up around here?

> Not to mention the languages that have been statically compiled will run way faster than the equivalent JS.

Are you talking about WebAssembly here? Cause it sounds like you are.

The startup performance of HotSpot is largely related to the SDK and class library. Remove it and you have a very expedient startup.

Pull an engine out of a car and it gets lighter, too.

I don't think this is a significant issue because it's a one time cost. It could be done at browser startup. Even for a cold start, Hotspot and V8 are comparable.

$ time java com.Hello real 0m0.096s user 0m0.103s sys 0m0.013s

$ time node /tmp/hello.js real 0m0.073s user 0m0.067s sys 0m0.007s

Funny ;-)

I actually worked for Sun/Oracle on VM's but on the embedded not hotspot team. The trick is actually simpler and its called MVM, which is something Sun kept avoiding on the desktop/server for some stupid reason but we did do it on mobile and it made startup almost instant.

The trick is to share one VM instance between multiple apps. So when the OS starts you start the VM process and then fork with a lot of the JITted code already in place so you get almost instant startup and reasonable process isolation.

In mobile where there are some restrictions this is very practical. On the desktop/server this gets a bit tricky with bytecode manipulation, classloaders etc. But this is totally doable.

My personal uninformed theory is that Sun or Oracle didn't do this because they didn't care. MVM has two use cases: faster startup/lower overhead on desktops (they don't care about that).

Server efficiency in small scale deployments (which they don't care about either). AFAIK Google did some work on MVM for App Engine Java, but those are just rumors.

MVM relies on the runtime system to enforce security isolation. In a system like Android that allows unrestricted loading of native code, this scheme can't work, since there's no way to get arbitrary native code to play along with the runtime security model.

Personally, I feel much more confident with the kernel enforcing application isolation than I would feel about relying on the Java security model.

As a sibling comment pointed out, Android already does this. To learn (an insane amount of) more detail about how this works, read this article I wrote a while back on how Process Loading on various systems is optimized.

http://www.cydiasubstrate.com/id/727f62ed-69d3-4956-86b2-bc0...

It's not the same thing at all. Android first makes a process template, then takes clippings for each process it wants to run. That's not the same as running different applications in the same process.

The person who worked for Sun/Oracle, the one who first mentioned this technique earlier in this thread, was quite clear that the technique involves loading a single VM and then "fork[ing] with a lot of the JITted code already in place and reasonable process isolation". You are, of course, correct that this is not the same as "running different applications in the same process", but that is not the technique that was described.

> The trick is to share one VM instance between multiple apps. So when the OS starts you start the VM process and then fork with a lot of the JITted code already in place so you get almost instant startup and reasonable process isolation.

I missed that last part. Thanks for the correction.

FYI that's how we implemented MVM in CDC mobile which isn't necessarily applicable for hotspot.

The MVM RFE for hotspot has been something desktop developers have asked for since JDK 1.2 days. It sort of got sidelined when the JDK started sharing loaded classes (the rt.jar) which is important but not quite MVM. Even the weakest of Android devices is probably more powerful than our "smartphone" targets in terms of heap. Using process isolation was often not an option back then since common OS's at the time (symbian) didn't really have isolation.

That was a great read! Thanks for sharing!

Android does this, unless it changed recently. The common parent process is called "Zygote".

See eg http://www.elinux.org/Android_Zygote_Startup

HotSpot has something called "Class Data Sharing" these days, in which pre-processed data is mmapped into the VM and shared between VMs as a result. More and more stuff is being put into the AppCDS files with the result that the VM has to do less and less work to start up. The jigsaw work also includes stuff like this via the jimage file format.

> I can picture Google's lawyers using that in court against Oracle.

Google chose not to use OpenJDK (and they've obviously not complied with its license so far), so it being open source is irrelevant.

> WebAssembly doesn't have a GC, and probably won't for some time.

Yeah... because it's assembly. That's the whole point. You don't WANT a GC. It would be the wrong layer to place a GC in. Ask most professional game developers if they would consider using a platform with a GC. They won't, because you can't build a realtime system without stuttering if you're dealing with a GC.

1000s of games have shipped with gc. Every Unreal and Unity game uses gc. Every XNA game used gc. Many of the largest most popular AAA games use gc. So yes we can ask professional game devs if they want a gc. Most will say "yes". Like any tool you know when to use them and when not to

>Many of the largest most popular AAA games use gc

Like, actual GC and not smart pointers? I would like a list.

These days a great deal of game logic is scripted in a language with GC: C#, JS, and Lua mostly.

https://wiki.unrealengine.com/Garbage_Collection_Overview

https://en.wikipedia.org/wiki/List_of_Unreal_Engine_games

TSubobjectPtr are smart pointers, but regular pointers will still be garbage collected if they are UPROPERTY() and in a class derived from UObject

Tim Sweeny talks about GC a bit in https://www.st.cs.uni-saarland.de/edu/seminare/2005/advanced...

The presentation is a little dated, but they're still using GC, so...

Further, a lot of shared data structures in super high perf C++ end up with "Free Lists" to allow multiple threads to perform atomic delete operations on the data structure. Point being that memory is no longer freed deterministically.

"If there's other threads working here, atomic compare and swap this pointer to the free list, otherwise last one out clean up."

Take a look at some of the code from Chapter 7 from Anthony Williams "C++ Concurrency in Action" [0].

[0] https://manning-content.s3.amazonaws.com/download/0/78f6c43-...

In most of those cases, the performance-sensitive game engine itself does not use GC; GC is used in the scripting language that implements the high-level game logic.

I wasn't saying game developers never use GC's, I was saying they don't want the platform to make that decision for them. I'm not against GC at a language level, but I am against it in something that's supposed to be "assembly". It's just the wrong layer to place a GC in.

Anyway people are getting caught up in games, I'm just using that as one example of many where a GC may not be wanted.

> WebAssembly probably won't [have a GC] for some time

Citation needed :)

> It also doesn't have ~20 years of performance tuning on different microarchitectures the way the JVM does.

JS engines have been tuned very well over the years. Remember that Web Assembly is designed to run in a JS engine.

> Remember that Web Assembly is designed to run in a JS engine.

Is it? It's a very low-level bytecode that's designed to be efficiently compiled in one pass, and completely unrelated to the JS engine. It isn't meant (at least at this time) to have an optimizing JIT (like HotSpot or V8). It is meant as a good target for languages that don't rely on/can't benefit greatly from good JIT optimizations, such as C/C++ (or Rust).

Of course, one could compile a JVM to wasm, provided that wasm allows applications to write to executable memory (which I doubt).

It is (though that's not the only thing it's designed for), but in a JS engine setting it only uses a few specific parts of the JS engine, such as the JIT backend to do the actual code generation work and interoperability with JS.

Indeed, wasm won't likely ever expose directly executable memory. But in the future it may expose JIT capabilities like creating and calling new functions, declaring "patchable" code fragments which can be manipulated through APIs, and so on, and with these capabilities it isn't unreasonable to think about porting JITing VMs onto wasm.

> The JVM is already open source.

From what I understand open source and imunity from litigation are orthogonal. Open source code and still be a patent minefield.

OpenJDK has a full patent grant[1]. It is 100% open source in every way possible, its use is unrestricted, and it is released under the same license as Linux. AFAIK, Oracle has never sued anyone making use of it for whatever purpose whatsoever. It is important to remind the undisputed fact that -- at least so far -- Google has chosen not to use OpenJDK (or, in any case, they did not comply with its license), and therefore its open source status is irrelevant to the court case.

[1]: That grant is automatic, by the open source license. In addition (and unrelated to OpenJDK), there is an explicit patent grant to conformant implementations of Java (be they based on OpenJDK or not).

I see, good point.

I was saying in general, if something is released as open source, that doesn't usually come with a default immunity from patent litigation.

Of course, if you don't conform, Oracle will sue you into oblivion. Java is free software as long as you use it in exactly the way Oracle wants, right? It's for this reason that Java is poison to any of my projects. I'll choose Node-fucking-JS over Java, because nobody is going to sue me for using JavaScript the way I want.

No one expects JavaScript to run unchanged across all VMs anyway.

> Of course, if you don't conform, Oracle will sue you into oblivion.

What? OpenJDK is as free as they come. No conformance with Java necessary, you can do with it whatever you damn well please. You can use it to implement .NET if you want. Oracle has never (to the best of my knowledge) sued anyone for the use (or modification) of OpenJDK.

Okay, so can I modify it so that it doesn't conform to the Java specification? Can I modify it bit-by-bit until it's bytewise identical to Harmony? Of course I can't. Because if I do, Oracle will sue me.

Oracle is lying. They claim to be offering free software, but will sue you once you take advantage of that freedom. It's a fucking trap.

Do you really expect the technology community to embrace Oracle technology after Oracle spits in their food and shits in their sink?

> Okay, so can I modify it so that it doesn't conform to the Java specification? Can I modify it bit-by-bit until it's bytewise identical to Harmony?

Absolutely (on both counts, although if it is identical to Harmony you may be in violation of the license due to a collision between GPLv2 and ASL, as ASL imposes further restrictions which GPLv2 does not allow -- see the next paragraph).

> Because if I do, Oracle will sue me.

They will not. In fact, they explicitly allow you to do whatever you want with it. They are not placing any restrictions; their own license (same as Linux's) does not allow them to do so (section 6 says: "You may not impose any further restrictions on the recipients' exercise of the rights granted herein").

Oracle sued Google, not you. They didn't sue them for using Java, and they certainly didn't sue them over OpenJDK. Of course, it has been Google's PR department's strategy to make you think that the lawsuit may apply to you, but it doesn't. The circumstances leading to the lawsuit were very unusual.

> Oracle is lying.

Maybe about other things (I don't know a large company that doesn't lie), but not about this.

> They claim to be offering free software, but will sue you once you take advantage of that freedom. It's a fucking trap.

Again, to the best of my knowledge, that has never happened.

> Do you really expect the technology community to embrace Oracle technology after Oracle spits in their food and shits in their sink?

I don't expect anyone to do anything other than to understand the facts and then make their decisions[1]. Oracle has earned the distrust of many developers, but I don't find Google to be any more likable. Personally, I'll take Oracle's old-school greed over Google's sneaky espionage and manipulative PR, but that's just me.

----

[1]: I am willing to bet that very few people understand what the Oracle v. Google court case is all about, and the loudest voice was by far Google's PR.

I am biased towards Oracle because I do want Android to be closer to orginal Java and wish both companies collaborate for future Java versions. However I do agree that Google's PR has tried to picture Google as victim. If we see Google specifically didn't choose OpenJDK to avoid GPL. So they wanted to make profits but avoid collaboration. If this makes them bring to use GPL, let it be.

> If we see Google specifically didn't choose OpenJDK to avoid GPL.

As far as I know, when Android started to be developed, OpenJDK didn't exist

You're right, but it existed early enough (before Android's release) to be adopted by Android. Here is the relevant timeline -- http://www.pcworld.com/article/253666/a_timeline_of_oracles_.... OpenJDK was released in May 2007.

> [1]: I am willing to bet that very few people understand what the Oracle v. Google court case is all about,

And what it was about?

First, we should separate the motivation for the lawsuit from the legal arguments. Corporations don't go to court to win legal arguments; they go to court to defend or promote their business.

I) The motivation for the lawsuit was that at the time, Java's main revenue stream was licensing Java IP (either an actual implementation or the permission to implement the spec) for use in mobile devices. That was how Sun made money off of Java. Java's commercial license allowed people to freely implement the specification as long as they pass the compatibility test, unless the implementation was for use in mobile devices -- this required a paid license. Google then created an implementation of (a part of) the spec, and didn't reach a licensing agreement with Sun (they couldn't agree on a price; I think Sun wanted $25M for the first five years or something). Google used Java IP to directly attack said IP's main revenue source. That is not to say that what they did wasn't legal, but that is why what they did mattered to Sun (and then Oracle) enough to sue them. Google needed Java because developing a language ecosystem suitable for phone application takes years. Google knew that such a lawsuit is forthcoming -- you can't come at a company like that and not expect one -- but hoped that they could defend their actions in court, at least on legal grounds. So that's what the lawsuit was about.

II) Now for the legal case. Oracle wanted to get their (in their view -- fair) share of Android no matter how, so they used -- as anyone does -- any legal argument they could think of which would get the desired business result, from patents to copyright. Some stuck, some didn't. In the end, it was the API copyright argument that convinced the (appeals) court. But before I say a few words about that, let me reiterate: Oracle didn't set out to copyright APIs; they set out to get a share of Android, which attacked their main Java revenue stream directly using their own IP. Anyway, Google claimed that APIs aren't copyrightable, but Oracle won that round. Now, I will not go into this question itself -- there are good arguments for both sides -- but only into its ramifications. Google started a massive PR campaign to convince people that that ruling is disastrous for them, but it isn't, and for several reasons.

First, it applies only to language-level APIs; not REST protocols, recently also referred to as APIs. This is not open to interpretation. Web "APIs" are not copyrightable for completely other reasons, reasons which are impossible (as much as anything in law is impossible) to argue against. Second, the court ruled that some of Google's argument regarding the copyrightability of APIs are really fair use arguments, and this is the second crucial point: if something is copyrighted, that does not mean you may not freely use it for some purposes. When it comes to interfaces, one fair use defense has been interoperability[1]. Courts have ruled it legal to "violate" copyright for the purpose of creating components that are interoperable with the original. This means that if you implement an API in order to break vendor lockin, you're probably within your right to do so. However, that is not what Google did in this particular case (Android is not interoperable with Java, only very, very partially). So while Google may still lose the fair-use question (which is still on the table), nearly all other implementors would still be protected by fair use.

To sum up the legal issue, while it has been in Google's best interest to cause panic, the ruling does not apply to any other player but Google because:

1. It only applies to language-level APIs; it does not and cannot apply to web APIs and other protocols (either software or hardware).

2. Implementing an API is protected by the interoperability fair-use defense, even if Google's particular (and highly unusual) use isn't.

And in any case, unless you use a company's (language-level) API against it, there is little chance they'll come after you; if you do, well, you knew they'd come after you even before this court case.

III) Finally, it is important to note that all along Google had the option of freely using OpenJDK. The use of OpenJDK does not require conformance with the Java standard; it is completely unencumbered. The reason Google opted not to do it was because at the time they were afraid phone manufacturers would not like OpenJDKs license, GPLv2 + the classpath exception, even though Android also make use of Linux, which is also licensed under the GPL. This reasoning seems to have changed now.

[1]: https://www.cs.duke.edu/courses/cps182s/fall02/cscopyright/C...

I think after a legal counsel it was impossible for Oracle not to move to courts. Else others would create spin-off's from Java and cite Android as an example.

> Google then created an implementation of (a part of) the spec, and didn't reach a licensing agreement with Sun

If Google used Apache Harmony, the ones implementing it where Apache Foundation. Why Oracle didn't sue Apache?

> hat is not to say that what they did wasn't legal, but that is why what they did mattered to Sun (and then Oracle) enough to sue them

Sun didn't sue Google in the 5 years Google used Android

> First, it applies only to language-level APIs; not REST protocols, recently also referred to as APIs. This is not open to interpretation. Web "APIs" are not copyrightable for completely other reasons, reasons which are impossible (as much as anything in law is impossible) to argue against.

And exactly why is different? The ruling talks about API and API's has nothing to do with being a language level API or a "web API".

> Second, the court ruled that some of Google's argument regarding the copyrightability of APIs are really fair use arguments"

As far as I know, the appeal court ruled that the fair use question must be decided in the forthcoming case, nothing more. As any other copyright case

> To sum up the legal issue, while it has been in Google's best interest to cause panic, the ruling does not apply to any other player but Google because: >1. It only applies to language-level APIs, not web APIs.

You have to make your case why only applies to lkevel language API's

> 2. It is protected by the interoperability fair-use defense, even if Google's particular use isn't.

No, it has to go to court so yes, any player must be concerned

> Why Oracle didn't sue Apache?

Simple: because they didn't have the motivation (see point (I)). Their goal wasn't to copyright APIs (that just was one of the many tactics used in court, and the one that just happened to win), but to get their share of Android. Suing Apache wouldn't have helped their goal in the least.

Also, while not relevant to the question, I will note that even if APIs are copyrighted, Harmony has a much stronger fair-use defense than Android: 1. It was done for the purpose of interoperability and 2. it was not intended to make a profit. Each of these is a fair use defense alone, and certainly together, but neither is true for Android (although the first might be partially true).

> Sun didn't sue Google in the 5 years Google used Android

Because they didn't have the resources and the inclination, as the company was already in a downward spiral and such a lawsuit could only have hurt them at the time. Google, however, fully expected them to.

> The ruling talks about API and API's has nothing to do with being a language level API or a "web API".

That's only because the term "API" -- until very recently -- was only used to describe language-level API, while what today is known as a web API has always been called a protocol, so don't let the recent overloading of the term confuse you.

More to the point, the reason copyright may apply to APIs but certainly not to protocols is because copyright law is not only concerned with the intent, abstraction, or use of the work, but also with its form. This why other IP protections (like patents) exist, to cover works whose form does not match those covered by copyright.

In particular, copyright requires a fixed representation (in text, image, recording etc.). So, for example, if I tell you a story, the story isn't copyrighted. But if I write it down -- or even record it -- then the text of the story or the recording is copyrighted. It doesn't matter that the purpose in all three cases was the same: to convey a story (once that fixed form is copyrighted, the protection may extend to non-exact reproductions, but that original fixed-form must exist). While a protocol may be described in a text and that text is copyrighted, the described protocol itself cannot be, because it does not have a fixed representation in text (a completely different text could describe the same protocol). A language API does have a fixed form.

> the appeal court ruled that the fair use question must be decided in the forthcoming case, nothing more. As any other copyright case

Yes, but that was the point. Google made some arguments that they believed would make APIs non-copyrightable. The court ruled that those arguments are fair-use arguments which are not relevant to the question of copyrightability, but may be used as a defense once APIs are copyrighted.

> No, it has to go to court so yes, any player must be concerned

Quite the opposite. It is the copyright holder who has to sue you. Any company would sue you if they believe they have a case, and you would still need to defend yourself in court when they do. The question of API copyrightability had never been decided, so even before the ruling there was a good chance you'd be sued, and you'd still need to defend yourself. Now, it is true that a ruling against API copyrightability would have removed this threat (which had existed even prior to this case), but now your position is not much different than before, only that except making two arguments (as Google did) -- that API are not copyrightable and if they are, you've made fair use of them -- you only have one: fair use.

> That's only because the term "API" -- until very recently -- was only used to describe language-level API, while what today is known as a web API has always been called a protocol, so don't let the recent overloading of the term confuse you.

A protocol is not the same as an API and I don't hink that the, for example, Google Maps interface is a protocol

> More to the point, the reason copyright may apply to APIs but certainly not to protocols is because copyright law is not only concerned with the intent, abstraction, or use of the work, but also with its form. This why other IP protections (like patents) exist, to cover works whose form does not match those covered by copyright.

But a web API is not a protocol and a communication protocol is not what is used to call Google Maps, beeminder or other web applications.

> Yes, but that was the point. Google made some arguments that they believed would make APIs non-copyrightable. The court ruled that those arguments are fair-use arguments which are not relevant to the question of copyrightability, but may be used as a defense once APIs are copyrighted.

NO, the appeal ruling had nothing to do with Google arguments about fair use. If there is a new trial to rule about the fair use is because in the original trial, the ruling about fair use was stalled by the jury and Judge Alsup stated that API's can't be copyrighted. And the Appeal court didn't wanted to rule about fair use

> Quite the opposite. It is the copyright holder who has to sue you.

Yes, but fair use is not automatically granted, so, you're in danger to be sued by the copyright holder. Microsoft can sue WINE developers or Amazon can sue anyone developing a clone of the AWS API.

> A protocol is not the same as an API and I don't hink that the, for example, Google Maps interface is a protocol

However you want to call it, unlike a "proper" API, it is not of the right form to be copyrighted, regardless of the ruling pertaining to "proper" APIs.

> NO, the appeal ruling had nothing to do with Google arguments about fair use.

The court didn't rule on fair use, but it did address it. They said that now that it has been clarified that interoperability is not relevant to the copyrightability analysis, it may still play a role in connection with fair use. Namely, that Google interoperability argument isn't an argument against copyright, but rather a fair-use defense, which should be argued separately (http://www.fosspatents.com/2014/05/oracle-wins-android-java-...).

> Yes, but fair use is not automatically granted, so, you're in danger to be sued by the copyright holder

That danger had existed even before the court case; the ruling simply did not lift it. Fair use is granted neither automatically nor in court. It is a defense to be used if you are sued.

> However you want to call it, unlike a "proper" API, it is not of the right form to be copyrighted, regardless of the ruling pertaining to "proper" APIs.

Still waiting an explanation about web API's not being a proper API. They are not protocols

> That danger had existed even before the court case; the ruling simply did not lift it. Fair use is granted neither automatically nor in court. It is a defense to be used if you are sued

So anyone is on danger of being sued by a copyright holder of an API.

> Still waiting an explanation about web API's not being a proper API.

It's not that they're not APIs (you can call anything you want an API -- copyright law doesn't care what it's called), but because they don't have a fixed form. In order to copyright something you need it to be a particular fixed text (or image, or recording). There is no such text for web APIs. You can come up with multiple texts (or images) all describing the exact same API. That is not the case for "plain" APIs. Plain APIs -- like all code -- do have a fixed form. But no fixed form means no copyright.

The very beginning of the definition of US copyright says[1]: "Copyright protection subsists, in accordance with this title, in original works of authorship fixed in any tangible medium of expression, now known or later developed, from which they can be perceived, reproduced, or otherwise communicated, either directly or with the aid of a machine or device" (emphasis mine). There are other requirements, but without a fixed form no copyright may apply[2].

The bits of text comprising the names of JSON fields, while fixed, are too short (and lack the necessary structure) to be considered "a work" for copyright purposes.

> So anyone is on danger of being sued by a copyright holder of an API.

That has always been the case (because it had never been settled that code APIs are not copyrightable). A different ruling may have removed that particular threat, but it didn't. It certainly didn't add new legal threats.

But let me give you this advice: If you're a big company with deep pockets and you're coming after the income of another big company with deep pockets using something that may be law-related -- expect to be sued. In fact, I don't need to give you this advice because if you're a big company with deep pockets you probably know this, as you're already in the habit of suing and/or being sued all the time.

[1]: https://www.law.cornell.edu/uscode/text/17/102

[2]: The law also says: "In no case does copyright protection for an original work of authorship extend to any idea, procedure, process, system, method of operation, concept, principle, or discovery, regardless of the form in which it is described, explained, illustrated, or embodied in such work", which is the source of Google's argument that copyright doesn't even apply to actual APIs, but that's a whole other discussion, and is far from simple. E.g. a book's table of contents could also be argued to be a procedure or a concept, yet it is certainly copyrighted. Oracle said that an API is like a table of contents. There are good arguments for arguing either way. But again, not having fixed form stops the discussion in its tracks.

> It's not that they're not APIs (you can call anything you want an API -- copyright law doesn't care what it's called), but because they don't have a fixed form. In order to copyright something you need it to be a particular fixed text (or image, or recording). There is no such text for web APIs. You can come up with multiple texts (or images) all describing the exact same API. That is not the case for "plain" APIs. Plain APIs -- like all code -- do have a fixed form. But no fixed form means no copyright.

You're joking, isn't?

> The bits of text comprising the names of JSON fields, while fixed, are too short (and lack the necessary structure) to be considered "a work" for copyright purposes.

What are you talking about? Do you really know what an API is and how web services are defined? You're confusing argument names with the API definition.

Yes, it's utterly laughable. The sad thing is this isn't just him. This is actually what Oracle is arguing and the hoops they are jumping through to try and copywrite APIs for what they want but try and get around the very obvious problems.

You also can't label APIs as 'language level' either. It's ridiculous.

> This is actually what Oracle is arguing

Not at all. Oracle is arguing nothing regarding web APIs, and I'm arguing nothing about library APIs. My argument's and Oracle's do not overlap. Oracle is saying that library APIs are copyrightable, and I am saying that even if they are, web APIs cannot be.

> You also can't label APIs as 'language level' either. It's ridiculous.

You don't need to label APIs in any way (and what you find ridiculous has little bearing on the law). The relevant question to this particular discussion is whether what you're trying to copyright is a piece of text or not. A library API -- like all code -- is text. A web API isn't. If there's no fixed text to copyright, all other discussions are moot.

You could also claim that copyrighting code while not copyrighting algorithms is ridiculous because in your mind they are the same. However, as far as copyright is concerned, an algorithm can be described in many different words -- and therefore there is no fixed form to copyright -- while a piece of code is fixed, and therefore could potentially (and, in fact, is) be copyrighted.

What piece of (substantial) text is replicated when you implement someone else's web API? If there isn't one then there is nothing to copyright.

The question of how web services are defined is not material. The question is whether a "API" = "fixed text" or not. For library APIs, the answer is yes, for web APIs the answer is no.

It is clear that you don't j=know what an API is if you say that a web API like Goole Maps or Beeminder is not a library API.

Or perhaps you're just trolling

For the purpose of this discussion there's no need to argue what an API is because it matters little. Like I said, it is like arguing whether a story is copyrightable or not. It turns out that whether something is a story is not the relevant question, but rather the (first) question is how is that thing (call it a story or foo) recorded in fixed form in tangible media. Java's APIs are recorded in fixed and tangible form; the Google Maps API isn't. Again, what it is may matter when you start asking further questions, but the very first thing in the US copyright code says that in order for something to be eligible for copyright it must be fixed in tangible media.

Just to make clear: I am not arguing that Java's APIs is copyrighted, just that there is good reason to see why -- from the perspective of copyright law -- the Google Maps API isn't, regardless of the status of the Java API. They could be absolutely identical from your perspective, and yet different enough from the law's perspective due to some minor detail that you -- but not the law -- may consider inconsequential.

You're saying that I can do whatever I want with OpenJDK until it cuts into Oracle's revenue stream, at which point Oracle will sue me and keep suing me until they find an argument that shuts me down?

No thanks. Free software is based on irrevocable legally-granted rights, not some horrible company's largesses. If OpenJDK really is free software, I should be able to hack it such that it no longer conforms, stick it in a phone, and ship millions of copies without being sued.

Google's case isn't unusual at all. The same logic applies to anyone reimplementing an existing API with anything but bug-for-bug compatibility. Your view means that we can improve nothing, delete nothing, and deprecate nothing, since as soon as an API implementation diverges from the original, it's suddenly not for "interoperability", but rather an illegal appropriation of somebody's revenue stream? That argument is blatantly inconsistent with free software principles.

I'm going to give you the benefit of the doubt and assume that you're just an ardent Oracle fan, not a paid shill. It's really hard to tell the difference sometimes. Please reconsider the effect Oracle's litigation has on the entire software ecosystem. Consider that we wouldn't even have Unix if its view prevailed. You should be ashamed of yourself.

> You're saying that I can do whatever I want with OpenJDK until it cuts into Oracle's revenue stream, at which point Oracle will sue me and keep suing me until they find an argument that shuts me down?

Not at all. I'm saying that you can do whatever you want with OpenJDK/Java, but if you don't abide by either of their licenses -- the commercial or the open source license -- Oracle will come after you, but probably only if you pose a serious threat to them, or else they wouldn't bother. Do you know of a company of Oracle's size that acts any differently? In fact, Google expected the timid Sun to come after them, and it was only Sun's decline that stopped them.

> Free software is based on irrevocable legally-granted rights, not some horrible company's largesses.

It is, but free software does have a license, and you don't get those rights until you comply with the license. That applies to OpenJDK just as it does to Linux, both released under the same license, BTW. Google is adopting Java's open-source license just now. They had their reasons, but they most definitely did not until now.

> I'm going to give you the benefit of the doubt and assume that you're just an ardent Oracle fan, not a paid shill

I am far from a fan. Oracle is a terrible company, and I wish it was a different company that had bought Java. But Google is worse. So I guess you could say that in this case, Oracle is the lesser of two evils. But what really bugs me is that the huge Google PR machine has done everything in its power (and quite successfully) to confuse everyone about this case, and convince people that the lawsuit has any bearing on them whatsoever. The ferocity Oracle's legal team is legendary, but it doesn't come close to the legendary manipulativeness of Google's PR. Employing paid shills is much more a Google thing than an Oracle thing; Oracle prefers lawyers.

I was, however, a fan of Sun.

> Please reconsider the effect Oracle's litigation has on the entire software ecosystem.

And what effect is that? I know that Google tries to convince people it has far-reaching effects, but the actual ruling doesn't seem to imply any of that.

> Consider that we wouldn't even have Unix if its view prevailed.

That's just Google PR talking. Let me make this short and hopefully clear: Google copied part of an Oracle/OSS API and broke compatibility in order to make a profit. They didn't try to build an interoperable alternative to Sun's Java, but to get into a ready-made market of Java tools and developers. The question of API copyrightability had never been settled, and whoever would have come after Unix after the ruling would (and certainly could) have done so before. Even if APIs are copyrightable, there's still fair use, two of those defenses -- interoperability and non-profit -- would apply to most implementations, but possibly not to Google's.

Sure, maybe APIs shouldn't be copyrighted (again, this is only language-level APIs) -- I can see good argument for either side -- but the effect of their copyrightability on the software industry is at worst minimal, and is most likely nil because of a thing called "fair use". I am not aware of another case where a replication of an API was done both not in order to build a compatible product and for profit.

Besides, their "view" wouldn't have even been made (as I said, Oracle didn't set out to make a legal point) if Google had only licensed Java in one of the two available ways, one of which they are finally doing now and the other they could have probably afforded at the time. Google knew full well that Sun would come after them, but hoped they could win in court.

> That's just Google PR talking. Let me make this short and hopefully clear: Google copied part of an Oracle/OSS API and broke compatibility in order to make a profit. They didn't try to build an interoperable alternative to Sun's Java, but to get into a ready-made market of Java tools and developers.

What's wrong with copying part of an API? You yourself admitting that I can do whatever I want with OpenJDK, including copying only part of it. I can then use this copy to make a product that turns a profit. If Oracle will sue me to this practice, then Google, not Oracle, is on the right side here.

> What's wrong with copying part of an API? You yourself admitting that I can do whatever I want with OpenJDK, including copying only part of it. I can then use this copy to make a product that turns a profit.

Absolutely, as long as you comply with the open source license, which is plain GPL in this case (just like Linux). Google didn't -- until today -- as they released Android under the Apache license rather than the GPL license. They always had the option of going GPL -- in which case Oracle would have not come after them -- but they didn't want to[1].

> If Oracle will sue me to this practice, then Google, not Oracle, is on the right side here.

Google did not comply with the license (either license, actually). Their legal claim was that they don't need to comply with any license, because if APIs aren't copyrighted, then they're not Oracle's to give (i.e. to license). That legal claim, too, is perfectly reasonable (as is its opposite), but it seems clear to me that it was Google who was at least ethically, if not legally, wrong in this case.

In any event, Java is also licensed with a standard open-source license (through OpenJDK), that you can use -- wholly or partially -- for any purpose whatsoever; Oracle has never sued anyone for using its code or APIs in this way for whatever purpose.

---

[1]: Which is why people in the free software community, even though they believed copyright shouldn't apply to APIs, were pissed off at Google for going to great lengths (and basically asking for this lawsuit) all in order to avoid the GPL option, which has been open to them all along.

For example, here is what the FSF had to say in 2010 on the matter (they were against Oracle's lawsuit because one of Oracle's tactics was to claim software patent infringement, which the FSF is against, as am I):

"Oracle's lawsuit threatens to undo all the good will that has been built up in the years since... Unfortunately, Google didn't seem particularly concerned about this problem until after the suit was filed. The company still has not taken any clear position or action against software patents. And they could have avoided all this by building Android on top of IcedTea [an early OpenJDK distribution], a GPL-covered Java implementation based on Sun's original code, instead of an independent implementation under the Apache License... It's sad to see that Google apparently shunned those protections in order to make proprietary software development easier on Android." (emphasis mine. Source: https://www.fsf.org/news/oracle-v-google)

I will add that while Oracle has treated some of its acquired open-source projects abominably -- e.g. OpenSolaris :( -- it has been very good to OpenJDK (more than Sun), so much so that virtually all development on the runtime at Oracle is done on OpenJDK these days. Also, Oracle still maintains more open source software than Google (and possibly any other company save for RedHat), but I'm not too hopeful about its future treatment. However, OpenJDK is now way too big for Oracle to ruin even if they tried.

You have a few things ass backwards with a fundamental lack of technological knowledge on more than a few points (to put it mildly), which is not unusual for lawyers and won't be for Oracle either.

> Google needed Java because developing a language ecosystem suitable for phone application takes years.

Google did not create a language 'eco system' from Java. They created an entirely new one. The only resemblance it has to Java is the fact that a few APIs look the same, and they presumably did this to help developers and phone manufacturers with familiarity and existing code. This is done every other day of the week in the software industry and presents no problem.

There is a fundamental comprehension problem about what the difference is between a set of APIs that a language uses, the language itself and its implementation.

> Anyway, Google claimed that APIs aren't copyrightable, but Oracle won that round.

The concept of APIs has been done and dusted before in the 80s and 90s and it will be again. If APIs are copywritable then there is no software industry. What's amusing is they they knew this was a problem because they had to build in a laughable fair use clause for interoperability. There is no situation where an API isn't used for interoperability and code reuse.

I think you need to re-read this back, carefully, and comprehend how utterly idiotic trying to separate 'web' APIs from 'other' APIs in terms of copywrite actually is. There is no difference and pigeon-holing this is plain laughable. The fact that you mention this shows that you, and probably Oracle, know what huge problems this causes. The whole argument is completely null and void and I'm afraid it very much is open for debate. Once again, we have legal people not knowing what they're talking about.

> Courts have ruled it legal to "violate" copyright for the purpose of creating components that are interoperable with the original. This means that if you implement an API in order to break vendor lockin, you're probably within your right to do so. However, that is not what Google did in this particular case (Android is not interoperable with Java, only very, very partially).

I'm afraid this is yet another piece of laughable misunderstanding and shows how shaky Oracle's legal case actually is. No one is under any obligation to implement a complete API. For example, there are many C standards and many implementations only implement a subset of a particular standard with a long list of caveats - and that's from compilers that claim to be C compatible. Implementing a subset of an API, taking code and compiling it on a new platform happens every other day of the week in software.

To attempt to claim that Google has no fair use argument because they didn't implement all of Java's APIs is completely laughable. There are a long list of exceptions going back decades that Google can use to completely invalidate this nonsense.

However, that's the big problem Oracle has. They have had to attempt to try and pigeon-hole what Google has used in Android as Java by jumping through a lot of nonsensical hoops.

> while it has been in Google's best interest to cause panic

There should quite rightly be panic. This means there is no software industry, and it goes way beyond Google and Oracle.

> It only applies to language-level APIs; it does not and cannot apply to web APIs and other protocols

This is complete tripe. It is impossible to pigeon-hole APIs in this manner because they exist at many different levels, and even this ridiculous definition still causes fundamental problems.

> Implementing an API is protected by the interoperability fair-use defense, even if Google's particular (and highly unusual) use isn't.

There is nothing unusual about what Google has done, but you would know that if you knew what you were talking about. Sadly, we have a lot of people, mostly legal, who simply don't.

As for what Google are doing with OpenJDK, I think it probably has more to do with tidying up their platform and multiple Android implementations more than anything else. Curiously, using OpenJDK and overriding core APIs with Classpath would also give them a pretty good example as to how ridiculous this whole thing is.

> You have a few things ass backwards

You may think this is the case and are entitled, of course, to your opinion, but:

> with a fundamental lack of technological knowledge on more than a few points (to put it mildly), which is not unusual for lawyers and won't be for Oracle either.

I am not a lawyer nor work for Oracle (or Google for that matter). Neither have I ever worked for Sun, Oracle or Google. However, I am a software developer and have been one for over twenty years. I have also collaborated with both Oracle and Google on OpenJDK (yes, Google has been collaborating with Oracle on OpenJDK throughout most of the court case, as much of Google's infrastructure is built upon OpenJDK).

> This is done every other day of the week in the software industry and presents no problem.

That's news to me, and I am quite familiar with the industry. I am not aware of many more cases where such a wholesale copying (without license compliance) of an API has taken place for reasons other than source/binary compatibility. There are a few (UNIX is a famous case, but that's been challenged in court already), but none recent. Again, regardless of how ridiculous, laughable or preposterous you think copyright law is, it most certainly does not apply to languages (like SQL or the Java language itself) nor to any other algorithm, protocol or API (as in web APIs) that does not have a fixed representation in text.

> I think you need to re-read this back, carefully, and comprehend how utterly idiotic trying to separate 'web' APIs from 'other' APIs in terms of copywrite actually is.

Again, the matter of whether you think this is idiotic or whether this really is has absolutely no relevance to the law. If I tell you a story it is not copyrightable; if I write the same story down, it is. If I record myself while telling you the story, it is also copyrightable. The same story, serving the same purpose may or may not be copyrighted depending on the very particular form it takes. You may think this is idiotic (although you may be persuaded otherwise if you studied the purpose of copyright vs other forms of legal IP protections), but that's how it's always been.

> I'm afraid this is yet another piece of laughable misunderstanding and shows how shaky Oracle's legal case actually is.

I don't know if you've noticed, but I'm not arguing Oracle's case. I think there are good arguments to copyright APIs and good arguments against, but I really -- in all honesty -- don't have a final opinion on the matter. You think Oracle's case is laughable -- fine. My point is that no matter if Oracle is right or wrong, from the perspective of copyright law a ruling on library APIs does not translate to web APIs regardless of how similar or even identical you believe they are. Copyright law is very specific about the forms protected works may take. If a work is a specific, fixed text it may be copyrighted (if it meets some other conditions); if not -- it may not.

> This means there is no software industry, and it goes way beyond Google and Oracle.

That's your opinion. However, the ruling does not apply to any API or protocol that is not recorded in fixed media, and interoperability has been accepted as a fair use defense before. I am not aware that the software industry is predicated on copying and re-implementing significant APIs (not a function here and there, which is definitely fair use) for purposes other than interoperability.

> Curiously, using OpenJDK and overriding core APIs with Classpath would also give them a pretty good example as to how ridiculous this whole thing is.

Maybe. I know that I and others wanted them to do this before the lawsuit ever happened, because everybody saw it coming (once Sun's fate was decided), but Google didn't listen. They preferred getting sued (which they had anticipated) over using GPL software.

All code can be a patent minefield. I predict we'll see patent trolls going after browser makers as soon as WebAssembly becomes popular. The fact that Java has a single, notoriously litigious owner actually works in its favour here - anyone trying to patent-troll Java will be issuing an invitation to Oracle's attack dogs, whereas WebAssembly could easily suffer a tragedy of the commons where no one browser vendor wants to spend the money to defend it.

http://www.w3.org/Consortium/Patent-Policy-20040205/#sec-Req...

That only covers patents held by members of the working group.

"a VM that clearly has restrictions and is owned by a litigious organization"

Indeed - if there's one thing that this case has shown it's that java is poison legally. If a company the size & power of Google is being forced to take a project the size & importance of Android places they really don't want to technically, well, that's a very big deal. And I wouldn't like to think what would happen to any smaller group that wanted to take a java technology down a path that isn't Oracle-approved.

No. Variety breeds better products and WebAssembly is a pretty different thing from the JDK. You might as well say that web assembly competes with LLVM.

I don't see anyone adding Java back into the browsers, that ship probably sailed.

I also have a huge problem with the litigious nature of Oracle... However, this would mean that something was settled and might reduce the likelyhood of future lawsuits especially in regards to patents or copyright. WebAssembly didn't go thru that process which might mean that if it picks up some anonymous patent holder might start attacking it. It is vendor neutral though, which is both a plus and a legal liability as there is no single "responsible" entity.

The only advantage I can think of to putting the JVM in a browser is if you intend to run java code. (God, I hope not). It's not like they're lighting the world on fire with their performance running dynamic languages. (I mean, it's fine, but it's nothing special)

Besides, we already saw what java-in-the-browser looks like, and it was a huge failure.

> It's not like they're lighting the world on fire with their performance running dynamic languages. (I mean, it's fine, but it's nothing special)

Yes, they pretty much are[1], and yes, it is[2].

Their 80 kLOC JS compiler is on par with V8, they're matching or beating PyPy when running Python, and their Ruby performance is out of this world. The downside, though, is that it has a long warmup time, which makes it unsuitable for web pages.

[1]: https://www.youtube.com/watch?v=OUo3BFMwQFo

[2]: https://twitter.com/ChrisGSeaton/status/619885182104043520

> Hotspot is absolutely incredible technology, and it's competitors are still many years away from coming close to matching it's capabilities.

Especially when it comes to sandbox-escape security vulnerabilities! Among web technologies, HotSpot has historically beaten even Flash in that area.

This is a commit to the Android code base, what does this have to do with HotSpot, WebAssembly or even the JVM?

Thanks for ruining my holiday season. What's the precedent situation if they do settle it? Is Oracle's latest victory binding even if the final resolution of the case isn't decided by the court?

The federal circuit's opinion concerning the copyrightability of APIs is only binding precedent on the federal circuit itself and only when interpreting 9th circuit law. The FC's usual jurisdiction is over patents, not copyrights, and they only got involved in this case because Oracle originally asserted some patents that failed to find traction. In particular, the real 9th circuit would be free to draw their own conclusions should they hear a future case concerning the copyrightability of interfaces.

Or that Google has simply decided to use OpenJDK and abide by its license, an option that has always been open to them (yet they have so far rejected).

This is an interesting comment. GPL licensed code is divisible, so Google could technically achieve the same thing by adding a comment in the NOTICE file that said Api method signatures are Copyright Oracle, GPL, the rest is Apache. But the NOTICE file doesn't say anything about Oracle at all! Nor did they carry over the openjdk LICENSE file, which adds to the mysteriousness. I only did a cursory examination of the commit. Someone please post if I'm wrong about this.

> GPL licensed code is divisible, so Google could technically achieve the same thing by adding a comment in the NOTICE file that said Api method signatures are Copyright Oracle, GPL, the rest is Apache.

That doesn't work. GPL is divisible (in fact, you can do whatever you want with the code), but the license must be applied to the entire deliverable, up to (excluding) classpath-linking which is explicitly exempted by the OpenJDK license -- so in this case, the whole runtime minus Google-only packages if they're classpath-linked only.

My thought was that Google may simply choose to make Android GPL + classpath exception. The reason for not doing so when Android was young (I can only assume) was the fear that phone vendors would balk at a runtime that doesn't let them make proprietary changes (of course, Linux doesn't either, but I guess the thought was that the phone manufacturers are likely to make changes that are closer to the application). But now, given Android's popularity, phone vendors would swallow whatever license Google imposes on them (and would still be free to make proprietary changes to classpath-linked portions of the runtime).

This is a win for everyone: Google gets to expend less effort maintaining the runtime, plus they get Oracle off their back (at least for future Android versions); Oracle gets to have Java (or something close enough to it) on lots of smartphones, and the developer community gets to have true Java interop (with all new Java features), and probably a higher-quality runtime.

Later, Google would be free (but not compelled, although that depends on a future settlement) to make Android fully Java compliant with one of the Java standards, but that is an orthogonal issue.

Barely on topic, but since you're the person to ask, would the use of OpenJDK on Android mean that it'd support Pulsar/Quasar?

Hmm, this may be orthogonal, but it certainly can't hurt. Quasar already has AOT instrumentation and we're working on making it available on Android as-is. OTOH, even if Android goes OpenJDK, it's doubtful they'll support agents, so you'll have to rely on AOT instrumentation anyway.

But to get us back on point, it will certainly make the work of migrating any Java library to Android easier.

This is perhaps the most interesting comment on this topic. The implications for such a licensing change would be enormous for the development community, I believe. The days of OEM's not releasing their source code, for changes to the Android ASL parts, may finally be over.

I'm curious, though, as to why you believe Google would expend less effort developing the runtime considering the different trajectories of ART and the JVM.

> I'm curious, though, as to why you believe Google would expend less effort developing the runtime considering the different trajectories of ART and the JVM.

Because the runtime is much more than just the VM. If you look at the breakdown of the JDK source tree[1], you'll see that it's 77% Java and only 18% C/C++ (although the Android ecosystem is already making indirect use of some of that Java code, when it uses tools like javac, javadoc etc.). Also, Google might choose to use HotSpot or parts of it instead of or in addition to ART.

Of course, all of this is pure speculation (and wishful thinking) on my part.

[1]: https://github.com/dmlloyd/openjdk

> the fear that phone vendors would balk at a runtime that doesn't let them make proprietary changes

In retrospect, the ability for phone vendors to make proprietary changes to the system has been a huge problem for Google, Android as a platform, and the community in general.

This diff is more explicit about what's going on:

https://android.googlesource.com/platform/libcore.git/+/aab9...

    Change dependency from libart -> libopenjdkjvm.

There are also diffs adding lambda support, tweaking various classes for compatibility with applications that use reflection to access internal capabilities, and fixing lots OpenJDK compatibility bugs.

Android still needs to run dex bytecode somehow, so there are two possibilities for how N will work.

Option one is that Android stick with ART and replaces Harmony with OpenJDK: from a technical perspective, that wouldn't be the end of the world, especially since the Harmony implementation is rather inefficient.

Option two is that Google ports Hotspot to run on Android and, then has PackageManager convert dex bytecode back to Java bytecode on device. That would be awful, since ART is built for low-end devices and, well, Hotspot isn't.

In favor of option one is that Google is still developing ART. In favor of option two is Oracle being Satan incarnate.

I also wouldn't be surprised if Oracle has compelled Google to simply ship a copy of Hotspot, allowing developers to ship "authentic Java" APKs instead of dex-bytecode ones, with the two environments running in parallel, with two different zygotes.

dang, want to change the headline from "mysterious commit" to "Google switches Android to OpenJDK"?

Option two wasn't even worth mentioning. There is absolutely no way Google is going to abandon ART and switch to a significantly less performant VM. AOT is here to stay.

> AOT is here to stay.

No it isn't. Read the source. ART has a JIT now.

As a guy who used to work for Sun on JITted code and is now doing AOT for https://www.codenameone.com/ I've got to say that JIT always beats AOT in runtime. It can also beat it in startup when properly designed (MVM, caching etc.).

Generalizations about which technique beats the other technique reflect a lack of technical maturity. Both have advantages, and you're doing a disservice by advocating the use of one or the other exclusively ignoring differences in environment, circumstances, and workload.

The very fact that Oracle has started to address the slow startup times of JVM applications by finally working on AOT compilation is an admission of the technical immaturity of the JVM. Also, the only relevant environment here is mobile and startup times are paramount in this environment.

> The very fact that Oracle has started to address the slow startup times of JVM applications by finally working on AOT compilation is an admission of the technical immaturity of the JVM.

Not at all. If you listen to the talk introducing that work, you'll see that it is designed to address a very particular (and relatively unusual) use-case, which is important to some specific (yet lucrative) Oracle customers.

The issue they're addressing with AOT is not startup time, it's warmup time and it's intended to help out, specifically, high frequency trading companies. Currently such companies have been known to engage in risky behaviour like, um, submitting bogus trades to the market and then immediately cancelling them in order to force JIT compilation of their codebase.

The HotSpot AOT work is actually a hybrid. The AOT compiler (Graal) has several modes and to get the best performance the AOT compiled code actually is compiled with profiling code. Thus it runs slower than it could do, but the profiling data is then used to trigger further JIT compilation in order to reach the best possible peak performance. So it ends up being a hybrid approach in which JITC still features heavily, and the AOT work is done to reduce the length of time taken to reach peak throughput.

Fair. But having worked on both and currently working on AOT I'd say I should be biased for AOT not against it...

Startup time/warmup is obviously a well known JIT weakness but if its well written its just really hard to beat. E.g.

if(x) { invokeMethodX() } else { invokeMethodY() }

Say x is related to user details/preferences and for a specific install will never change... AOT can't do anything about that...

JIT can inline invokeMethodX() then optimize across methods and eliminate the if altogether. This isn't some theoretical exercise... Its stuff that exists today in JIT's. AOT provides consistency which is a good thing, you can write low level code (e.g. C can be handcoded to a level that's pretty great) so I'm mostly talking about higher level languages (Java etc.).

AOT can do most of that as well - see likely() annotations.

And it's highly unlikely the JIT can get rid of the if entirely - it needs to check that it's assumptions hold true, otherwise it needs to drop out into the none-JITted code.

Profile-guided optimization exists and is used by projects like Firefox is the real world. The advantage of JIT compilers is getting the profiling data dynamically, but that's usually going to be worse off than a well-tuned profiling suite especially if there are manual annotations.

As far as Android is concerned - AOT is here to stay. And yes, I'm aware ART has a JIT and it's probably used for devices that cannot handle the overhead of AOT, but I imagine it's seldomly used considering the specs of today's phones and the requirements of Google's CTS.

A JIT makes many tradeoffs but it is always capable of producing code at least as good as an AOT. How much better that code is depends on many factors, such as the language, the application, and how much time/energy you're willing to spend on optimization (the latter might lead to choosing to generate code that's less optimized than an AOT).

A slightly bigger difference is not between JIT and AOT, but whether you can dynamically load code at runtime or not. If not, that opens the door to some whole-program optimizations, or, at least, removes the need for guards, generated by the JIT, that are triggered when new code is loaded. In any case, mobile applications don't load code dynamically.

> the latter might lead to choosing to generate code that's less optimized than an AOT

With ART compilation happening on the device (during installation instead of runtime, but still) it suffers of the same trade-off of compilation performance vs. performance of the generated code.

> In any case, mobile applications don't load code dynamically.

Android has a DexClassLoader. I'm sure some people use it.

> it suffers of the same trade-off of compilation performance vs. performance of the generated code

Sure, which is why HotSpot may be better.

> Android has a DexClassLoader. I'm sure some people use it.

I didn't know that (not an Android dev), but that only means that some form of JITting may be beneficial anyway (depending on how popular this feature is).

Of course a JIT is capable of producing code as good as an AOT and perhaps even better since it can capture more profiling data. The problem with a JIT is the startup time and this has not gone unnoticed by Oracle as even they've started working on AOT.

Not startup but warmup (i.e. the time until the application is fully optimized). There are other problems with a JIT on small devices, such as increased memory and energy consumption (each may be significant or not, depending on how the JIT works). I am a big fan of JITs, but as with everything in software, it is a tradeoff.

Increased memory consumption and reduced page sharing between processes using the same code means lower performance. Memory bandwidth and cache capacity are a much bigger bottleneck than CPU throughput for most use cases. A JIT compiler is not competitive in performance with an AOT compiler if both are using the same code generation implementation. And profiling is limited in what it can do, especially when AOT compilation with PGO as used by projects like Firefox is considered.

> A JIT compiler is not competitive in performance with an AOT compiler if both are using the same code generation implementation.

If a JIT uses the same code generation as an AOT, it will work as an AOT, namely, only when the application starts, without additional burden during its runtime. Most good JITs use profiling information, which is collected at runtime -- this has some memory overhead, but results in better code.

> And profiling is limited in what it can do, especially when AOT compilation with PGO as used by projects like Firefox is considered.

Modern JITs are like _very_ sophisticated PGO AOTs, but they have true profiles exposed to them. Besides, the overhead for even a good JIT is not that large, it's just that on mobile devices even a bit more RAM is more than you're willing to afford.

The main benefit from ART on Android is having all of the code generated in advance and mapped from storage. There's no waste from code being generated at runtime in-memory. Wasting memory by generating code dynamically certainly hurts performance so claiming that a JIT compiler is always just as good is not true.

There's room for a JIT compiler but the sanest baseline is mapping AOT compiled code from storage. It's way faster than a baseline interpreter and way lighter than a baseline method JIT.

As a sidenote, giant commits including giant dependencies in the tree are the perfect time to include new backdoors. Nobody will know who really introduced them.

Wish people didn't do that. Separate repos are not hard to create...

> Harmony implementation is rather inefficient.

A lot of stuff have been fixed or improved from the original Harmony code over the years, performance-wise the android implementation of the core library and the openjdk implementation are now similar (openjdk has a few more intrinsics and makes a more liberal use of native methods).

> That would be awful, since ART is built for low-end devices and, well, Hotspot isn't.

That is plain Google FUD to not follow the Java standards.

There are lots of embedded devices, more constrained than Android phones, running commercial compliant JVMs like Atego, Jamaica, J9 among many others.

Also Sun/Oracle Hotspot implementations have existed since the J2ME and Embedded Java early days for devices with just a few hundred KBs, not a few hundred MB like Android.

I'm pretty sure those constrained devices are running bytecode interpreters that aren't nearly competitive with native code in terms of speed, with UIs that wouldn't cut it on even a low-end smartphone these days.

Then you should educate yourself, as they are running JIT and AOT code.

I can provide links for product documentation and real products if you wish.

pjmlp: original poster above referred to Hotspot, not arbitrary JVMs. Clearly Android devices can run VMs with both JIT and AOT (they do, after all), what OP questioned was whether Hotspot in particular (not some arbitrary VM) is a good VM on a mobile device. Given Hotspot's somewhat underwhelming startup speed even on desktop class computers, I think that's a very reasonable technical question to ask, and not FUD in any way.

Hotspot requires as little as 128 KB RAM and 1 MB ROM

http://www.oracle.com/technetwork/java/embedded/javame/embed...

If one adds the GUI client APIs, then the whole SDK grows up to 5MB ROM.

http://www.oracle.com/technetwork/java/embedded/javame/embed...

Some commercial products using the above runtimes are the Gemalto's M2M modules or evaluation boards like the ARM Keil F200.

But I expect Google fanclub to downvote me.

The document you link to is not about Hotspot, it's about Oracle's Java ME VM, which is a different product from all I can tell. It doesn't mention the term "Hotspot" at all.

Hotspot is Oracle's Java VM JIT compiler, not a separate product.

They don't produce different JITs from scratch for each VM implementation.

Thanks for links and info.

I think comparability between Java (SE or ME) and Android would be a great win fit both communities :-)

You are right.

The code for these embedded devices is JITted and I know because i worked on them. It also included some AOT but not much. In the past one of our guys benchmarked against an older version of Dalvik and had us at about 5x with those VM's.

To be fair they are not hotspot and it does have startup time issues because unlike the embedded version of our JIT's it doesn't feature MVM. However, a lot of this is configurable and easily fixable. Hotspot is remarkably tuneable and if MVM is added (which is possible) could probably beat ART in startup time as well...

ART is pretty fast by now though and pretty well understood. I don't think Google would switch to hotspot and I don't think it will need that for compliance either. They might just reuse some libraries that can be common and that's it.

Every time an Android device installs its monthly security update, it spends an inordinate amount of time, with the screen on, "optimizing" every single app.

I'm curious about big commits like this one:

In the context of the recent juniper attack where some unauthorized code was committed without anybody noticing for years, it seems like it would be easy to hide a backdoor in such a big commit.

How do you go about checking the integrity of the code when you have so many files?

8902 files were changed, most added, and the commit says it's just importing openJDK files. Is there anybody checking that the source file imported haven't been modified to include some kind of backdoor?

edit: replaced "cisco" with "juniper"

> ...recent cisco...

you mean juniper perhaps?

Woops... it's fixed!

I like this thinking. Someone with a big export of code repos could do some automated diff/analysis on open-source code to find vendored imports that have been modified from upstream or is outdated to facilitate manual security audits.

The same way I validate scary merge commits: Do it myself, and compare what I get to what they got.

The commit references ojluni. This relates to "luni" in Android source which stands for lang util net io. Sounds like there are plans to replace the harmony implementation with OpenJDK one. License differences are definitely interesting but could be also related to performance and completeness - luni is fairly small set of classes whereas ojluni import brings in a ton more.

But what I don't understand is why they're importing the full AWT API! That's nuts.

Entire AWT and Swing API. Nuts indeed!

I'm starting to wonder if Oracle got them to go JCK. Would be a technically crappy development that serves little actual purpose. Or maybe something's cooking and we don't know yet.

IIRC, Oracle's claim was that Google's use of the API wasn't fair use because they didn't implement all of it; they weren't trying to build a compatible product. So importing enough of Java to be fully compatible might be a result of the lawsuit, yeah.

Since Android's zygote does preloading of system classes and ART does precompilation to native code, I think having these many additional classes in the core is bound to have an adverse effect on app startup times and device memory consumption. I'm surprised they're doing this so sneakily.

I took a closer look at the commits, and I think it's not as bad as it looked at first. They imported all classes of OpenJDK in the February commit posted here, but then removed things like Swing at some later point of time. It's not there in the master now.

Oh cool, so just a new libcore implementation based on openjdk, which should be a good thing actually.

> But what I don't understand is why they're importing the full AWT API!

Some really old and completely unrelated non-awt classes depend on interfaces defined in java.awt. But in this case it simply seems a mass import.

Its exciting to see all the goodies from java sound finally coming in. I could care less about cobra though Im just glad we get better support for audio coding from java without having to resort to jni and the native layer in c/c++.

What's mysterious? Legit curious.

Google has made a point of not using GPLed code (except Linux) or Sun/Oracle code in Android, for technical and legal reasons. There's even a lawsuit about it. So it's surprising to see GPLed Oracle OpenJDK code being committed anywhere near Android.

Interesting, isn't Android ASL?

I recall seeing a "libopenjdkjni" related review on android-review earlier today. It was just a makefile change though, no real code diff. Unfortunately I can't find the review anymore.

"libopenjdkjni" looks like someone slammed their hand in the keyboard.

This one: https://android.googlesource.com/platform/libcore.git/+/aab9...

    -LOCAL_SHARED_LIBRARIES += libart libnativehelper libdl
    +LOCAL_SHARED_LIBRARIES += libopenjdkjvm libnativehelper libdl

Replacing ART with OpenJDK JVM?!?

This is the one I was thinking of: https://android-review.googlesource.com/#/c/193835/

It was 'libopenjdkjvm' not 'libopenjdkjni' as I recalled.

Change dependency from libart -> libopenjdkjvm.

Does this mean they're discarding ART for OpenJDK's VM?

It could. Not a bad thing, the HotSpot VM in OpenJDK is incredibly fast.

Does that hold on 32-bit ARM CPUs with 1GB of RAM?

> the HotSpot VM in OpenJDK is incredibly fast.

True, but only on x86.

It does seem bizarre that Google would put in all the effort to develop ART and then retire it so quickly.

On a tangential note, I guess this means Android is finally getting Java 8 support as well as Java 9, 10, 11, etc

Does this mean no more DVM?

Dalvik was replaced by ART in Lollipop

So Lambda's are finally comming

oh, i hope so!

I was hoping this would be a link to code committed by AI. More of those, please?

Reply on news.ycombinator.com