As is always the case with such rewrites, the big question is whether the improvements came from the choice of language or because they updated a crusty legacy codebase and fixed bugs/bottlenecks.
Reduced memory consumption for cloud applications was apparently also the primary reason IBM was interested in Swift for a while. Most cloud applications apparently sit mostly idle most of the time, so the number of clients you can multiplex on single physical host is limited by memory consumption, not CPU throughput.
And Java, with the JIT and the GC, has horrible memory consumption.
It has higher level ergonomics that something like rust lacks (as much as I like rust myself), doesn’t have many of the pitfalls of Go (error handling is much better for example) and is relatively easy to pickup. It’s also in the same ballpark performance as rust or C++.
It’s not perfect by any means, it has several issues but it’s quickly becoming my preferred language as well for knocking out projects.
Rust and Swift are pretty much the only two choices and Rust is arguably much more pain in the ass for the average joe enterprise coder.
You can share memory not only at the machine level, but between different applications.
Each extra GB per core you add to your shape, actually costs something. Hence every GB/core that can be saved results in actual cost savings. But even then, usually every extra GB/core is ~5% of the CPU cost. Hence, even when going from 10 GB/core (sort of a lot) to 1 GB/core, that only translates to ballpark ~50% less HW cost. Since they did not mention how many cores these instances have, it's hard to know what GB/core were used before and after, and hence whether there were any real cost savings in memory at all, and if so what the relative memory cost savings might have been compared to CPU cost.
Given that they also experienced a 90% reduction in Memory Usage (presumably from Java GC vs Swift AOT memory management) - it seems more likely the gains are in fact from the difference in languages.
As for performance and locality, Java's on-the-fly pointer reordering/compression can give it an edge over even some compiled languages in certain algorithms. Hard to say if that's relevant for whatever web framework Apple based their service on, but I wouldn't discount Java's locality optimisations just because it uses a GC.
You mean it does escape analysis and stack-allocates what it can? That would definitely help, but not eliminate the GC. Or are you thinking of something else?
Thinking about it more, I remember that Java also has some performance-hostile design decisions baked in (e.g. almost everything's an Object, arrays aren't packed, dynamic dispatch everywhere). Swift doesn't have that legacy to deal with.
I’ve replaced Java code with Python a few times and each time even though we did it for maintenance (more Python devs available) we saw memory usage more than halved while performance at least doubled because the code used simpler functions and structures. Java has a far more advanced GC and JIT but at some point the weight of code and indirection wins out.
I am old enough to have seen enterprise C and C++ developers.
Where do you think stuff like DCE, CORBA, DCOM has come from?
Also many of the things people blame Java for, where born as Smalltalk, Objective-C and C++ frameworks before being re-written in Java.
Since we are in a Apple discussion thread, here is some Objective-C ids from Apple frameworks,
https://github.com/Quotation/LongestCocoa
I also advise getting hold of the original WebObjects documentation in Objective-C, before its port to Java.
This is true to some extent but the reason I focused on culture is that there are patterns which people learn and pass on differently in each language. For example, enterprise COBOL programmers didn’t duplicate data in memory to the same extent not only due hardware constraints but also because there wasn’t a culture telling every young programmer that was the exemplar style to follow.
I totally agree about C++ having had the same problems but most of the enterprise folks jumped to Java or C# which felt like the community of people writing C++ improved the ratio of performance sensitive developers. Python had a bit of that, especially in the 2000s, but a lot of the Very Serious Architects didn’t like the language and so they didn’t influence the community anywhere near as much.
I’m not saying everyone involved are terrible, I just find it interesting how we like to talk about software engineering but there are a lot of major factors which are basically things people want to believe are good.
> I’ve replaced Java code with Python a few times ... while performance at least doubled
This is not “Java slow, Python fast” – I expected it to be the reverse – but rather that the developers who cranked out a messy Spring app somehow managed to cancel out all of the work the JVM developers have done without doing anything obviously wrong. There wasn’t a single bottleneck, just death by a thousand cuts with data access patterns, indirection, very deep stack traces, etc.
I have no doubt that there are people here who could’ve rewritten it in better Java for significant wins but the goal with the rewrite was to align a project originally written by a departed team with a larger suite of Python code for the rest of the app, and to deal with various correctness issues. Using Pydantic for the data models not only reduced the amount of code significantly, it flushed out a bunch of inconsistency in the input validation and that’s what I’d been looking for along with reusing our common code libraries for consistency. The performance win was just gravy and, to be clear, I don’t think that’s saying anything about the JVM other than that it does not yet have an optimization to call an LLM to make code less enterprise-y.
As a counterpoint: Look at Crazy Bob's (Lee/R.I.P.) Google Guice or Norman Maurer's Netty.IO or Tim Fox's Vert.x: All of them are examples of how to write ultra-lean, low-level, high-performance modern Java apps... but are frequently overlooked to hire cheap, low-skill Java devs to write "yet another Spring app".
Yeah, that’s why I labeled it culture since it was totally a business failure with contracting companies basically doing the “why hire these expensive people when we get paid the same either way?” No point at ranting about the language, it can’t fix the business but unfortunately there’s a ton of inertia around that kind of development and a lot of people have been trained that way. I imagine this must be very frustrating for the Java team at Oracle knowing that their hard work is going to be buried by half of the users.
Nowadays to add to your comment, all major free beer implementations, OpenJDK, OpenJ9, GraalVM, and the ART cousin do AOT and JIT caches.
Even without Valhala, there are quite a few tricks possible with Panama, one can manually create C like struct memory layouts.
Yes it is a lot of boilerplate, however one can get around the boilerplate with AI (maybe), or just write the C declarations and point jextract to it.
Let's say you have an object that looks like A -> B -> C. Even if the allocation of A/B/C happened at very temporally different times and inbetween different allocations, the next time the GC runs as it traverses the graph it will see and place in memory [A, B, C] assuming A is still live. That means even if the memory originally looks something like [A, D, B, Q, R, S, T, C] the act of collecting and compacting has a tendency to colocate.
I don't know which of these is more important on a modern machine, and it probably depends upon the workload.
The downside is fragmentation and the CPU time required for memory management. If you have an A -> B -> C chain where A is the only owner of the B and B is the only owner of C, then when A hits 0, it has to do 2 pointer hops to deallocate B and then deallocate C (plus arena management for the deallocs).
One of the big benefits of JVM moving style collectors is that when A dies, the collector does not need to visit B or C to deallocate them. The collector only visits and moves live memory.
I suspect this puts greater emphasis on functionality like value types and flexibility in compositionally creating objects. You can trend toward larger objects rather than nesting inner objects for functionality. For example, you can use tagged unions to represent optionality rather than pointers.
The cost of deep A->B->C relationships in Java comes during collections, which still default to be halting. The difference is a reference counting GC will evaluate these chains while removing objects, while a reference tracking GC will evaluate live objects.
So, garbage collection is expensive for ref-counting if you are creating large transient datasets, and is expensive for ref-tracking GC if you are retaining large datasets.
In my experience Java is a memory hog even compared to other garbage collected languages (that's my main gripe about the language).
I think a good part of the reason is that if you exclude primitive types, almost everything in Java is an heap-allocated object and Java objects are fairly "fat": every single instance has an header of between 96 and 128 bit on 64-bit architectures [1]. That's... a lot. Just by making the headers smaller (the topic of the above link) you can get 20% decrease in heap usage and improvements in cpu and GC time [2].
My hope is that once value classes arrive [3][4], and libraries start to use them, we will see a substantial decrease in heap usage in an average java app.
[1] https://openjdk.org/jeps/450
[2] https://openjdk.org/jeps/519
Many other GCed languages, such as swift, CPython, Go, do not use a moving collector. Instead, they allocate and pin memory and free it when not in use.
The benefit to the JVM approach is heap allocations are wicked fast on pretty much all its collectors. Generally, to allocate it's a check to see if space is available and a pointer bump. For the other languages, you are bound to end up using a skiplist and/or arena allocator provided by your malloc implementation. Roughly O(log(n)) vs O(1) in performance terms.
Don't get me wrong, the object header does eat a fair chunk of memory. Roughly double what another language will take. However, a lot of people confuse the memory which the JVM has claimed from the OS (and is thus reported by the OS) with the memory the JVM is actively using. 2 different things.
It just so happens that for moving collectors like the JVM typically uses more reserved memory means fewer garbage collections and time spend garbage collecting.
Swift is not garbage collected, it uses reference counting. So, memory there is freed immediately when it is no longer in scope.
Chapter 5, https://gchandbook.org/contents.html
I know about the trade-offs that a moving GC does, but the rule is about double memory usage, not ten times more like a 90% reduction would seem to imply.
If memory is an issue, you can set a limit and the JVM will probably still work fine
Frankly, it just takes some motivated senior devs and the tantalizing ability to put out the OP blog post and you've got something management will sign off on. Bonus points you get to talk about how amazing it was to use Apple tech to get the job done.
I don't think they seriously approached this because the article only mentioned tuning G1GC. The fact is, they should have been talking about ZGC, AppCDS, and probably Graal if pause times and startup times were really that big a problem for them. Heck, even CRaC should have been mentioned.
It is not hard to get a JVM to startup in sub second time. Here's one framework where that's literally the glossy print on the front page. [1]
Then the next annual report talked about improved scalability because of this amazing technology from Google.
Delving into Java arcana instead of getting first hand experience in developing in Swift would've been great opportunity wasted to improve Swift.
However, they chose to replace an existing system with swift. The "arcana" I mentioned is start up options easily found and safe to apply. It's about as magical as "-O2" is to C++.
Sure, this may have been the right choice if the reason was to exercise swift. However, that shouldn't pretend like there was nothing to do to make Java better. The steps I described are like 1 or 2 days worth of dev work. How much time do you think a rewrite took?
I’m sure you’re right, there must’ve been ways to improve the job of deployment. But if they wanted to reduce resource usage and doing it in Swift aligned with some other company goal it would make sense they might just go straight to this.
Saving few weeks or months by learning 3rd party technology instead of applying and improving first party technology would be amateurish.
> However, that shouldn't pretend like there was nothing to do to make Java better.
This seems like constant refrain that Apple or anyone choosing their own tech over someone else's owe absolute fair shot to stuff they didn't choose. This is simply not the way world works.
Yes, there are endless stories companies spending enormous resources to optimize Java stack even up to working with Core Java team at Oracle to improve on JVM innards. But those companies are just (although heavy) user of core technology rather than developer of competing one. Apple is not one of those users, they are developers.
And not what I'm advocating for. Sometimes rewrites are necessary.
What I'm advocating is exercising a few well documented and fairly well known jvm flags that aren't particularly fiddly.
The jvm does have endless knobs, most of which you shouldn't touch and instead should let the heuristics do their work. These flags I'm mentioning are not that.
Swapping g1gc for zgc, for example, would have resolved one of their major complaints about GC impact under load. If the live set isn't near the max heap size then pause times are sub millisecond.
> This seems like constant refrain that Apple or anyone choosing their own tech over someone else's owe absolute fair shot to stuff they didn't choose. This is simply not the way world works.
The reason for this refrain is because Java is a very well known tech, easy to hire for (which Amazon that you cite heavily uses). And Apple had already adopted Java and wrote a product with it (I suspect they have several).
I would not be saying any of this if the article was a generic benchmark and comparison of Java with swift. I would not fault Apple for saying "we are rewriting in swift to minimize the number of languages used internally and improve the swift ecosystem".
I'm taking umbridge to them trying to sell this as an absolute necessity because of performance constraints while making questionable statements on the cause.
And, heck, the need to tweak some flags would be a valid thing to call out in the article "we got the performance we wanted with the default compiler options of Swift. To achieve the same thing with Java requires multiple changes from the default settings". I personally don't find it compelling, but it's honest and would sway someone that wants something that "just works" without fiddling.
Decades ago, I was working with three IBM employees on a client project. During a discussion about a backup solution, one of them suggested that we migrate all customer data into DB2 on a daily basis and then back up the DB2 database.
I asked why we couldn't just back up the client's existing database directly, skipping the migration step. The response? "Because we commercially want to sell DB2."
And yes, Apple is huge and rich, so they can get fast machines with less memory, but they likely have other tasks with different requirements they want to run on the same hardware.
You can actually witness this to some degree on Android vs iPhone. iPhone comfortably runs with 4GB RAM and Android would be slow as dog.
[1]: I don't dispute the results, but I also like to note that as a researcher in Computer Science in that domain, you were probably looking to prove how great GC is, not the opposite.
> iPhone comfortably runs with 4GB RAM and Android would be slow as dog.
This has nothing to do with RAM. Without load, Android wouldn’t even push 2GB, it would be still slower than iPhone because of different trade-offs they make in architecture.
Anyhow, that was just an anecdotal unscientific experiment to give you some idea--obviously they are two different codebases. The literature is there to quantify the matter as I noted.
Unless one knows exactly what ART version is installed on the device, what build options from AOSP were used on the firmware image, and what is the mainline version deployed via PlayStore (if on Android 12 or later), there are zero conclusions that one can take out of it.
Also iOS applications tend to just die, when there is no more memory to make use of, due to lack of paging and memory fragmentation.
For user defined stuff we’ve recently gained records, which are a step in that direction, and a full solution is coming.
What java is getting in the future is immutable data values where the reference is the value.
When you have something like
class Foo {
int a;
int b;
}
var c = new Foo();
When values land, the representation of `c` can instead be (the JVM gets to decide, it could keep the old definition for various performance reasons) something like [type, a, b]. Or in C++ terms the memory layout can be analogous to the following:
struct Foo { int a, int b };
struct Foo c;
c.a = 1;
c.b = 2;https://learn.microsoft.com/en-us/dotnet/csharp/language-ref...
Even records are not value-based types, but rather are classes limited to value-like semantics - e.g. they can't extend types, are expected to have immutable behavior by default where modification creates a new record instance, and the like.
The JVM theoretically can perform escape analysis to see that a record behaves a certain way and can be stack allocated, or embedded within the storage of an aggregating object rather than having a separate heap allocation.
A C# struct gets boxed to adapt it to certain things like an Object state parameter on a call. The JVM theoretically would just notice this possibility and decide to make the record heap-allocated from the start.
I say theoretically because I have not tracked if this feature is implemented yet, or what the limitations are if it has been.
Valhala is supposed to bring language level support, the biggest issue is how to introduce value types, without breaking the ABI from everything that is in Maven Central kind of.
Similar to the whole async/await engineering effort in .NET Framework, on how to introduce it, without adding new MSIL bytecodes, or requiring new CLR capabilities.
Create C like structs, in regards to memory layout segments, and access them via the Panama APIs.
I don’t think it’s meant to be a postmortem on figuring out what was going on and a solution, but more a mini white paper to point out Swift can be used on the server and has some nice benefits there.
So the exact problems with the Java implementation don’t matter past “it’s heavy and slow to start up, even though it does a good job”.
My expectation is that if you put the work in you can get actual hard numbers, which will promptly be ignored by every future person asking the same "question" with the same implied answer.
If the "just rewrite it and it'll be better" people were as right as they often seem to believe they are, a big mystery is JWZ's "Cascade of Attention-Deficit Teenagers" phenomenon. In this scenario the same software is rewritten, over, and over, and over, yet it doesn't get faster and doesn't even fix many serious bugs.
If the "just rewrite it and it'll be better" people were as right as they often seem to believe
I confess to having been part of the cascade at various parts of my career.
It’s only Rust (or C++, but unsafe) that have mostly zero-cost abstractions.
Swift and Rust also allow their protocols to be erased and dispatched dynamically (dyn in Rust, any in Swift). But in both languages that's more of a "when you need it" thing, generics are the preferred tool.
This is not a bad thing, I was just pointing out that Go doesn't have a performance advantage over Swift.
This just isn't true. It's good marketing hype for Rust, but any language with an optimizing compiler (JIT or AOT) has plenty of "zero-cost abstractions."
JVM also like memory, but can be tailored to look okayish, still worse than opponents.
Do you know where Java EE comes from?
It started as an Objective-C framework, a language which Swift has full interoperability with.
https://en.wikipedia.org/wiki/Distributed_Objects_Everywhere
Yes, having lived on the daily build bleeding edge of Swift for several years, including while macros were being developed, I have indeed heard of them.
> Do you know where Java EE comes from?
Fully aware of the history.
The point stands: it is substantially harder with Swift to make the kind of spring style mess that JVM apps typically become (of course there are exceptions: I typically suggest people write Java like Martin Thompson instead of Martin Fowler). Furthermore, _people just don’t do it_. I imagine you could visualise the percentage of swift server apps using an IOC container using no hands.
Then I can start counting.
Any language can be a shit show, when enough people beyond the adoption curve write code in them, from the letcoders with MIT degree to the six week bootcamp learners shipping single functions as a package, and architects designing future proof architectures on whiteboards with SAFe.
When Swift finally crosses into this world, then we can compare how much of it has survived the world scale adoption exposure, beyond cozy Apple ecosystem.
It’s the history, standard libs, and all the legacy tutorials which don’t get erased from the net