Java Development on an Apple M1 – A One Year Review
rieckpil.de
rieckpil.de
At the last company we had Docker, Java 8, Wildfly 10, Gradle 4 and I didn't manage to make it run locally on M1.
I was pleasently surprised with how smooth it was to connect to Ubuntu VM with Jetbrains Gateway. You use native Jetbrains app on your computer (Intellij Idea in my case), but everything is executed and compiled on the remote machine (Ubuntu VM). Another huge upside is that Docker is running on Ubuntu, which is a lot faster than on OSX. Downside is that your are dependant on the Internet ofcourse.
Officially the product is still in Beta, but it worked good enough for me.
VSCode have had some more luck with their frontend/backend architecture making it easier to pull off, glad to see jetbrains is on the move, as I prefer them.
No relation, just a super happy user.
Probably not asdf's fault, though. I've had multiple issues with gcloud not being compatible with my setup, spamming errors like /tmp/_MEIRZ3igG/libssl.so.1.1
But what asdf doesn't solve, though, is the setup for new devs. Python projects can sometimes take days to get running properly on a machine because of various differences, asdf solves some of it but replaces it with other installation steps instead.
That's what I like about a dockerized setup, if it works one place it works for everyone (almost, nix is probably better).
I like Docker for a lot of things. That said, on code inside containers is pretty awful, in my experience, and I'd much rather use docker-compose with something like Traefik to route outside Docker so I can run my service locally and everything works as I expect it. You can always tell a project that I work on because there's a bash script in there that fires up tmux with docker-compose, all services under nodemon, ngrok, etc. all good to go. ;)
I think the "Remote-SSH" plugin is a better fit for a comparison though, but @vital101's comment is not wrong.
I wanted to transition to this development model a year ago, but unfortunately X11 forwarding was slow on mobile connections, and as far as I'm aware (if you know a workaround let me know), on Linux, RDP can't be made to just share a single app, only full desktop sessions.
We have also tried Jetbrains Projector, which is basically a different rendering engine for Java Swing. A remote IDEA instance was rendering the UI through HTML, i.e. you could develop in a browser. It worked relatively well, but there were some issues around copy/paste, etc.
They have an electron-based client that improves on some of these issues that stem from running inside a browser:
https://jetbrains.github.io/projector-client/mkdocs/latest/i...
Testcontainers Cloud (https://www.atomicjar.com/2021/11/announcing-testcontainers-...) may be another tool to use for those that only want to "outsource" the container part of running intgeration tests.
I went back to remote controlling the full-featured IDE via Projector, their earlier approach too remote development. That works extremely well for what it is. Probably the richer Gateway client will catch up in a while, but not quite yet.
Thanks for mentioning this. I was going to give this a try this weekend but that's a dealbreaker for me.
Sounds like they _are_ making fast progress at least.
The main issue I've seen is the gateway/target JVM combo gets wedged in some weird state somehow that persists across retarts and I found myself killing idea processes by hand on the remote server. But hey it's Beta! Works pretty excellent considering.
they do state on this page https://lp.jetbrains.com/projector/ "If you're not sure which solution you should choose, please consider using Gateway."
I couldn’t see myself going back to just running everything outside of at least some containerised environment at a minimum even if not fully remote.
I do a bunch of front end web stuff as well and now the thought of just running npm on my local machine gives me the chills. It’s akin to attending an orgy without a condom :)
I also have a 16G 2015 MBP running Monterey and IntelliJ with no problem at all though.
On my M1 Max MBP Jetbrains Rider stars in under two seconds. Loading the quite large project takes another two.
On my previous laptop, an i7 MBP, it took in the order of minutes to get Rider to the point where I could actually start writing code with codecompletion. It sounded like a jet taking off while Rider blasted all cores at max to enable the smart completions.
The M1 Max isn't even slightly warm in the same case. Haven't found out a single spot where it would've even slightly stuttered.
I noticed I fell out of the "flow" a lot less on M1, because actions that took a few seconds on other machines are now instantaneous.
Regarding the Intellij Idea and Docker, my old 16GB MBP 2015 Mojave was struggling hard and getting very hot with 100K LoC project. I would definitely use the Gateway on old machine aswell.
For a more apples to apples comparison, Intel 12th gen and AMD Ryzen 6000 series would also give you a lightning fast experience comparable to your M1, especially since modern systems come with faster memory and faster storage than your 2015 machines, contributing to the perception of speed.
There seems to be a work around with qemu and UTM but I cannot speak to its performance or how viable this solution is.
Though this got me thinking a little bit, I have ordered a M1 Max 32GB which I have been waiting for over 5 months now. I am actually looking into getting the M1 air with 8GB as I think it will suffice for my Java development. I don't need to run VM's on that machine.
I'm gonna think about this through this weekend. I don't really need the firepower from the M1 Max as it has similar single core performance to the M1 air. And if this product matures, I will really not need that much RAM either.
I was hoping to use it to do most of my stuff remotely, instead of a remote client or a mouse/kb sharing program like Barrier.
Even worse, given that Docker and Kubernetes basically take Java/.NET application servers to other programming languages.
Talk about fitting square pegs into round holes.
This approach was way lighter weight than pushing enormous images around.
But people who use keycloak often have little idea what Java is. They write code in Go and call Keycloak interfaces via REST API. They just need to start that thing and connect to some database.
With docker they'll get it up and running in minutes.
With EAR they'll spend next few days I bet it.
> Even worse, given that Docker and Kubernetes basically take Java/.NET application servers to other programming languages.
No they don’t. Not sure what you intended to convey here.
You’ll always be able to build a unique deployment solution for Java, python, or a native binary. But containers let you solve this problem the same way for every program.
Per the HN Guidelines:
"Please don't post shallow dismissals, especially of other people's work. A good critical comment teaches us something."
It's not hard to imagine that using a big container solution designed for another operating system (and performance hit) is not the most elegant solution here. Especially when the alternative is an environment variable and zip file you already mentioned yourself. There is no step three.
We've been upgrading our dependencies for arm64 support; for the most part it is as simple as updating our pined version to a newer version of the jar. Sometimes the native code is in a separate jar so you just add it (opencv works this way).
But how is that related to the database? Wouldn't you just run the database in a container as a side car?
Or you need to try something on another project that uses a different version of your database server. Or you need to pinch-hit on a project that has a half-dozen service dependencies at particular versions, none of which you have installed and running already.
The ability to very easily spin up clean installs of a bunch of services at arbitrary versions is incredibly useful. Containers aren't the only way to do that, but Docker does make it pretty damn convenient.
Also, Java 9 broke backward compatibility promise, and Java 17 broke it further by disabling a lot of default modules. Docker helps A/B testing multiple variation of JDK for our builds.
We definitely need Docker. It's a live saver for so many of our Java projects.
Not to mention, every single CI/CD engineers in a big company will mandate Docker as a packaging requirement anyway. So why not do it right and use Docker anyway?
Does Java not have a version manager that lets you install multiple JDKs side-by-side and easily switch between them? For ecosystems I'm familiar with like Node.js and Rust this as is simple as a single command to install a given version and adding a text file with the required version to each repository(/directory) (it then gets used automatically when running code in that repo).
The answer for why not use Docker for development if you're using in production (on Windows/macOS) is that it's much slower. So the same reason that I have separate debug (fast compile time) and release (optimised) builds. If you're on Linux then Docker is great.
But often times, our laptop don't have enough memory to perform the build and various tests. So we have to offload that to the CI/CD pipelines.
No we don't need them.
Yes, Yes, Yes, often teams use shells script, gradle or maven.
The same config you'd put in the docker yaml or whatever would go into the maven or whatever config except that the devs would be familiar with the existing tools and wouldn't spend countless hours mucking about with docker files.
Also unless you're using docker on x86 AND linux, likely things will not work or at least you'll run into mem, or perf or other compat issues.
After working with Java/Python for years, I had forgotten about the hell people go through with other langs deploying on diverse OSes/arches.
If I need to test my code locally against a service some other team develops, the only exchange needs to be "here's a docker compose file you can run with one command", not a wiki with shell scripts and other dozens of instructions which were likely outdated 3 years ago.
As opposed to my assertion: "a Java dev knows one of {gradle,maven,shell} is almost always true.
I have been coding since 1986.
There was a time before Docker.
It meant for me that in the beginning IDEs, Java and things like node ran all in Rosetta mode and were very slow. I was almost starting to regret my choice. Sometimes there were Arm fixes committed for things like NodeJS but they were not yet released so I needed to build my own version. Same issues with the JVM to find something able to run smoothly. (Felt that the slowdown was extra painful for JIT like languages) Also for Docker it meant waiting for some months to have something that would work properly but luckily I could manage without at the start. I was happy to see that major blockages improved in some months and for some lagging dependencies I was pushing some fixes myself.
At this moment almost 1.5 years later everything works smoothly. The IDE, Java, Node etc. I still had to navigate around some specific dependencies but the fully native M1 development flow is so smooth compared to my previous Intel MacBook. I am quite happy.
The JetBrains IDEs were bordering unusable under Rosetta. They would speed up over time as Rosetta did its work, but it was multiple tens of minutes of slow as molasses until things got better and after each IDE restart you were back at square one.
Thankfully, JetBrains released updates to their products to run on a bundled ARM JVM very quickly
I remember that first month as being very painful to use the IDE. It was really dramatically slow.
The error message was from the native JDK code, so was not very useful, I just gave up and searched the bug tracker for "M1" until I found something that looked close. IIRC it was some weird error caused by code that was objectively wrong for several years, but the race/error condition had never been observed on an Intel machine, if I remember correctly. Thankfully it was in an EA build of OpenJDK, otherwise I probably would have given up and thrown up a VM in the cloud to run it in.
And here’s a post talking about it in the context of C++11: https://www.arangodb.com/2021/02/cpp-memory-model-migrating-...
On x86, a write by core A to memory will be available to core B if core B reads from main memory.
On aarch64, a write by core A will not immediately get published to main memory (will likely stay in cache (L1, L2, etc.), so even if core B tries to read from main memory it won't see the value from core A.
Ultimately aarch64's "weak"(er) memory model is more efficient as the programmer/compiler can make more efficient memory accesses. This results in fewer cache invalidations between cores. The problem in practice is that tons of production code has been written which assumes the x86 memory model. It may also just be a concurrency bug which doesn't manifest on x86 but does on aarch64 like in the post.
Again, this is a simplification of what happens but I think it illustrates the difference to some degree.
I had a little chuckle at Spotify being in the list of developer tools. I feel the same. No techno, no code.
If it's more ambient music (like those "lo-fi beats to study to" livestreams that are always running on YouTube) I can usually let that run and be a replacement to the white noise coming from the fan in my room. But I still keep the volume pretty low.
> I remember Carmack talking about productivity measurement. While working he would play a CD, and if he was not being productive, he'd pause the CD player. This meant any time someone came into his office to ask him a question or he checked email he'd pause the CD player. He'd then measure his output for the day by how many times he played the CD (or something like that -- maybe it was how far he got down into his CD stack). I distinctly remember him saying "So if I get up to go to the bathroom, I pause the player".
> You know what's pretty hardcore? Thinking that going to the bathroom is essentially the same as fucking off.
0: http://bookofhook.blogspot.com/2013/03/smart-guy-productivit...
I realize it's mostly talking about testing infrastructure rather than Java code but it feels sad that we end up here - I remember Java's top selling point being "write once run anywhere" and I genuinely believed the JVM would shield you from most issues with CPU architectures. But it seems like they managed to sneak back in through the back door.
It seems that those CPU architectures indeed sneak back through a back door... The same on Android, there you have also many libraries which need to be build and published for specific architectures. But luckily new architectures are not added regularly.
After skimming the article, it looks like it's mostly another long complaint about Docker which has been discussed many times at this point.
Before people thought Java would save everyone from ever having to think about native dependencies. Turned out not so much, although it works "OK" for desktop these days.
Now everyone thinks Docker is going to save everyone from having to think about native dependencies. In a few years everyone will realize that Docker only works like that when deploying from linux to linux on the same CPU architecture.
Another alternative that I'm currently considering is to rent some VPS in my city and use it as docker host. I'll be dependant on the Internet, so that's not very nice, but might be an option to consider.
I wish Apple would extend Rosetta to VM support. That's really missing piece of puzzle when it comes to migrating to ARM. qemu is not good enough.
Ryzen 9590X: 20:09 (20 min 9 sec)
Raspberry Pi 3: 15:46
Raspberry Pi 4 8GB: 4:34
That settled it for me.
At least CircleCI has machine ARM runners but not docker ones.
https://docs.gitlab.com/runner/install/
You must be talking about shared runners in GitLab.com.
Available through compose as well.
Unless you want to wait 6 months or even a year to do any work reliably without these 'issues'. Even by then a more faster machine would be worth buying anyway.
And now there is the mac studio, M1 ultra, 128GB ram, 64 cores. An absolute beast of a machine. Fits in a backpack, so it's just as portable as a laptop assuming you work from docking stations anyway.
I had similar issues early on with testcontainers but once that was resolved, developing Java on the M1 has been a breeze.
It's also cold most of the time; it takes ages to get up to what feels like "room temperature". I'm besmitten.
I'm still afraid to make the jump to M1.
Is swapping Intel for Arm64 transparent enough to preserve the peace of mind back-end developers need, here?
Maybe with Java the answer is yes, but how about non-java environments?
These days, Docker supports multiarch images, so it's fairly trivial to build one image that supports with AMD64 and ARM64 transparently. CI tools like CircleCI support runners for AMD64 and ARM64, so you can even run your test suite on both architectures for additional confidence, if needed.
For me, Docker containers have always been about reproducibility of builds (which some people will argue about, but it does a good job 99% of the time) and consistency in deployment. You don't need to have an artisanal deployment methodology for each application... you just need to have a way to deploy docker containers. Even for single static binaries like Go projects often produce, wrapping them in a docker container just makes it easier to abstract away the deployment problems across projects. For more difficult to deploy languages like Python, you get similar benefits to having a single static binary by wrapping all the dependencies up into a neat little container.
Plus, once you have a standard unit of deployment like a docker container, you gain access to the broader ecosystem of container tools with minimal effort, such as running each container within a Firecracker microVM if you need isolation.
I think around next year I might considering asking for an m1 upgrade for my aging mbpro.
I have had an M1 MacBook Pro for over a year and it did take extra work building SBCL Common Lisp from scratch, setting up brew for M1 architecture, and experimenting what would work for me on Docker.
M1 Macs are awesome, I love mine, but I understand devs who want to stick with Intel.
One of the reasons M1 is easy for me is that I do a lot of dev using mosh/ssh, tmux, Emacs on Intel VPSs.
Tasteless jokes aside, great seeing you writing useful stuff for the world in your post “large, Franconia-based manufacturer” life!
I guess their abacus broke
This is a haiku
We Fixed it in this release
Worse Worse Is Better
They're angry, a lack of Ken
Clicks downward arrow