Hacker News Reader

Comment by exabrial | Hacker News Reader

What I really want is an ARM server that can accelerate our JVM workloads as much as M1/M2 does, without regards to power density.

Why? In colo you’re usually not as concerned about density. We’re renting a chunk of rack space for a bottom basement price of $200/month and we can easily stick a router and compute / storage nodes with 64 TB of disk and boat loads of memory in that rented space; plenty for our needs without coming anywhere close to the power allocation.

When we were purchasing a few servers for our colo, we ultimately just went with AMD because the price per core was there. It ends up being less performant than an m2, maybe m1 ish.

Ultimately we’re a small market though. The ‘sexy’ thing to do is waive your AWS invoice size around at other hipsters during cocktail parties; not to own your own hosting even though you’ve paid for it serval times over. I don’t see this changing anytime soon.

View on HN

Have you tried Ampere? It's pretty good as far as ARM servers go, and even has acceleration for hip stuff like Tensorflow and Pytorch. I personally haven't experimented with Java apps on it, but it should have a high enough IPC to compete with other similarly dense ARM options.

And of course, Nvidia has their 72x2 core Grace chip coming soon. It will be very interesting to see how that stacks up against the ARM incumbents.

Actually pretty excited to NVidia offering, but we spent our budget :(

i could see it changing a bit if oxide.computer is successful in shipping lots of units, since they've already got major cool factor with their podcasts.

I'm curious about these cocktail parties you speak of - over here, in-person tech meetups have been pretty rare since covid

Even post covid?

How does M1/M2 accelerate JVM apps, and in general how do you optimise your JVM apps to make full use of the resources?

I doubt M1/M2 do anything special for the JVM, it's just that they are overall better performing for the same power. In a server setting though I don't see them having the same advantages.

It has lot higher memory throughput, so mid-range M1 chip produces results comparable to my 3970x Threadripper on some of my workloads (scala compilation) while it consumes just a fraction of the power the Threadripper requires.

I’d love to see a cluster of M2 SOCs placed together on a single PCB with built in networking. Maybe direct PCIe to PCIe communication between the SOCs in a hypercube topology. Having multiple CPUs spread out around the box will help with cooling which is my limiting factor for power density. My tasks are data parallel and CPU bound with low ram and disk requirements. I could probably design such a PCB myself but the cost of doing so will unlikely be paid back unless my compute needs go up by another order of magnitude. I am many years away from that being viable. For now I’m sticking to AMD chips. Unless someone else wants this as well for which I could amortize the costs.

It would seem this devkit is even more ideal for you? 80 cores and buy as little memory as you want. Not to mention it does not need clustering and runs any aarch64 distro of your choosing

I checked out the benchmarks and it seems that the 80 core version would be ~6.5x M2 and would indeed be cheaper on the assumption that the only way to get M2 chips is to first buy a Mac. But if (an I know it isn’t going to happen) M2 was at the Jetson Nano price of $150 then it starts to become attractive again. It’s 2.3x faster than Ryzen 7900X, I’m not sure if the dev kit has a price performance advantage over that and I’ve seen the Ampere Altra chips on their own for much more so I’m thinking the dev kit has a discount which probably comes with a purchase limit.

The dev kit has no purchase limit. Same is true for the dev platform workstation 32, 64, 80, 128 core (and a 96 faster cores option if you ask nicely).

https://www.tomshardware.com/news/ampere-64-core-arm-worksta...

For those keeping score the Ampere Altra 128 core 3GHz Supermicro telco edge server being used in 5G base stations benchmarks equal to 100 RPi4 and 22% more energy efficient.

https://youtu.be/UT5UbSJOyog

Interesting, when I first saw these chips my thought was Epic performance for Epic prices, now it looks like 2x Ryzen performance for 2x the price. Which is a huge improvement, I could save on the other ancillary costs of putting together a computer so it could probably be quite cost effective overall. I think I’m just outside of the target market for these chips, my workloads are in bursts so power consumption isn’t a big deal for me. Given that it’s a big deal for others I would expect Ampere Altra to price their chips to a point to go after that market which makes total economic sense. It’s generally not good business to target cheapskates like myself. I do love the chips though.

I'm a proponent of ARM, but out of curiosity, why would Oracle's hardware not be your first choice in accelerating JVM workloads?

Probably because of the memory throughput?

I think that is what’s we’re seeing.

Because it's not as performant per watt as an M2 (and in most cases not even an M1), and it also isn't as performance per rack unit. In a full-depth 1u tray you can fit a crapload of M1 Mac minis which perform better than anything oracle can stick in that 1U.

There is a difference in the type of workload and the ultimate total size (i.e. if you have a single 2-rack workload then the M1 and M2 aren't an option at all -- those only make sense in parallel instantiated workloads and container-based loads).

After experimenting with a shipment of M1's (before they had to be commissioned for an office that was still in the process of being built and furnished) I'm pretty confident that similar JVM performance isn't currently available on anything else if your max allocation per instance is 8-core or lower. As soon as your cores-per-workload is higher the cost/density swings back in favour of Intel again because the Ultra in the studio doesn't fit in 1U and energy/cooling COP becomes on par with classic servers again. Maybe on M2 that's not the case, but when I ran the experiment the M1 was all that was available.

Maybe once the FPGAs for Intel get public bitstreams it might be as efficient on Intel again, but right now, it's not (and also not on AMD), and thus also not on anything Oracle.

Note: didn't test Ampere, but would love it.

We also have ‘normal’ workloads like databases. Not confident in the ports Oracle would provide

With Asahi Linux, I could actually see people shoving M2 Mac Minis in a Server rack in the future.

Does it have ECC?

No.

See https://news.ycombinator.com/item?id=26058061 about the M1. If ECC were added between M1 and M2, Apple would almost certainly have announced it as a feature.

Reply on news.ycombinator.com