Ampere Altra Dev Kit has launched 32, 64, 80 core arm64 processor | Hacker News Reader

Ampere Altra Dev Kit has launched 32, 64, 80 core arm64 processor

ipi.wiki

132 pointsmariuz106 comments

What I really want is an ARM server that can accelerate our JVM workloads as much as M1/M2 does, without regards to power density.

Why? In colo you’re usually not as concerned about density. We’re renting a chunk of rack space for a bottom basement price of $200/month and we can easily stick a router and compute / storage nodes with 64 TB of disk and boat loads of memory in that rented space; plenty for our needs without coming anywhere close to the power allocation.

When we were purchasing a few servers for our colo, we ultimately just went with AMD because the price per core was there. It ends up being less performant than an m2, maybe m1 ish.

Ultimately we’re a small market though. The ‘sexy’ thing to do is waive your AWS invoice size around at other hipsters during cocktail parties; not to own your own hosting even though you’ve paid for it serval times over. I don’t see this changing anytime soon.

How does M1/M2 accelerate JVM apps, and in general how do you optimise your JVM apps to make full use of the resources?

I doubt M1/M2 do anything special for the JVM, it's just that they are overall better performing for the same power. In a server setting though I don't see them having the same advantages.

It has lot higher memory throughput, so mid-range M1 chip produces results comparable to my 3970x Threadripper on some of my workloads (scala compilation) while it consumes just a fraction of the power the Threadripper requires.

With Asahi Linux, I could actually see people shoving M2 Mac Minis in a Server rack in the future.

Does it have ECC?

No.

See https://news.ycombinator.com/item?id=26058061 about the M1. If ECC were added between M1 and M2, Apple would almost certainly have announced it as a feature.

I’d love to see a cluster of M2 SOCs placed together on a single PCB with built in networking. Maybe direct PCIe to PCIe communication between the SOCs in a hypercube topology. Having multiple CPUs spread out around the box will help with cooling which is my limiting factor for power density. My tasks are data parallel and CPU bound with low ram and disk requirements. I could probably design such a PCB myself but the cost of doing so will unlikely be paid back unless my compute needs go up by another order of magnitude. I am many years away from that being viable. For now I’m sticking to AMD chips. Unless someone else wants this as well for which I could amortize the costs.

It would seem this devkit is even more ideal for you? 80 cores and buy as little memory as you want. Not to mention it does not need clustering and runs any aarch64 distro of your choosing

I checked out the benchmarks and it seems that the 80 core version would be ~6.5x M2 and would indeed be cheaper on the assumption that the only way to get M2 chips is to first buy a Mac. But if (an I know it isn’t going to happen) M2 was at the Jetson Nano price of $150 then it starts to become attractive again. It’s 2.3x faster than Ryzen 7900X, I’m not sure if the dev kit has a price performance advantage over that and I’ve seen the Ampere Altra chips on their own for much more so I’m thinking the dev kit has a discount which probably comes with a purchase limit.

The dev kit has no purchase limit. Same is true for the dev platform workstation 32, 64, 80, 128 core (and a 96 faster cores option if you ask nicely).

https://www.tomshardware.com/news/ampere-64-core-arm-worksta...

For those keeping score the Ampere Altra 128 core 3GHz Supermicro telco edge server being used in 5G base stations benchmarks equal to 100 RPi4 and 22% more energy efficient.

https://youtu.be/UT5UbSJOyog

Interesting, when I first saw these chips my thought was Epic performance for Epic prices, now it looks like 2x Ryzen performance for 2x the price. Which is a huge improvement, I could save on the other ancillary costs of putting together a computer so it could probably be quite cost effective overall. I think I’m just outside of the target market for these chips, my workloads are in bursts so power consumption isn’t a big deal for me. Given that it’s a big deal for others I would expect Ampere Altra to price their chips to a point to go after that market which makes total economic sense. It’s generally not good business to target cheapskates like myself. I do love the chips though.

Have you tried Ampere? It's pretty good as far as ARM servers go, and even has acceleration for hip stuff like Tensorflow and Pytorch. I personally haven't experimented with Java apps on it, but it should have a high enough IPC to compete with other similarly dense ARM options.

And of course, Nvidia has their 72x2 core Grace chip coming soon. It will be very interesting to see how that stacks up against the ARM incumbents.

Actually pretty excited to NVidia offering, but we spent our budget :(

i could see it changing a bit if oxide.computer is successful in shipping lots of units, since they've already got major cool factor with their podcasts.

I'm curious about these cocktail parties you speak of - over here, in-person tech meetups have been pretty rare since covid

Even post covid?

I'm a proponent of ARM, but out of curiosity, why would Oracle's hardware not be your first choice in accelerating JVM workloads?

Probably because of the memory throughput?

I think that is what’s we’re seeing.

Because it's not as performant per watt as an M2 (and in most cases not even an M1), and it also isn't as performance per rack unit. In a full-depth 1u tray you can fit a crapload of M1 Mac minis which perform better than anything oracle can stick in that 1U.

There is a difference in the type of workload and the ultimate total size (i.e. if you have a single 2-rack workload then the M1 and M2 aren't an option at all -- those only make sense in parallel instantiated workloads and container-based loads).

After experimenting with a shipment of M1's (before they had to be commissioned for an office that was still in the process of being built and furnished) I'm pretty confident that similar JVM performance isn't currently available on anything else if your max allocation per instance is 8-core or lower. As soon as your cores-per-workload is higher the cost/density swings back in favour of Intel again because the Ultra in the studio doesn't fit in 1U and energy/cooling COP becomes on par with classic servers again. Maybe on M2 that's not the case, but when I ran the experiment the M1 was all that was available.

Maybe once the FPGAs for Intel get public bitstreams it might be as efficient on Intel again, but right now, it's not (and also not on AMD), and thus also not on anything Oracle.

Note: didn't test Ampere, but would love it.

We also have ‘normal’ workloads like databases. Not confident in the ports Oracle would provide

There is something interesting happening here. For the longest time we've been doing everything we can to efficiently share a CPU with a low number of cores across a large number of threads. But as the number of cores increases we have the opposite problem: how to efficiently make a number of cores cooperate on the same task. Once the number of cores meets or exceeds the number of threads you can do away with a lot of the tricky bits that make our machines messy and insecure, a CPU per thread is so much cleaner. Especially if you can throw in some more isolation. And once you go down that road giving each CPU its own dedicated on-die memory is a relatively small step and you end up with a cluster-on-a-die that scales up and down from a power consumption point of view with the requirements because you can simply power off those chunks of CPU/RAM that you are not currently using.

Sort of like the ideal machine for the Qubes OS.

This problem has been solved - albeit sub-optimally for general purpose computing.

GPUs are massively parallel processors with 1000s of cores. While CUDA isn’t the easiest to work with, some Python libraries such as JAX and Tai-chi are attemting to remove the bar between CPU and GPU computing completely.

A program using either cab transparently switch between a CPU or GPU backend.

GPUs do not have thousands of physical cores, though. What's called a "CUDA core" is essentially a programming slash marketing abstraction for a SIMD lane. A closer analogy to a CPU core would be a Streaming Multiprocessor (RTX 4090, for instance, has 128 of them). But that comparison is still moot, because GPUs are simply not designed for executing branchy scalar code. They'd be laughably slow at it, so outside of easily vectorizable code that already takes advantage of SIMD instructions, I don't see how you could offload any CPU tasks to a GPU.

I think the major architectural leap would be MIMD with a standard ISA.

I think something along the lines of a Pentium Pro or an ARM core. Pentium Pro had 5.5 million transistors. A modern CPU has about 1000x more, so about a thousand Pentium Pro-grade processors would fit in die like a modern 7770X.

I'd take that over my GPU any day.

The hard and expensive part is, obviously, memory, cache, and interconnect. The even harder part is software. And the less hard part I'm intentionally oversimplifying is power consumption.

I'd love to see solutions like that. But what you are talking about is probably a niche market.

I don't think so. I think the core problem are network effects and momentum.

If someone could wave a magic wand, and there were OS, app, compiler, video game, etc. support for both MIMD and current architectures, I think MIMD would take over overnight.

Most of what computers do is ridiculously parallel. From each browser tab getting an isolated CPU, to having a spreadsheet spread out among cores, to rendering fonts in a document.

However, given a universe with trillions of dollars invested in the status quo, a disruption would need some sort of rather complex pathway, with some niche markets, some growth strategy, etc. As someone pointed out, Intel tried with Phi and failed.

I think the big driver could be security.

https://en.wikipedia.org/wiki/Multiple_instruction,_multiple...

An example of MIMD system is Intel Xeon Phi, descended from Larrabee microarchitecture.

https://en.wikipedia.org/wiki/Larrabee_(microarchitecture)

Its x86 cores were based on the much simpler P54C Pentium

https://en.wikipedia.org/wiki/Xeon_Phi

That is the core from the generation before the Pentium Pro. Larrabee was supposed to be a GPU, wasn't good enough to compete at that, then they rebranded it as Xeon Phi but cancelled it a few years ago.

I tried buying those but they wouldn't let me.

Well, FWIW xeon phis are about $40 on ebay now. Theyre obviously not much more than toy's now.

While there is a trend towards GPGPU this isn't there at the level where you can just go out and buy it like the dev kit in TFA. GPU compute capacity is still limited to a subset of all the use cases that you can use an ordinary CPU for and even though there is a chance that in the future the two will converge for now they are far enough from each other that I don't see the gain in the same sense that a large number of 'regular' cores would.

The fact that programs can switch between those two execution contexts seamlessly is a nice benefit but still isn't the same as having many ordinary CPUs. I've used GPUs extensively and I'm really happy to see their capabilities more integrated in everyday languages but they are at best for now a co-processor like device, a chunk of hardware that you offload specific parts of your workload (typically: the numerical chunk of it that is massively parallel).

This is a shot in the wrong direction. GPUs are accelerators for code that, ultimately is run by CPUs. They might be headed to be more all-purpose computing hardware, but they aren't there yet, most importantly, from OS perspective.

To elaborate on this: OS creates processes, assigns ids to them, assigns other physical resources to them s.a. association with namespaces (which later gives them user permissions, network access, virtual memory access, filesystem access etc.) and then these processes are associated with some GPU resource.

If and when OS will start creating processes entirely on GPU, then it will make sense to talk about how GPUs are solving the problems of threads per core etc. For now it's a moot point.

This is not said to discourage though. I really feel like CPU-centric model of what we call "computers" is not a good one going forward. The "periphery" is growing smarter with each generation, and wants to do its own computing, and spread its load somehow, and we keep coming up with ad hoc solutions that don't mix well with CPU-based concurrency, s.a. async I/O or CUDA. We really need a different concept of concurrency that would be more uniform and at the same time more flexible across different devices that can do work concurrently, and this, interface if you will, must come from the operating system, not as a user-space library to be truly effective.

>because you can simply power off those chunks of CPU/RAM that you are not currently using.

That is how IBM has handled their mainframes for quite a long time. You can hot-plug them in/out.

That is what a server processor is. A Beowulf cluster on a die.

Not quite. Beowulf clusters tend to use regular networking interfaces for IPC (and sometimes more specialized ones). This would not even necessarily have to be one die, that doesn't matter so much as long as it gets packaged as a single SKU. All of the interconnect would be the same as it is on this 80 core sample here. Typically I have about 400 processes running on a machine with several hundred more threads. That's within striking distance of this kind of device. Just a few more iterations and we'll be there, and a multi socket solution would already get very close. The next step would be to integrate the memory.

Ampere Altra 128 core telco edge server benchmarks = 100 RPi4 Beowulf cluster and 22% more energy efficient. https://youtu.be/UT5UbSJOyog

There is no such thing as on-die memory. DRAM is made by a completely different semiconductor process and cannot be integrated with large-scale logic. The closest thing to memory-on-die is/was eDRAM which scaled up to at most tens of MB per chip, but at 7nm even IBM has abandoned eDRAM and gone back to SRAM.

Not DRAM but SRAM would be possible and there are many processors with on-die L3.

If you only need ~1MB per core, L2 already gives you that.

I worked at an ASIC foundry in the late 1990's. We had an embedded DRAM option but it was basically the DRAM process with standard cells ported to the DRAM process. We only had a couple of customers ever choose it because the logic / standard cells were much slower in the DRAM process compared to the normal ASIC process.

https://www.gigabyte.com/Enterprise/Server-Motherboard/MP72-...

Here's another offering from Gigabyte, dual socket and supports the 128 core processors. EATX formfactor.

Gigabyte's servers built around these are also pretty awesome. All the bells and whistles you'd expect from server gear, but with a lower idle power draw. https://www.gigabyte.com/Enterprise/ARM-Server

Yes, but where can you actually buy these? What's the total cost. I see on this page "request a quote" which usually means it's not worth exploring for a home user.

Nice to see these kind of systems finally get into a reasonable price range. Not raspberry pi-cheap, but not Apple-expensive either.

edit Comparing apples and oranges — this is probably closer to AWS Graviton systems, but in a desktop form factor.

$33/core is way better than most arm offerings for sure.

FYI the 80core variants are available at Hetzner for about 200 EUR/m - running at the full 3GHz

Have you tried it? I’m curious how it performs. It seems like it’d make a decent media transcoding workhorse.

I'd love one on my desk.

I saw them in their offerings, but didn’t try them. A minimum 300 EUR payment is too much to then figure out my stuff doesn’t work or scale how I expected. I wish I could use a trial instance for like 3 hours first.

Same here. I'd happily pay the monthly rent if it's worth it, but not if I find after a few hours testing that it's slower for my purposes than 2 AX101s (2 because of the price ration and SSD sizes).

I might try it on Scaleway, but I wouldn't be surprised if the Hetzner machine has different performance characteristics.

Oracle gives forever free 4 vCPU w 24GB RAM A1 (Ampere Altra) instances.

Thanks!

I benchmarked an "embarrassingly parallel" simulation on Hetzner machines this week. I saw 50% more performance per dollar on the Ampere-based RX170 than the Ryzen-based AX101. That impressed me a lot.

Do you have any numbers on how these cores do compared to AMD ones for JVM workloads?

Scaleway has AMP2 instances which are VMs using those cores. You can get a 60 vCPUs one at 140€ / month. And, you can pay by the hour, too: https://www.scaleway.com/fr/amp2-instances/

(also, Oracle cloud has them in their free tier).

That's pretty beefy for what they call an IoT dev kit. What are the IoT use cases for a machine like this?

If I had to guess, edge compute for networks of far less powerful IoT devices.

Vehicles, industrial control.

Neither seems to ned 80 cores?

If you think of a modern car as a miniature data center on wheels you wouldn't be too far off, and cars aren't the only vehicles.

Idk for most safety-critical stuff you'll run a realtime OS, and all important sensor polls will be pinned to a core. I assume industrial controls could have 40-or-so sensors that you would want to minimize latency for because if you miss a reading the possibility of something breaking/someone dying opens up.

Something like this isn't being used for that kind of realtime control, the memory hierarchy is too hard to nail down in a way that doesn't negate dedicating cores to tasks. It's probably meant for 'embedded' applications like storage appliances.

GM uses it in the Cruise Origin autonomous robotaxis.

Do they use the cores to babysit sensors, or as general compute?

to make the car go More here

https://www.linkedin.com/feed/update/urn:li:activity:6977673...

Judging from that, like I said not using the cores to babysit individual sensors using hardware separation of many cores to simplify hard real time constraints, but instead as a block of general compute.

I think maybe they just want to handle small API requests for many millions of devices at once.

IoT just means embedded. Cell tower, maybe?

That is a really good price for that many cores. Especially the 80 core variant competes favourably with much more expensive (last gen) 64 core Epyc Rome CPUs.

Single gigabit NIC? I don't get why they don't include more, given the 3 x16 pcie slots...

I think that's a reasonable decision. Devs who need lots of networking throughput usually have their own NICs cards from a preferred vendor (with the necessary features) anyways.

I don't think a single gigabit nic is really reasonable on something as light as a raspberry pi, and this sure isn't that. Multigig has been out for years and ten gig for even longer. It's kind of ridiculous that things are still stuck on gigabit.

I just specced out a new network for a friend's business (they call it an intentional community.. ~50-100 people on-site each day, most are digital nomads).

They were willing to spend whatever it took to provide reliable WiFi throughout the ~20 acre property. Equipment cost was concern #4 or #5.

We still went with gigabit fixtures and not 2.5G. The ISPs were only providing a 200mbps pipe (symmetrical), intra-LAN communication is not super common in this type of deployment, and no client devices currently on-property supported 2.5G.

So, yeah, for us, it still made sense to go with gigabit. Even when we look out over the next 5 years (expected lifetime of this new equipment) we didn't see the need for 2.5G growing _that much_.

> It's kind of ridiculous that things are still stuck on gigabit.

I’ve been under the impression that there are patents (expiring soon) keeping the cost of 10G hardware high enough to prevent it from becoming the default choice. I don’t have a concrete source for that info though.

If you look at "Developer Platform" the prebuild model. There's a option to add a 4x10GbE NIC.

They offer a 4x10Gb NIC in their larger offer on the bottom of the page. I'd guess they would throw that in, if you throw some more money at them.

The other side of this is that I'd rather have the slots and a minimal on-board NIC so that I can install a 100g nic of my choosing. I hate motherboards where they force you to take 2x 10G rj45 ports that I'll never use and which burns power until/unless I can figure out how to disable them and recover the power.

Do they really burn that much power if not active? I had naively assumed that there was a simple detection circuit to keep most of the equipment deactivated until use.

An sfp slot with an unused module does pull power from the switch/nic, but that shouldn't be an issue if you pull out the module. (I don't know if rj45 based connectors draw power)

Optional 4x10GbE on the COM-HPC Ampere Altra module.

Haskell and ghc don't work on these devices as aarch64 isn't as well supported as amd64

Give it some time. The more these devices get available the better the software gets. I remember back when AMD64 cpus freshly emerged and you would try to compile some bigger or special software it simply didn't work and you had to compile it for 32bit, e.g. Wine.

At this scale you are probably going to use Erlang instead

Indeed, that's a very good match. Better still because it would allow you to pretend you have larger machines than you really have at the penalty of some slowdown.

There are always rough edges on other platforms, but these machines are good workhorses -- and would be an excellent choice for further development, to smooth out aarch64 support.

I wish these machines (or Apple Silicon with Asahi Linux) had existed at the time I worked on GHC and aarch64 support was starting to take shape. Would have been a game changer in terms of productivity...

This is exciting.

I did have a 32 core Threadripper.

I spend a lot of time trying to write parallel code. On my 12 core intel NUC (a mobile CPU chip) My bank simulation shards money between threads and can handle 700 million transactions per second. I am sure you could with more optimisation increase this and it scales per number of threads due to the sharding.

I've got to assume you're not persisting anything to a database :-)

Not yet, when I add a Java NIO nonblocking socket server, the throughput drops to 240k per second with 6 transaction generator threads and 6 server threads.

Adding persistence would drastically slow things down.

Assuming there is no powercut to multiple availability zones, you could keep data in memory on one machine in a circular buffer and then persist every second on the other availability zone. Depends how long it takes to restore power.

Serious question: what is a bank simulator? Isn't it just adding and subtracting numbers from accounts?

same here, and any related search failed or it's already pointing to this thread.

When I play around with multithreaded concurrent algorithms, I try solve a simulation of moving money between accounts in a thread safe way. It really is just deduct from one account and then add to another account.

If you don't use a mutex or a lockfree algorithm, you can end up with more money (money creation) or money destruction (money missing) when you transfer money between accounts and there is another transaction to the same account in a similar time frame. This is due to a race hazard where addition and subtraction are each individually 3 operations and if they interleave then they cause the writeback of an incorrect result and omission of another result:

  thread 1             thread 2
  read 1000 
                    read 1000
  calculate 1000 - 25
                     calculate 1000 - 50

           write 950
  write 975

This simulation has money creation, because the two transactions don't see eachother.

So I add the money up at the end of the simulation to see if it is equal to the money that the simulation started with.

When I say money sharding, I am not sharding by account. If an account has 12,000 in it, and I have 12 threads, each thread stores 1000. The fast path is a transaction goes to that thread and that thread handles the transaction if the amount of money being deducted is less than or equal to 1000. If the transaction amount is greater than what is available in one thread, then it has to be routed to other threads.

I never generate a transaction greater than what is available than all threads, so that part always works, for routing to accounts that have enough money.

An extremely slow path would have to transact a bit from multiple threads until the transaction is complete.

I've never worked on fintech or banking software, so I don't know how it works in practice but I did try implementing an order matching engine (which from my perspective is just a sort ascending + sort descending of participants bids or asks) I haven't worked on parallelising that yet.

You've invented a distributed smurfing/structuring solution :)

I'd never heard of COM-HPC before.

In this design, it appears that the memory slots are perpendicular to the PCIe slots, and thus perpendicular to typical server front->back airflow. Is this a common design element in all COM-HPC CPU boards? If so, it seems like a flawed design..

COM-HPC only refers to the (hand waving a lot here) dimensional form factor and pinouts, of course, so there are some different variants. I don't know if I'd call this design typical but it's not unusual; I think I've seen ones with soldered RAM and also ones with slots that are parallel to the PCB, sort of like a SO-DIMM slot -- and also actual SO-DIMMs, too! They're pretty niche in practice.

This thing is going to be a monster running ESXi arm edition.

VGA output? Come _on_.

Reply on news.ycombinator.com