Amazon's homegrown Graviton processor was very nearly an AMD Arm CPU | Hacker News Reader

Amazon's homegrown Graviton processor was very nearly an AMD Arm CPU

theregister.co.uk

98 pointsrbanffy38 comments

Key takeaway: "The con: it's too damn slow. It does well on the Phoronix Test Suite. It does poorly benchmarking our website fully deployed on it (nginx + php + MediaWiki and everything else involved). This is your "real world" test. All 16 cores can't match even 5 cores of our Xeon E5-2697 v4" [0].

Granted, x86 has seen far more optimization work than ARM, but its unfortunate (but not unsurprising) they can't even come up with 1/3 of the speed of a 2016 Xeon. Amazon will need to rethink pricing, if they don't want this to get pigeonholed into running ARM-specific tasks only.

[0] https://twitter.com/david_schor/status/1067456537264832512

It's a shame that Qualcomm's Centriq was dropped. The performance of that (custom 46/48-core ARM chip) was incredible.

https://www.nextplatform.com/2017/11/08/qualcomms-amberwing-...

Might not be dead: https://www.anandtech.com/show/13597/just-when-you-thought-i...

Cavium ThunderX2 and Ampere eMAG are definitely no worse, probably better.

ThunderX2 is great. It's a shame that ThunderX (ie. v1) had pretty terrible performance because I think that put some people off looking at the current range of Caviums.

Ampere looks particularly promising. Their first release is based on the AppliedMicro X-Gene 3 IP, which they acquired [0]. The first product is a 16nm chip with 32 ARMv8 cores @ 3.3GHz, with 8x 72-bit DDR4-2667 channels (with ECC), and 42 lanes of PCIe Gen 3, in a 125W TDP at a cost under $1000 [1]. They supposedly have a pretty aggressive roadmap, with 7nm parts available next year [2].

They have a pretty solid looking team, lots of former Intel folks [3]. I seem to recall reading they had several hundred employees in total, and at least several hundred million in funding, though I can't remember where I read that.

It would have been far more impressive for Amazon to launch with a product like this, or Cavium's ThunderX2.

[0] https://fuse.wikichip.org/news/776/x-gene-3-gets-a-second-ch...

[1] https://cdn.amperecomputing.com/documentation/hardware/eMAG/...

[2] https://www.theregister.co.uk/2018/09/18/ampere_shipping/

[3] https://amperecomputing.com/leadership-team/

> they can't even come up with 1/3 of the speed of a 2016 Xeon

Ultimately it's the total cost to run your application that's important, not the raw speed of a particular core. So 1/3 speed is no problem if it's 1/4 the price (indeed this is the kind of trade-off you'd hope for in an ARM server).

Sadly from the same tweet thread that quote came from: 'All in all: Amazon having Arm-based offering is a great first step, but it does not seem to be a particularly appealing one at the moment, (at least for me: especially since the cost difference is nonexistent if I was to match my existing performance.)'. So, for this particular application at least, the A1 instances aren't giving a good cost/performance ratio.

Ultimately it's the total cost to run your application that's important, not the raw speed of a particular core.

Not necessarily. A core that's 1/3 the speed can multiply your response time by 3x. https://static.googleusercontent.com/media/research.google.c...

It depends on how CPU-bound you are. If you spend a lot of time waiting for DRAM, disk or network, then you won't notice.

It's a bit like answering e-mail on a Celeron Chromebook compared to your dual Xeon Platinum quad Nvidia desktop. You won't notice the difference. 90% of the time the only difference I notice between my light-and-slow laptop I use for meetups and conferences and my heavy-and-beefy one is that the puny one has no built-in fan.

When running Python code, my very methodologically questionable benchmarks indicate the size of the L1 cache makes a huge difference, so I'll take an ARM64 box with a Core i9 worth of transistors used as cache over the i9 with an ARM64 worth of cache any time.

Disk or network sure, but I don't think you can by any reasonable definition divorce "core performance" from the cache hierarchy and memory bandwidth that feeds it, at least in the context of the GP's post.

It's important to note here that it's not the chip which has to be cheaper, but the whole setup. If a arm chip is 1/3 of the speed, I need three cpus, three mainboards, three switch ports, more ram (because I have to run three OS instances) etc.

Only if you are CPU-bound (which is rarely the case).

No. If the system, for my task, is a factor 3 slower (in the sense it can handle 1/3 of the tasks in a given time), I need three of them. And as a parallel comment points out, this does not solve the issue of latency.

How much time does your CPU spend waiting for IO during your typical tasks?

Please read the whole thread. We are talking about a web server which is a factor of three slower. I really don't care why. The /package/ is factor 3 slower. Assuming that the rest of the system, i.e. mainboard etc., is at a similar cost level, it's not enough that the CPU is factor 3 cheaper. That was my point.

And that is why I said it didn't make sense for ARM to be on Public Cloud, unless it is super cheap or you are on a very specific niche like Cloudflare and do low level optimisation while paying for your own electricity usage. CPU cost of a system percentage has dropped over the years when Memory prices are sky high.

The only contender here is EPYC2.

This could easily just be a lack of optimization for ARM. IIRC cloudflare reported a while ago that they were able to achieve 10x speed ups for some compression and cryptography tasks with relatively low effort.

That doesn't mean much for the typical web-dev though?

Web developers are too busy developing their scripts and webpage. They won't have the time to drop down to the C intrinsic or Assembly level to speed things up.

Furthermore, the problem could be fundamental to this chip. There is only a 1MB-per-core last-level cache (L2 cache). That likely isn't enough for anyone running your Linux / Database / PHP / Web Server stack. The chip does outstanding on "small" benchmarks like C-Ray, but it does horrible in practice, where you have more code (probably more than 1MB of "hot" code) running on the chip.

Web apps spend a lot of their time waiting for the database, for memcache, from their queue servers and so on.

Is it about power?

https://twitter.com/eastdakota/status/976560820611031040

Presumably ~1/3 less power means AWS gets a better margin on these.

Possibly. If your workload is IO bound, the various offloads to accelerators will be much better for you than a faster, hungrier CPU that'll do more NOPs waiting for the rest of the machine to catch up than a spartan ARM core.

"Semiconductor industry watcher David Schor shared SciMark and C-Ray benchmarks for the 16-core Graviton. In the SciMark testing, the AWS system-on-chip was twice as fast as a Raspberry Pi 3 Model B+ on Linux 4.14."

Yip, at 4x the core count it is only 2x as fast. I also suspect that 2x RPi's would be less power usage than one of these platforms.

But then this is it's first generation dip into this area and they will be seeing what people actually want to run and will fine-tune their custom silicon from that for version 2.

As for pricing, they need to make this more tempting so they can get the metrics they need to see what version 2 should focus upon.

What I'm wondering is would such a chip marry up with an FPGA and present a more flexible and appealing option.

[EDIT ADD] It would appear this set of tests are heavy on single threaded usage and with that, fair results and like many, got taken in on the jonalistic spin. So it's in this test effectively comparing a single core, so mentioning it is a 16 core cpu compared to a RPi, does twist how the results are viewed.

> Yip, at 4x the core count it is only 2x as fast. I also suspect that 2x RPi's would be less power usage than one of these platforms.

That's almost certainly wrong information. Cortex A72 is an out-of-order CPU unlike Cortex A53 in RPI3. Also Graviton is running at 2.3 GHz vs 1.4 GHz for RPI 3b+.

You'd expect 16 core Graviton to outperform RPI3b's 4 core A53 by about 8-15x.

A little digging and it would appear this set of benchmarks are single threaded, so a 2x speed improvement given the clock speeds makes sense.

I have looked at some other benchmarks, and it looks like this offering only pays off on certain tests (N-Queens being one), how that pans out with real-world usage is what we will find out over time. Though the aspect that even in a few tests that this stands out, makes it viable for some real-world usage. Time will tell and at least even if it pays of in 1 out of 10 tests, those tests will translate into real usage. But one shoes does not fit everybody, but always good to have a choice and for that, this is good.

If you look a couple years out, if there's an a2, etc. there are a few things that could notch up performance. There's the 7nm process (unless Amazon's a super early user of it like Apple--seems unlikely), there's newer ARM IP, and they could have accelerators off to the side for compression and public key crypto, as the Seattle chips were planned to do.

I don't think the goal of ARM server chips should be core-for-core near-parity with Intel big cores anytime soon, because those have much bigger die area and power needs, so you potentially pay in number of chips per wafer, yields, density, and power use. Instead you keep something reasonably small, try to get single-thread perf good _enough_ for a chunk of apps, and try to provide much better cost per unit throughput.

(They could also go for efficiency improvements to further cut power-related costs or make up for more consumption by faster cores. These seem far-out but theoretically: mobile-like big/little arrangements so light loads use even less power, or boxes with low-power RAM, or (least far-out) more of their Nitro IP moved onto the SoC. Who knows, just saying it's a wide-open design space.)

If this takes off enough for Intel to really pay close attention (not a certainty!), they do have their own cool scale-out experiments in Xeon D (an SoC Facebook originally wanted with relatively few cores and low clocks, with some SKUs including compression/crypto acceleration) and their Atom server chips. Getting something good and Intel responding with something better isn't the worst possible result.

What versions of nginx and php? Was caddy given a try? I have no idea how much optimization was given towards PHP on non-x86. But I've seen some videos and presentations that show significant improvements in the latest PHP versions. Lower on-chip caching and behaviors could account for a bit of this wrt the latest version.

I'm more surprised it isn't roughly 1/2 the price it's being pushed at. It seems to be more expensive than the x86 side.

Phoronix recently did some more in depth benchmarking and the system does indeed suck pretty badly in terms of PHP performance. And Apache. But they seem to be pretty good at things like calculating Mandlebrots or 7-Zip compression at least.

https://www.phoronix.com/scan.php?page=article&item=ec2-grav...

> non-NUMA Forgive my ignorance, but isn't a "non-NUMA" architecture just UMA?

Came here to say the same.

More performance benchmarks: https://www.servethehome.com/putting-aws-graviton-its-arm-cp...

Anyone know what kind of ARM license Amazon has? Just a processor license or a full architecture license? I'm curious if Amazon is doing the actual CPU design.

They're definitely using stock A72 cores now. Amazon is very secretive so we won't know if they're designing their own cores until they ship. (Unless they post job openings for "custom ARM core designer".)

Thanks. I saw it was based on A72, but I couldn't tell if all the discussion of customization was referring to modifications to the core itself or just the SoC.

I note that AMD's ARM chip came out not long before AMD must have found out internally that Zen was very good performance-wise. That's probably why they stopped working on it. Why compete with yourself?

I'm more curious on how Amazon makes these chips. Are they just licensing the Arm IP with some customer specific changes. Then pushing it off to someone like Global Foundries to manufacture?

Yeah, they (Annapurna) license various IP blocks, integrate them, and it's probably fabbed at TSMC.

Discussed https://news.ycombinator.com/item?id=18539297

What you linked is the original press release. What this story is about is the development history of the chip.

It warrants its own discussion.

Reply on news.ycombinator.com