Coming Soon – Graviton2-Powered General Purpose EC2 Instances | Hacker News Reader

Coming Soon – Graviton2-Powered General Purpose EC2 Instances

aws.amazon.com

132 pointsjedberg53 comments

I've been extremely skeptical of ARM EC2 instance types getting any sort of traction, as I don't believe most companies would bother to port their software to another architecture. This line pointed out something I never considered:

> Based on these results, we are planning to use these instances to power Amazon EMR, Elastic Load Balancing, Amazon ElastiCache, and other AWS services.

Amazon running their PaaS services on top of their own silicon is really an interesting prospect. I wonder how much hardware is allocated to running their platform services vs. EC2 instances for customers, as there's definitely an opportunity for Amazon to port these workloads and decrease the dependence on Intel.

> as I don't believe most companies would bother to port their software to another architecture

For a Java/Node/Python application, this means changing one line in a docker file, and running preprod/integration tests.

Assuming you have no native dependencies, which most bigger applications have somewhere in their dependency graph.

This is why writing 100% Pure Java is so important.

Heh, and useless. I was about 20 lines into small example. I needed the stat() command... not in java.... sigh.

The performance won't be the same though, as things might be much more optimised for x86-64.

Python libraries frequently aren't only in Python, and use native bindings. Node packages less so, but it still happens.

Java might surprise you, too, especially on EMR. I've found math, compression, and machine learning libraries that reach out to native code using JNI.

All of the major distribution families support Arm archs and the heavy lifting for most of the things that reach out to native code have already been ported.

Native libraries are most often used to make boring things faster, use popular C libraries, and are already ported.

Unless your in-house library is using native bindings for custom code, it very likely will be a drop-in replacement.

ARM isn't so esoteric these days like it might have been 5-10 years ago.

Definitely exciting news! For some reason I was picturing that you might have difficulty building things like xgboost or getting an optimized version of BLAS, but that doesn't seem to be the case. (the former is supported and the latter is important enough that ARM ships their own version)

That's probably why EMR doesn't support ARM-based instances [1].

[1]: https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-s...

Disclosure: I work at AWS

Stay tuned!

> For a Java/Node/Python application, this means changing one line in a docker file, and running preprod/integration tests.

Ironically, the theory contradicts the practice and native code is often the easiest to port on new archs.

I have ported to ppc64be, POWER, armv7 and aarch64 several HPC stacks, it is often the native app made in C/C++/Fortran which is the easiest to port/recompile/cross-compile simply because:

- Their build system has been made from the beginning to handle multiple architectures.

- The compilers for C/C++ are the first thing to be ported and tested on a new arch.

At the opposite, python / node / java embed pretty often native modules hidden somewhere, with hand rolled compile scripts in pip / npm / maven and which are everything excepted robust and portable.

This hasn't been the case for most Java projects I deal with, native dependencies are fairly rare. Can't comment on Python or Node.

I rented a cheap ARM-based machine last year, just for running off-site monitoring from.

My monitoring application was written in golang, and that made cross-compilation trivial - although I guess I'd made a little extra-effort to keep it all native and avoid CGO.

I don't know what the total breakdown is, but at AWS we get a bill every month for our own spend, and managers get it for the whole team. We don't have to pay it, obviously, but it's to show what the spend is.

it’s really cool that AWS internally has this as a feature but there’s entire startups with good funding and customers that exist to solve this exact shortcoming of AWS billing

It's just a question of whether your company is willing to embrace vendor lock-in with AWS. Sure you can do lots of cool things with proprietary AWS services. Probably more than a few customers can save a boatload of money by porting legacy workloads to ARM. But the price is being utterly dependent on AWS and losing control over your cloud bill.

That's not necessarily a bad trade-off, mind you, it depends on what business you're in, what your architecture looks like, and what your org's capabilities are. But it is a trade-off and customers should be mindful of the downsides when they make these decisions.

> I've been extremely skeptical of ARM EC2 instance types getting any sort of traction, as I don't believe most companies would bother to port their software to another architecture.

I think you miss a much bigger elephant in the room: Amazon is too small to get good offer when negotiating with fabs.

May sound weird to people from silicon valley.

Fabs, above all, like reliable clients with stable business because cashflow certainty is of vital importance in the fab business. Small orders are invariably very pricey, but so are big orders from companies that are "come and go" clients, whose own prospects and plans are uncertain.

> Amazon is too small to get good offer when negotiating with fabs.

Amazon literally has custom silicon in every machine they add to AWS now in the form of the nitro system. They are pretty damn big customers that are not “come and go”. If ELBs start using graviton, that’s a sizable subset of the instances in AWS data centers that isn’t even reliant on external customer demand.

Nitro is an FPGA card as I was told, a very different kind of beast.

> damn big customers that are not “come and go”.

I think, they are. There is no guarantee for a FAB that they will not revert to vendor CPUs if over the years their team can't keep their design competitive. And their accounting...

Only way to guarantee so if for them to do a contract with a cash bond — a practice not uncommon in the industry, but the digit will be much bigger than for, say, a commodity 65nm asic, and amount of wafers they need.

They're saying a Neoverse N1 core is ~40% faster than a 3.1 GHz Skylake thread; that's impressive.

Also note that they've changed the branding from A1 to M6g, making ARM a peer of Intel and AMD. So their 2020 lineup will probably be:

M6: Intel Ice Lake-SP

M6a: AMD Rome

M6g: ARM Graviton2

That is correct. Three lines, one for each processor type.

Does the Neoverse core have hyperthreading? I know that vCPU threads on the major cloud providers are hyperthreads instead of real hardware cores.

Disclosure: I work for AWS on EC2 infrastructure (among other things)

No, the Graviton and Graviton2 processors do not have SMT. Each vCPU (or logical processor, for a bare metal instance) is a core.

A1 is not hyperthreaded. Unsure about this new instances, but probably not. From https://aws.amazon.com/ec2/instance-types/

"Each vCPU is a thread of either an Intel Xeon core or an AMD EPYC core, except for A1 instances... Each vCPU on A1 instances is a core of an AWS Graviton Processor."

> They're saying a Neoverse N1 core is ~40% faster than a 3.1 GHz Skylake thread; that's impressive.

That's hugely impressive. I don't remember any ARM processors beating Skylake in single threaded workloads before. To pull ahead by 40% seems almost unbelievably good. (I understand Skylake can clock much higher than 3.1GHz, but I guess cloud instances are mostly around this value due to the high core counts.)

I wonder how much of the performance delta is due to the mitigations for the branch prediction vulnerabilities.

I wonder, as an AWS customer, what's the advantage of an ARM instance over a more traditional AMD64 one?

According to AWS marketing: 20% lower cost and up to 40% higher performance over Amazon EC2 M5 instances, based on internal testing of workloads.

*assuming your programs can be ported to ARM

Since those are your programs you have source code, so yes, you can port them to ARM.

The actual question is whether you want to put that effort or not. You'll have to decide if the savings through instance are worth it or not.

If you're not in the business of writing C, whatever you're doing will most likely just work these days.

I'm curious as well, I wonder if they'll be cheaper compared to AMD64. Would be nice to build a cheap sandbox environment with a bunch of these machines.

If I was to adopt these ARM instances in production, for my services running on the JVM, I suppose I need to ensure things can actually build, integration test it all together, load test it, and review performance metrics.

Then I also need to ensure that all the tooling I use (for example - agents running on the box, other processes decoupled from my service, etc) are all working as expected without functional or performance regressions.

I’m actually not sure what else needs to be looked at... but this is all new work without new value if I look at it on its own. Hard to justify to my CIO.

I don’t know what the pricing model is but I wouldn’t even think of adopting this unless there’s a significant price benefit, allowing me to run on much cheaper hardware.

Even then, it only makes sense to me if you have hundreds or thousands or tens of thousands of hosts provisioned where that price delta really adds up to savings.

This is all conjecture - I’ve never done a migration like this.

Presumably you wouldn’t migrate to these unless you had a good reason.

But it might make sense to build new applications on ARM.

Then don’t you worry that there’s only one real ARM offering and you’d be locked in lest you do the same kind of analyses for porting back to x86/x64?

Some software will run on ARM with very little work. (If you care about performance it might be a little more work!)

Some will involve one hell of a fight with your compiler, libraries, build tools, etc.

AMZN itself is a good early adopter for a new architecture since they have so many services such as S3, SQS, where they have huge fleets of machines running a single set of programs, so the payback time for the port should be low.

We use some arm instances for services that need multiple servers for redundancy purposes but are otherwise low overhead/usage. The cost savings are small but they add up over time.

Wouldn’t a security aspect fall in line here as well? Most of the specter and other attacks are for amd architecture.

I could be off base, would like to know an experts opinion as well.

Spectre has been found to exist on ARM as well.

Does ARM specify speculative execution? I would have guessed that's an implementation detail of Qualcomm or Marvell or Nvidia's implementation of ARM.

Yes, Arm designs not only instruction set, but also processors, which mentioned vendors then put into their SoCs or base their modified designs on. Here's a list of affected processors: https://developer.arm.com/support/arm-security-updates/specu...

General question: Who needs all this high-end, niche(?) services that AWS is cranking out like every week that are presumably even profitable in the long-term?

When you consider the volume of customers using the cloud, even a small percentage of interest can represent huge revenues. Competing over small value adds can win over a customer from a competitor or make the case for hybrid cloud.

Right, but I wonder how many businesses are created based on an 'inspiration' from new AWS services vs. those who aren't already running similar things and don't value the created IP as a competitive advantage; thus just move over to AWS.

In general I agree, many of their offerings are not clear how they're useful.

This one in particular is dead simple, though: ARM instances will be a better value for lots of AWS customers. While it will probably start out as niche, I expect it to catch on fast when people realize that their workloads are already portable and they can decrease their bill.

I was won over to ARM by the raspberry pi. Suddenly I wanted all of my software to be compatible with ARM. Back when the RPI was new I was definitely not alone, because when I did find issues with certain python libraries I usually found the issue opened by another dev — also so it will work on the RPI!

What does the performance comparison mean on this page? Does it mean a similar vCPU count A2 instance is ~40% faster than an Intel based M5 one? Or is the 40% improvement mainly through higher core count?

Anybody know the energy usage of the different chips?

Is pricing public?

I wonder why they didn't launch i3 style instances. Would ARM give benefit in IO heavy workloads?

AWS did some custom hardware for their i3 style instances which would be a significant effort to recreate on a difference processor/system architecture.

Disclosure: I work at AWS on EC2 infrastructure

We'll be launching configurations of these Graviton2 powered instances with local NVMe storage. The largest sizes will have a good bit of local storage, though not as much as I3. There are some really exciting numbers from customers building ultra high performance database engines who have early access. They'll be in the updated CMP322 talk at re:Invent today.

https://www.portal.reinvent.awsevents.com/connect/sessionDet...

Reply on news.ycombinator.com