AWS launches ARM-powered Lambdas

ARM is cheaper on AWS per the unit of work done, in the general case. But there should be no assumption that ARM is that snappy little brother that will launch your lambdas in no time while x86 is still starting up, this is the kind of idea that I get when reading the comments mentioning ARM. It won’t launch your virtualized environment that noticeably faster. It’ll just do more work for less money, and 40% is probably not the figure that you’ll see in real life.

It will probably take a while to ramp up - it's explicitly opt-in and I expect there's a huge amount of existing code that creates lambdas without being able to specify the architecture. For starters example Terraform and Cloudformation don't support it yet. And if you're using a 3rd party layer you need to wait for them to publish a layer that explicitly supports that architecture.

> This seems pretty huge ('19% better performance at 20% lower cost')

I believe the performance boost will not be meaningful and the only important factor is cost per GB*s, given that most Lambda workloads are IO bound or simple event handlers that barely take 1s to run anyway.

Nevertheless this is exciting stuff, specially given that NodeJS might be lambda's most popular runtime and thus we might only be a toggle flip away from a 20% discount.

From folks I know at AWS, services are all being heavily pushed to use Graviton to run their services, to the point of having to justify why they couldn't. The cost savings internally are just as significant.

With Nitro they disclosed that every EC2 machine deployed in the last 5+ years has at least one ARM chip because each server has a nitro card.

So reading the tea leaves... the first generation of EC2 ARM (a1) could have been almost as simple as two nitro cards hooked together.

Graviton2 is likely a bigger chip than what they're sticking on Nitro cards, but if Graviton1 was just the overhead for each Intel machine...

> Where in the world is GCE and Azure with ARM processors to compete?

I can't speak for Azure, but Google Cloud looks like they've gone all-in with AMD's EPYC platform for their premium option, and offers the E2 instance family for those that want cheap compute and don't care what the underlying platform is.

> Customers aren't going to wait much longer...

I can't imagine anyone outside of hobbyists and tiny startups changing their entire hosting platform due to it not having an ARM offering.

Microsoft is partnering ( as well as being an investor ) with Ampere. And Google is going with AMD EPYC offer vCPU as single CPU Core instead of a thread.

I mean Amazon has been preparing for this since 2015 when they acquired Annapurna Labs.

Considering TSMC somehow increased their planned capacity for 5nm by 50%. There may be a chance you will will see that going to Microsoft of Google coming in 2023.

> AWS’s dependency management is literally bundle it up locally and upload a zip.

That's true and not great IMO either, I also wish the zip wasn't necessary - in the 'bootstrap' binary case just let me upload the binary! - but in case you're not aware of layers (just guessing from 'dependencies') it can be improved a bit:

https://docs.aws.amazon.com/lambda/latest/dg/configuration-l...

> What happens if a ARM package can’t be built locally.

An Arm package can only not be built locally if you decline to use tools that'd allow you to do so?

> Then again, I wonder how much code will randomly break, plus AWS’s dependency management is literally bundle it up locally and upload a zip.

I much prefer using Docker containers since they added support for this. There's an established toolchain which supports things like this and it fits well into popular deployment pipelines from things like GitLab/Github.

As already mentioned, Docker images are also available and that's my preferred solution.

If your concern is more about having access to ARM hardware, CodeBuild is a reasonable option. CodePipeline is more expensive than it should be given the feature set, but CodeBuild is cheap and easy to use directly.

Alternatively, I've been intending to move some of my build to Lambda itself. Easy to bootstrap if you use an architecture independent language like Python to script your build. And if your code takes more than 15 minutes to build, should it really be a Lambda function?

> This is awesome, saving that much money can be great for startups.

This doesn't sound very realistic. Cost-conscious organizations would hardly consider lambdas as a reasonable option unless you're well within the free tier limits. Otherwise lambdas are far more expensive than simply handling requests directly with a service running on EC2/ECS/Fargate.

This is an important question and I hope they clarify it.

Lambda does not use hyperthreads/SMT on either x86 or ARM.

It's likely to be the same because an ARM vCPU has similar performance to an x86 vCPU. A lot of people overthink this.

Ignoring SMT (as others already mentioned it's not used for AWS Lambda), I also expect different vCPU scaling and they also hint at that in the detailed blog post for this feature [1]:

> Workloads using multithreading and multiprocessing, or performing many I/O operations, can experience lower execution time and, as a consequence, even lower costs.

[1]: https://aws.amazon.com/blogs/aws/aws-lambda-functions-powere...

> Ideally on Serverless infrastructure the system would "move" jobs around so they run on the optimal platform for what they are doing continuously? (Based on price or based on raw compute etc)

AWS Lambda supports about half a dozen runtimes, some based on interpreted languages, and some based on compiled languages. Thus you can't simply move golang or rust code between completely different architectures and expect things to work.

Also, one of AWS Lambda's usecases is having your runtime call precompiled binaries. I'd be pissed if my Lambda's ceased to work because AWS decided to run my trusty python-calling-C++ lib Lambdas in ARM just because they want to push people towards graviton.

yes, it's awesome. I also asked about this here: https://news.ycombinator.com/item?id=25267921

hopefully we'll see ML chips on Lambda soon too

can't you cross-compile x86_64 binaries on an arm machine?

I'm using fargate very happily (after price reductions).

For ubuntu / python-slim etc docker images - what changes are needed to let them target ARM (if any).

Ditto. My main project & CI are ready to be (cross) compiled to ARM, I'm just waiting for Graviton-powered Fargate.

That subset of opcodes is the ARM instruction set. So yes GCC will have no problem. Even the SIMD extensions are under ARM's purview.

The neural processors, GPU's, TPU's etc. aren't part of the CPU anyways (they're part of the SoC).

This fragmentation has some downsides but having a good compiler isn't one of them. It's more likely all the different ways these SoC's handle booting, connecting to peripherals, etc. This will create a huge burden on something like the Linux kernel if it attempts to support them all (for evidence, check out the recent work on Linux M1 support.. fascinating stuff).

ARM itself is proprietary, including the ISA and several micro-architectures that implement the ISA.

I don't think it's shocking that folks who licensed a proprietary ISA, and then paid a stiff license fee to implement their own microarch off of a proprietary reference microarch, would keep their chips proprietary.

risc-v is the only hope in this arena, and luckily David Patterson is generally signal enough that something might be a good idea.

Use docker buildx, it's the Buildkit engine which fully supports cross platform builds. It's essentially just what you mention, add a CLI flag and it spit out an ARM image even on your Intel machine. See: https://www.docker.com/blog/multi-arch-build-and-images-the-...

Depending on what your idea of a computer is, the reference carrier board for the Raspberry Pi 4 compute module has a pcie x1 slot. The jetson family of developer kits have pcie connectivity. The 4GB Jetson Nano has an m.2 E key slot which has a pcie x1 lane, with the right adapter available from Amazon or alibaba or where ever, you can plug in a pcie card of your choice. The Xavier NX has two m.2 slots, which again, with adapters can connect to pcie cards. Though I’m not sure of the use case as the jetson line has fairly capable gpus built in.

It seems doable. You can edit it under runtime settings: https://aws.amazon.com/blogs/aws/aws-lambda-functions-powere....

so...my use case for GPU lambdas is for inference. Say 500 inferences/day ~1 sec/inference, has to be real time as the output is user facing.

Can they really spin up quick enough to actually respond in real time? I don't really want to have to keep them warm as I'll end up paying through the nose for it.

It works with Go: https://awsteele.com/blog/2021/09/29/graviton2-arm-comes-to-...

You can run an arbitrary binary. Keywords for searching docs are 'bootstrap' (the entrypoint name) and 'custom runtime'.

In this way you can also run whatever version of the provided language runtimes you want, not limited to the official ones if you want newer python syntax or node stdlib features than are available, for example.

Not at all. If you want to run assembly code that targets a specific architecture you need to know what that architecture is going to be. At it’s heart Lambda is just a Linux container running a CGI app.

Unless you want to restrict lambda to only run interpreted code with interpreter & binary dependencies provided by AWS, or run things in emulation, you can't really hide it.

It's a question of how much you can do with the tool. If Lambda only supported, say, pure Python or Node libraries you'd run into cases where you couldn't use it or would see big performance penalties. Being able to push a container with, say, a high-performance image processing library which your Lambda needs makes it a lot more versatile but also forces you to think about things like x86 or ARM.

Overall, I see that as a win: you can get started by putting code in something like Python or JavaScript and not caring but when you do hit more complex dependencies you have a path which is an extension of what you've already been using rather than having to switch to a completely different service like ECS.

How would you use something like Go without knowing what you’re compiling for?

There are time where AWS makes sense, just like there are times where self hosting makes sense.

The cost that with self hosting you instead sink into paying for SREs? Or getting paged at 2am?

If you're not trolling, what function-as-a-service system are you self-hosting? How much time are you spending on ops?

10,000,000 300ms invocations with 1gb of memory is under 30 dollars. That's dirt cheap, over a month of CPU time that you can burst to tens of thousands of concurrent executions without caring.

Not saying AWS doesn't come with a premium, but lambda isn't exactly the poster-child for expensive compute.