Amazon has more than half of all Arm server CPUs in the world
theregister.com
theregister.com
None of the other cloud providers are using ARM at this scale yet. They’re using Ampere chips, whereas Amazon invested in building their own - it tells you the level of commitment Amazon have here.
My employer is moving workloads to ARM on AWS as it’s much better for price/perf, and our shiny new M1/M2 Macs run the same binaries. We’ve had very little issue doing the shift - Amazon Linux and Debian both support ARM well.
Must be really putting a dent in Intel’s data centre sales though.
Does that mean you're all running asahi linux, or are you somehow running linux elf binaries on macOS?
Or are you using VMs, just like the x86 + docker desktop folks are doing to run the same binaries on their laptops and servers?
And yes - it’s exactly the same as you can do if you use x86 servers and dev machines.
It’s just the fact that we can also do this for ARM on hardware that doesn’t suck that makes using ARM in production more practical for us.
"Does that mean you're all running asahi linux or are you somehow running linux elf binaries on macOS?" So you're running whatever linux districtuion you want or some particular flavor through Docker on Mac? And you don't have ELF binaries because you're using precompiled packages for ARM and don't need weirdo prebuilt ELFs, or you have a workaround?
Those containers contain a mix of custom built software and packages from the distro.
Plus as the Apple virtualisation framework now implements pretty standard virtual hardware - it uses virtio - we are finding things work vastly better in Docker for Mac - particularly as we can use virtiofs etc to pass files through from the host.
I thought Windows containers never went anywhere. Interesting to hear that they are still a thing.
Note on Linux containers:
Running Linux containers on Windows requires the use of LinuxKit or WSL. Docker Desktop for Windows requires you switch modes between Linux container mode and Windows container mode as you can't run both simultaneously. A workaround is to install an additional Docker daemon inside the WSL environment. Most people are going to install Docker Desktop and use WSL in Linux mode. It's fine. Hopefully my facts are up-to-date.
Or have they just embedded the QEMU on-demand translator with binfmt_misc in the ARM virtual machine and the M1/M2 is just powerful enough to make users not notice what's going on?
[1] https://developer.apple.com/documentation/apple-silicon/abou...
James Hamilton was adamant AWS build ARM expertise in-house [0] (one which they accelerated by acquiring Annapurna Labs), and he's been proven right [1] despite the obstacles he himself foresaw [2].
[0] On the Origins of AWS Custom Silicon, https://archive.is/4k5ul
[1] Low cost, low power servers for Internet-scale services, https://mvdirona.com/jrh/talksandpapers/JamesHamilton_CEMS.p...
[2] AWS Designed Processor: Graviton, https://archive.is/WLFBW
I wouldn't trust anybody to be right about the benefits of something this big if they don't also foresee the obstacles you'll face to realise those benefits.
It's a fool's errand to try to reason about Amazon's pricing. We don't have figures about how much a Graviton costs to build or to run, but we do know AWS has staggeringly high margins across the board. They have huge headroom to make price cuts if they want their homegrown solution to win.
And perhaps it is better! I have no doubt it's a good CPU. But we can't infer that from pricing.
We did some assessments of how well our applications ran on ARM and x86 based AWS instances, and quite simply it costs us less to serve the same quantities of end user traffic using Amazon's ARM instances compared to the x86 ones.
Incredible amount of computing capacity per u in the racks at this point on both fronts. Intel is betting on custom accelerators, but I don't know how well that fits on a rental (cloud) model unless you get a full server/cpu.
The real question is, why are other cloud providers not working on similar chips?
I suspect the answer is simple; they are nowhere near as flush with cash to buy up a CPU company and invest years to make their own chips. It's no small endeavor that AWS has gone through to get here.
Here's an alternative equally simple explanation: ARM doesn't actually have anywhere near the level of advantage in perf/$ that Amazon's pricing would imply. The price differential on AWS isn't coming from any kind of intrinsic advantage, but from Amazon subsidizing their ARM-based compute for strategic reasons.
CPUs that need an insane amount of power and have barely any applications in mainstream computing. Not exactly a good fit to provide to customers.
> The price differential on AWS isn't coming from any kind of intrinsic advantage, but from Amazon subsidizing their ARM-based compute for strategic reasons.
By running their own CPUs, AWS doesn't have to rely on Intel's ability and willingness to ship stuff and they don't have to pay Intel's margin either.
Obviously there's no real market for Sparc or POWER in the public cloud, and there probably never was a time window for creating a viable market either.
But if they'd wanted to get into the server ARM market, they would have had the expertise in-house, there was no need to buy out some startup for the team. That their expertise was on CPUs with different microarchitectures, but that shouldn't matter all that much (didn't matter when Apple bought PA Semi to work on ARM instead of PowerPC).
> By running their own CPUs, AWS doesn't have to rely on Intel's ability and willingness to ship stuff and they don't have to pay Intel's margin either.
Right, being in control of your own destiny rather than being tied to a single supplier is a big deal, and that's one example of what I meant by the strategic reasons. Having more leverage over Intel and AMD on pricing is another. Making it harder for some customers to move their workloads away (because they're bought into ARM and other companies don't have compelling ARM offerings) is a potential third.
Not having to pay Intel's margins is a potential advantage, but requires reaching sufficient scale to offset the costs of running your own CPU team.
But for any of this to make sense, they need to successfully move a large proportion of their customers over to ARM. Just having the servers available and nobody using them does nothing. That's exactly the kind of situation where you'd want to subsidize the prices for the ARM servers and build up the customer base. That's the case even if ARM has some intrinsic perf/$ advantage; you'd still want to subsidize it to the point where you're able to saturate your ability to manufacture and install ARM servers.
Apple is designing their own cores while Amazon is just using ARM Neoverse.
And Oracle is offering Ampere instance which IIRC are quite close to Gravirton cost/performance wise.
What? AWS spun out of Amazon.com having extra servers/infa after the holiday season rush and pushed for an internal way to monetize their extra compute/storage power.
Sure, when AWS was created Amazon.com didn’t use it.
However today, Amazon.com is built entirely on AWS. It has been using EC2 and S3 since at least around 2010 (by 2013 “legacy hardware” was almost non-existent). Until 2020, it was all “hidden” behind Amazon’s internal tooling. Around 2019 there started to be a big push to just use AWS directly ie teams get AWS accounts owned by the Amazon.com org. There is still internal tooling to hook the code repo, build systems, and “pipelines” to directly deploy to AWS resources.
Meanwhile, the “serverless” I was talking about in the previous post was an internal service that hosts a Lisp dialect called Datapath. As a developer, you write some Datapath and give them the money and it executes as much as you need it to. This service is the core of Amazon.com and was actively being migrated to Graviton.
Arm on the server isn't a radical idea and been proposed for a long time. Smartphones and servers share a lot of similar requirements like especially performance per watt
Intel wasn't interested in "low-margin" markets and was generally incompetent?
Weird thing is they had StrongArm in the early 2000s.
For that kind of work it's all about power efficiency and Amazon is able to specifically design their gravitons to have just enough cache, CPU speed, management components, etc. to get the job done and nothing more. A bog standard Intel CPU is designed for any and every workload so it's going to be idling and wasting power on components and transistors you never use or care about.
you HAVE TO HAVE those CPUs on devs laptops. Otherwise it just won't work. Macs with M1/M2 are pretty much ideal setup to develop for gravitons
Amazon took a gamble on server ARM chips. Not a bad or blind gamble, but they had no idea if their customers would take to them at all. I'm sure if they hadn't, we'd all be sitting here saying what an obviously dumb waste of money.
I'm a giant Intel fanboy, but I think they have a real challenge here.
Thinking about it, I knew that Intel not landing the first iPhone contract was a big loss on mobile but I don’t think anyone expected it to savage their traditional strongholds like this because that paved the way for so many tools not just being ported but heavily optimized for ARM. They’re certainly still doing a ton of business but now there’s a cap on how much profit they can take even if AMD completely founders.
2014 - AMD Opteron A1100: https://www.google.com/url?q=https://www.anandtech.com/show/...
The startup Calxeda (2008-2013) - https://en.m.wikipedia.org/wiki/Calxeda
Intel also had the 'xscale' ARM chips back in the late 2000s
In 2008 even Java developers tended to be antsy about running on non-x86 hardware. A decade later it’s close to a non-issue for a staggering amount of stuff.
Intel Xscale were a different beast. They got that through StrongARM from DEC as part of some law suit wheeling and dealing. They never seemed to really want to push it, preferring instead to try competing in mobile with a string of failed x86 cores.
Funnily enough Intel also had the best/most competitive ARM cores with XScale in the early 2000s which seemed like the default option for most high-end devices at the time. I wouldn't have been surprised if Apple would've just went with that (even if not x86) if Intel hadn't decided to can it without actually having any viable x86 SoC to replace it...
Don't they have to pay a fee to Arm for this, even though it is being developed by them?
And would Google (which is allegedly developing their own Arm Chip) and Apple (which also has their own chip) not also benefit from backing Risc-V as well?
I have not been able to get clear in my head the relationship between Risc-V and Arm, and even more so exactly what responsibility down stream each user has.
Would appreciate the some ELI5 from knowledgeable HNers.
---
Slightly related: is it more or less a given that eventually no one will use x64 chips (since Arm/Risc-V is more efficient)?
If not for servers, is it at least a given for desktop computers, as more software is written for ARM, and re-compilers like Rosseta improve?
If so, I know that Intel is investing in RISC-V, but has AMD given up the space entirely?
RISC-V has a good ISA, but there currently isn't a lot of silicon experience in the real world (relative to Arm and Intel, for example). And Amazon isn't going to roll their own to that degree. It's not worth the meager licensing costs.
It's like the difference between deploying Red Hat Enterprise Linux and Arch Linux. Each has it's place, but right now its advantages are in much different areas.
You can also get architecture licenses where you can implement your own CPU core (from scratch, or by making tweaks to an ARM design) - Apple has one of these, but Apple is a special case in that they were one of the original partners when ARM was originally founded in 1990.
RISC-V on the other hand is an architecture that is open for anyone to implement, but the RISC-V foundation doesn't sell you a core design - you need to create your own, or find a partner to sell you one. It's also much newer, so there's less of an ecosystem around it compared to ARM.
Because it's way cheaper than to do everything ARM provides on your own?
Gravitron3 is just using ARM Neoverse-V1 (which AFAIK are years ahead of any RISC-V cores). So they aren't different from Qualcomm/Ampere have been doing so far.
I think Ampere is planning to release a CPU with their own custom cores in the near future though. It's interesting whether Amazon will follow suite or not. It would be very bizarre if they tried switching to RISC-V in the near future though (I do find this semi -fanatical obsession with RISC-V a bit fascinating though but I don't think it's shared by many large corporations outside China..)
The first Graviton was launched in 2018, which means it was in development and manufacturing even earlier. You couldn't build a high performance RISC-V chip back then. Even in 2023, RISC-V is significantly behind ARM/x86 in performance and software support isnt great either (for example, Debian has supported Arm for over a decade, but RISC-V support only arrived a few weeks ago).
I could only buy 4 in the end.
Sidenote: has anyone noticed that there's no price difference for GCP to use ARM?
I politely disagree. We know for a fact, from December's RISC-V Summit, that Microsoft is influencing RISC-V to ease the burden on their own Windows for RISC-V effort.
Qualcomm was a partner for ARM, and they are already used to working with them. They will likely be partners again, with RISC-V devices this time around.
And Windows for ARM will go the same way as Windows for Alpha. Just an historical footnote.
RISC-V is inevitable.
Why? Some people here seem to be weirdly obsessed with RISC-V when it's likely to end up being even more closed/proprietary than ARM (at least on the high-end). Why would you give away your competitive advantage to everyone else by licensing your core designs?
ARM at least provides more or less an even playing field to everyone. It's much cheaper for competitors to catch-up with Gravitron since they can just license the exactly same core it's built on. If it was a RISC-V CPU and as much ahead it would be way easier for AWS to maintain it's moat.
Arguably Windows for servers is a joke in itself. What can it do, besides Active Directory and Exchange, that Linux can't?
(I know LDAP exists, but let's not pretend that AD isn't superior in almost every way imaginable)
Not that it's a good thing, but it's been my experience with some enterprise software libraries in practice.
For years GNU/Linux lacked server capabilities at the level of IO Completion ports, grated, it finally catched up.
The security model, that on GNU/Linux would require to enable SELinux, seccomp and several other knobs.
A proper ABI for drivers, with an userspace driver model for most scenarios, increasing server security.
Sharing files that aren't stuck in the NFS model.
Capabilities based security, ability to allow only execution of signed applications and group policies for executions.
Ah, and Azure runs on Windows, not Linux.
https://techcommunity.microsoft.com/t5/windows-os-platform-b...
These companies are not interested in selling to small companies that just want 1 to 100 machines. The only company that might is Ampere.
https://www.asacomputers.com/arm-servers.html
Oracle is using the ARM server chip from Ampere but they would be buying tens of thousands or more and getting volume discount pricing. I've heard rumors that Ampere customized their chip for Oracle's needs.
https://www.oracle.com/cloud/compute/arm/
Qualcomm was designing an ARM server chip about 8 years ago but killed the project. Most of those engineers went to Microsoft and now ARM itself.
I've heard stories about Chinese hyperscale datacenters wanting to design their own ARM servers but I haven't heard anything in a while.
They are low—end, low—load servers, if that matter.
For EC2 only used it for a single instance where I could redeploy the app easier by picking an ARM image
ARM presence in the server market will go away like poof. More than half of it.
From a business continuity perspective, Amazon doesn't benefit from being locked to a single vendor for its CPU core IP.
Like what? Are there any high-end RISC-V cores that proven to be competitive with Neoverse you could license?
It all just seem hypothetical to me at this point and I really struggle trying to understand who and why would design cores to license them to others? Looking at ARM it just doesen't seem like a great business model compared to making them yourself..
Look into Tenstorrent Ascalon, which is supposedly competitive with projected Zen5 performance.
Ascalon has, by the way, already been licensed to LG[0].
RISC-V is inevitable.
0. https://www.eetasia.com/jim-keller-on-ai-risc-v-tenstorrents...
> RISC-V is inevitable.
I'm sorry but you sound a bit like the "Crypto bros" from a few years ago.
Why would anyone share their own high-end cores when margins from producing and selling them yourself are much higher than form licensing (looking at ARM).
> ARM presence in the server market will go away like poof. More than half of it.
This is a borderline fetishistic delusion to think that this could happen immediately in the first place. The changeover from x86/64 to ARM for AWS customers didn't even go this fast, and that's WITH the help & stability provided by the extensive tooling & ecosystem around ARM.
RISC-V is nowhere near to being able to provide the same sort of tooling for years to come, and this comes from someone that loves the architecture's openness.
Or, what they've changed to take advantage of the difference.
The rest - which by mass, of course, is something approaching "all of it" - is mostly toxic fibre glass, epoxy and plastic that we don't really know what to do with. It's bad, even before you start talking about all the electricity and heat that was used to turn it into electronics in the first place.
I’m guessing there are no environmentally friendly alternatives to these materials than can be easily swapped in factories without much hassle.
[1] https://interestingengineering.com/innovation/new-biodegrada...
> The research will also provide Infineon with a fundamental understanding of the design and reliability challenges customers face with the new material in their core applications.
That is corporate-speak for: they have no idea.
And therefore, as a whole, the answer to that is: NO.
PCBs get delivered, sometimes without big components (coolers, screws/brackets fans removed), but also sometimes with those. They are shredded as a whole, including all soldered components, to a particle size of 2mm. Iron and aluminium are separated magnetically (in case of aluminium with an eddy current separator, iron with a static field). All three components, mostly-iron, mostly-aluminium and the rest are then melted down separately. Adherent plastics, electrolytes, epoxy, etc burn off during this process. Gas and ash are lead through separators that filter out the fly-ash as much as possible, which is buried somewhere as toxic waste, the rest is entered into the atmosphere after cleaning (afaik various liquids to bubble through to wash out soluble chemicals, which are then dried and buried). All three metalllic liquids are then metall alloys of various purities which are then cleaned and separated further by skimming off the slag, adding flux, electrolysis and other chemical separation methods. Most of the "other" fraction is copper, which is the main product overall, other rare metals such as gold occur in far smaller amounts and seemingly add only a little to the overall earnings in the process.
Please note that this is the process used in a relatively new plant in a Western European nation. I guess the "advantage" over the rest of the world is that at least there is a fly-ash separator and exhaust cleaning.
Facebook have been flip-flop’ing on their ARM server: https://www.extremetech.com/extreme/146850-facebook-arm-x86-...
Edit: I mean auto complete
One prefers locking and complicated instructions, the other is designed with open and simplicity.
At one point, Intel seemed to be dominant, and proven right with their proprietary tech, where as ARM chugged along keeping a low profile, and keep doing the good work. Now, finally the reality has caught up with them, and turns out openness and simplicity is finally taking a lead.
Stick to your principles FTW!!!
What machine is this?
ARM is a huge corp like anyone else.
ARM got a huge boost now since everything is switching to mobile and their chips are more energy efficient than Intels. Added to that Intel was also asleep at the wheel and missed the mobile boat…
It seems you're confusing ARM and RISC-V here.
ARM isn't simple, nor open.