Azure announces new AI optimized VM series featuring AMD's flagship MI300X GPU | Hacker News Reader

Azure announces new AI optimized VM series featuring AMD's flagship MI300X GPU

techcommunity.microsoft.com

95 pointslatchkey67 comments

I'm surprised by the negativity.

Microsoft has the know-how from hardware and software, drivers, APIs, firmware the resources, it's a major AMD customer in Cloud and consumer devices.

Do people think Microsoft and AMD will watch Nvidia cannibalize the market snd Microsoft writing cheques for whatever Nvidia demands?

It's like people forget a major economic rule: when margins are high you will attract competition.

Similar to the old Slashdot days, HN is still plagued with just outright anti-Microsoft people under any circumstance, even in cases where there is no justification.

A more recent thing is just pretending Microsoft (and especially Azure) doesn’t even exist.

If pressed, sure, people will admit that these may not be entirely imaginary entities, but if listing technologies or platforms then “oops” they’ll just forget.

The best example I saw was a “poster” of cloud big data technologies. Started with Amazon S3, went through Google Bigtable, and then in the corners had companies so small that their own marketing page is the only search result. No mention of Azure anywhere.

There were dozens of logos on that page representing companies with annual revenues smaller than what one my customers spent on a single Azure Storage Account by accident.

Meanwhile they're all committing code to github and pulling packages from npmjs.

If often wondered if microsoft should rebrand the entire company to github

They’re tiptoeing that way with the “copilot” branding.

https://arstechnica.com/information-technology/2023/11/bing-...

If developers are going out of their way to forget about Microsoft that seems like Microsoft's failing.

Not like Microsoft has ever made actively hostile design choices over and over again with it's namesake product or anything...

>If developers are going out of their way to forget about Microsoft that seems like Microsoft's failing.

Meh, the world of developers as a profession, is much bigger than some angry activists on Twitter and HN smelling their own farts in their echo-chambers and venting on their blog.

HN also likes to forget that enterprise also exists. And that IBM or SAP exist. Or that most of the tech that makes the world go round is very outdated and not the latest hip cool stacks from the start-up world.

Sometimes it's just another echo chamber where people prioritize emotions and feelings over facts.

The term I hear used Jupiter Broadcasting's Coder Radio is "dark matter developers" - imagine 9-5 Java 8 guys.

However, I don't think HN forgets they exist, I think a lot of HN users are those enterprise people who want to use better tools.

>imagine 9-5 Java 8 guys

That's like >90% of SW developers in the world.

After multiple cross-tenant vulnerabilities I'm certainly in the "never Azure" camp.

It is impressive to me the amount of time others are willing to spend in order to avoid even considering what is arguably the cleanest, most business-friendly stack in 2023.

You can do everything with one vendor, and some customers really love that. Ours certainly do. They don't even bother trying to audit us along those lines anymore. We make it as simple as possible at every level. Offering a successful B2B SaaS experience is more about the biz interactions and less about the nerd crap. If you found a way to make your internal tech more important than the biz, you likely have a problem.

Every time we talk about databases, I do a CTRL+F and check for 2 things: Is SQLite covered and are we being assholes about SQL Server for no good reason. SQLite gets a good rap on HN, so I more often prioritize awareness around the red-headed stepchild of RDBMS. Their hyperscale offering is end-game for 99.99% of business use cases.

I always end up at "How much money, time and frustration are your principles worth?"

> It is impressive to me the amount of time others are willing to spend in order to avoid even considering what is arguably the cleanest, most business-friendly stack in 2023.

This is totally correct if you are doing small scale stuff in the cloud or just random "enterprise" for sub-thousand person companies. Azure is playing a distant third in reliability, maybe even fourth behind Alibaba.

> You can do everything with one vendor, and some customers really love that. Ours certainly do.

HN probably leans towards people who build things with the cloud, whereas it sounds like your customers are people who want to pay people to run things in the cloud for them.

I've been burned by Microsoft and AMD.

The talking heads on CNBC would probably mention their synergy of "software, drivers, APIs, firmware the resources" but as a user of their end results - once bitten, twice shy.

Never mind cannibalising (?) the market.

There have already been several ChatGPT outages caused by a lack of compute capacity. Azure literally cannot buy Nvidia GPUs fast enough to satisfy customer demand. They are forced to buy alternatives here. There's no real decision to make here.

The outages you're talking about were at OpenAI, and doesn't seem to have impacted Microsoft's service at all.

But isn't the entirety of OpenAI compute on Azure?

https://www.cnbc.com/2023/06/01/microsoft-inks-deal-with-cor...

https://www.reuters.com/markets/deals/microsoft-inks-computi...

thanks for sharing, I didn't know

Azure is now leasing capacity from Oracle.

That's for databases.

https://news.microsoft.com/2023/09/14/microsoft-and-oracle-e...

GPUs https://www.techradar.com/pro/microsoft-is-renting-gpu-power...

Specifically for the parts of Bing that are rebranded OpenAI https://www.oracle.com/news/announcement/oracle-cloud-infras...

Interesting, thanks!

Microsoft is now literally buying Oracle compute to satisfy Bing AI demand.

https://www.oracle.com/news/announcement/oracle-cloud-infras...

Well, they are forcing bing ai on all users and all queries, I don't think I've ever looked at the output, and yet they keep shelling important money every query I make.

I could've sworn that it was once a given that people doing serious AI work needed CUDA-capable GPUs. Is that no longer the case?

Necessity is the mother of invention.

The article highlights that onnx, deepspeed, pytorch and tensorflow all support these AMD processors

Thanks, so I think you're saying that there are two related things happening that are eliminating NVIDIA's CUDA moat: (1) Developers are mostly using libraries instead of writing "to the metal" (CUDA), and (2) many popular libraries have added support for AMD GPUs. Does that capture it?

https://www.databricks.com/blog/training-llms-scale-amd-mi25...

https://embeddedllm.com/blog/vllm_rocm/

It's the opposite in my experience.

Pure PyTorch mostly works OK, but some libraries implementing crazy optimized, hand written kernels and such will have some trouble.

So (for instance) maybe you can run an LLM in a particular PyTorch framework, but flash attention 2 doesn't support your AMD card, so performance and memory use takes a hit.

Or maybe the library works on an Intel XPU with like 5 changed lines in the entire library (rename "cuda" to "xpu"), but no one bothered to add it, or maybe the dev doesn't even want to support the PR.

Not that I maintain any hugely popular deep learning code, but my own "hobby"/dev hesitance at adding multiple devices is that I don't own them to test them.

If I have an NVIDIA GPU, the only PyTorch backend I can promise even "somewhat works on my machine" is CUDA.

AMD also have HIP which is almost exactly CUDA, but target agnostic, so basically now CUDA code will now run on AMD hardware. So they have the low level via HIP, and have also integrated into the libraries like pytorch. It's a great bit chip out of that moat.

There aren't enough CUDA-capable GPUs to buy. So people will enter learn how to use products like the MI300X or they won't have AI.

It's also a matter of price. Nvidia sells their top GPUs at insane premiums. For Microsoft which is such a huge player and the major AI vendor and owns a lot of OpenAI they have no point into cornering themselves into Nvidia dependence.

They benefit from competition, not from bending to one vendor.

> Nvidia sells their top GPUs at insane premiums.

It is all sold out, for years. You can't sell something you don't have.

https://www.theregister.com/2023/09/08/tsmc_ai_chip_crunch/

Now that AI made Nvidia a trillion dollar company AMD has finally woken up and realized what they should have ten years ago: that they need to invest in the software side of things more. There is movement now, and you can actually do some AI things on AMD hardware, but it will take a long time for them to catch up to Nvidia.

AMD beat Intel in the server CPU market.

AMD and Apple both have TSMC to thank for their ability to make chips that beat Intel's. Unfortunately for AMD, Nvidia uses TSMC too, so they'll have to invest in software to compete.

They are investing in the software.

https://fortune.com/europe/2023/10/04/amd-lisa-su-nvidia-roc...

That's a bold assertion with no evidence to back it up

I've been running large scale data centers for years now with tens of thousands of machines. I'd personally never put an Intel cpu into something that I cared about. We used $30 cheapo intel cpus for our mining rigs cause they literally didn't need anything more than that. But, anything else we buy for servers, is always going to be AMD. One caveat to that would be if we have some customer that demanded Intel and we'd lose the business otherwise.

So yea, in my eyes, AMD beat Intel. I don't think I need any more evidence than that. ¯\_(ツ)_/¯

There's a big difference between AMD beating Intel in your eyes vs the market.

¯\_(ツ)_/¯

challenge accepted ;)

(hello!)

You seem to have forgotten where AMD was 10 years ago. Company was circling the drain but they should have invested heavily in the software stack for hardware they could barely afford to design? Brilliant strategy. Now's the time to invest because they finally have the resources. The ai train is not going anywhere anytime soon

It would have paid for itself a hundred times over in short order. And if they had started ten years ago they wouldn't have been far behind and the required investment would have initially been much less.

Ten years ago they were valued around $1 a share and teetering on bankruptcy. If they didn't ship Ryzen there would be no AMD today at least not what it was or is.

There are a lot of moving parts around all of this. AMD was still dealing with their fab breakup and fallout.

They basically cut all the fat and some muscle to keep pushing forward so I agree with the other poster that now is the time to focus on software and growth.

If AMD 10 years ago tried to compete with NVIDIA and Intel both simultaneously, would it have survived? They did well to stay in business, and now they can address the AI thing.

That's the joke. It still is. ROCm is nowhere near production-ready and if MS thinks devs will want to waste their time on random errors then good luck with that business. AMD cards are also super expensive so not sure what their competitiveness is supposed to be?

Expensive is not entirely relevant. They are available (for now).

https://twitter.com/sama/status/1724626002595471740

ROCm has also made a lot of advances in recent times.

https://www.databricks.com/blog/training-llms-scale-amd-mi25...

Thread about that tweet (or should I say xeet) from SamA: https://news.ycombinator.com/item?id=38274427

Microsoft can develop their own software. We are talking a company that places billions in orders to Nvidia, they can afford to develop the tools and obviously AMD will throw millions and millions in support to court such a customer.

The MI300 is a massive GPU compared to even the new H200.

It was designed as a combined CPU/GPU for supercomputers, with shared memory. But then the AI craze hit, so AMD spun a variant into a pure GPU AI accelerator real quick, which they could actually pull off because the GPU silicon is modular.

...So thats why it cost a fortune. Its really a jury rigged HPC product.

The H100 is, to some degree, also a jury rigged HPC product. The H200 is starting to reach a more "AI" balance between compute and memory bandwidth.

I can see MI300 being used for CFD, nuclear physics simulation, cloud GPU virtualization etc. but for AI it's simply not there regardless of AMD's and its affiliated startups' marketing.

You're thinking the MI250. MI300x is a beast for AI.

Vega was an ML monster on paper as well, handily beating 1080Ti. Then the reality set in...

These aren’t consumer cards. Apples and oranges.

AMD is "CUDA compatible" in theory.

Intel is taking a slightly different approach, and is going for "PyTorch compatible."

You will hear endless negative anecdotes about ROCm/OpenVINO, but they both do seem to be getting better with each update.

ROCm support for a lot of AI processes fixed that

Confusing as they also announced their own Cobalt AI chips too

No, that's about Microsoft's chips. This is about AMDs GPUs.

ROCm was a bull signal

Imagine the hallucinations we're gonna see with AMD drivers in the mix...

Reply on news.ycombinator.com