Hacker News Reader

Comment by ineedasername | Hacker News Reader

AMD seems to be making a push for the server market at the moment too. It looks like a good time for it as Intel struggles to get production on 10nm while AMD is already at 7nm for some of their range. But Intel has an incumbency advantage. I'm no expert on the server market though. Could someone with a bit more knowledge chime in with a how big a threat this really is?

View on HN

Mandatory reminder that TSMC "7nm" is not comparable to any Intel node and also doesn't actually represent a uniform 7nm gate width.

The technology and potential in Intels "10nm" and TSMCs "7nm" are similar. What matters is that Intel hasn't managed to successfully deploy a new process node in 4 years and thus competitors like TSMC, Samsung, and GF have had time to catch up and surpass Intels fab tech in various ways.

> I'm no expert on the server market though. Could someone with a bit more knowledge chime in with a how big a threat this really is?

Sys Admin here. People are starting to notice and raise eyebrows. It is certainly an option, especially for new stacks. The power/price/performance ratios on AMD are truly amazing.

It certainly helps that Amazon, Microsoft, Cray, and Baidu have all already started to use Epyc servers.

Student admin myself here at Uni. With Dell being our primary provider I did some general comparisons with the currently available EPYCs. Our Dual socket Skylakes (36 total cores without hyperthreading) with 128GB of ECC ram cost ~18K each. With an EPYC chip I could build 48+ core systems with 256GB RAM for ~3K less.

If these next gens have similar pricing, they look very competitive for the future of on-premise (and cloud) HPC stacks.

I was in your same place exactly 20 years ago when I convinced the dept head that we (mostly me) could build our own machines for the test lab instead of purchasing units from corp it. I should have productionized the building but otherwise it went pretty well.

1, you are right, you can build your own machines for way less than dell

2, you are now the support team. So when you get a call months, years later about hardware quirks, missing pieces or odd incompatibilities, expect it.

3, Buy extras and keep the machines largely the same. The fleet will be its own spare parts as it ages out. You will need extras for differential testing.

4, Keep a log and copy of all the bioses supplied by the motherboard manufacturer. Baseline and benchmark each server before it is put into production, retest as it is cycled out for other duties.

5, If you don't max out the memory when built, do track ram prices and there is a sweet spot of a dip just as the ram tech ages out. If you miss it, ram prices go back up again. An older machine with lots of ram will useful for a very long time.

If one plans it right, 100 node HPC cluster decays into a 75 node render farm into a 25 node distributed system test cluster, I think they could get 10 years of service out of the purchase.

I don’t disagree with any of that. I usually prefer building my own when I’m the maintainer/builder. But not my current situation, a few things to note:

* I have (hopefully) one semester left. After that, I’m gone.

* A good portion of our experienced team will be gone within the next two years, including the few grad students who helped build our cluster. Introducing “non-supported” hardware would likely introduce problems for the next group.

* The prices above were from Dell, not self assembly shopping. University policies mandate any university funded tech/infrastructure equipment (especially desktops/servers) be done through specific partners. For us, that’s Dell, GovConnection, and CDW. I host my own 6 node Supermicro Microblade setup (render farm) from CDW that took almost a full year to purchase because nobody in the Purchasing Dept. for clubs was aware of that rule. Until my hopping from Boxx to Thinkmate to “haha, funny story...” uncovered it.

> prices above were from Dell, not self assembly shopping

That is amazing. With those kind of prices folks might upgrade before their old hardware has properly aged out. At what point does AMD institute a buyback program for Intel hardware? (joking)

I don't see how this "incumbency advantage" would help Intel, exactly. The server market is highly competitive. If AMD puts out superior product at an attractive price, nobody is going to say "but we've used Intel for a decade, how can we suddenly switch to AMD?!".

Things aren't always that simple. For example AMD may be cheaper per core, but if you're running AVX heavy code then intel has a throughput advantage. At least it did with Zen, Zen2 has some improvements in that area, so we'll have to measure again.

Moving to the next intel generation is usually simple, incremental change. Moving between vendors means a lot of re-evaluating if you're crunching numbers.

If your workload needs a lot of PCIe lanes for IO or you need cheap systems with ECC ram then the story changes quite a bit.

Thats exactly what they would say. Reliability track record, existing software/project restraints and enterprise relationships matter more than the price tag. That's why Intel can still charge rediculous amounts for enterprise chips for some time.

I remember a time when Opteron had a very large part of the server market. It's about lack of competition, not about how businesses love Intel charging a fortune for their product.

Who's making chips to compete with Intel? AMD, IBM, and nobody else (ARM isn't high-performance yet). IBM's lock-in with POWER is scary to businesses. AMD wasn't competitive for most server needs. That left Intel to charge whatever they wanted (and they did).

With EPYC, there was finally a competitive offering at a great price (not to mention a better track record with meltdown and the constant stream of spectre variants). All the same software runs on the AMD chips too.

Market perception matters though, and I know many people -- IT admins included -- who think that Intel is just higher quality. Those people will be hard to capture quickly, and AMD will need to hold a lead for a while to change that perception

It doesn't help that Ryzen had few relatively high-profile issues:

https://news.ycombinator.com/item?id=13924192 https://bugzilla.kernel.org/show_bug.cgi?id=196683 https://www.phoronix.com/scan.php?page=news_item&px=Ryzen-Se... https://utcc.utoronto.ca/~cks/space/blog/linux/RyzenMachineL...

Not implying that Intel chips are flawless (still bitter about Haswell TSM), but as you said, market perception matters.

> nobody is going to say "but we've used Intel for a decade, how can we suddenly switch to AMD?!".

Actually, that is the case, it will take about 1-2 hardware generations of customers _asking_ for AMD before it's featured prominently by any vendor (except the extreme ones like SuperMicro)

If nobody is actually calling their HP/Dell reps asking for it then it will never happen.

Fortunately that already happened; HPE and Dell have Epyc servers that are comparable quality to their Xeon servers.

A project that I currently work on procured new hardware and one of the constraints for the selection was that pretty much exactly the same hardware must be available in three years. So this one will be buying intel for the foreseeable future.

I don’t consider that an unusual constraint.

>nobody is going to say "but we've used Intel for a decade, how can we suddenly switch to AMD?!".

Maybe not those exact words, but people will say "We are adding a server to our VMWare ESXi cluster, so if want it to be compatible with our existing hardware we better stick with Intel."

VMWare's high availability feature (a highly desired feature for any data center) won't work across different CPU architectures, so unless you are replacing your entire stack and not just adding server(s) then you have to stick with the architecture you already have in place.

These aren't "different CPU architectures". They would all be x86-64 so unless VMware uses some specific Intel feature it shouldn't be a problem.

I have been a Sys Admin for 7 years. It is a problem, vmotion won't work going from AMD to Intel or vice versa (at least not in any official way). In fact it is highly suggested when building ESXi stacks that you get the exact same model CPU for each server.

https://communities.vmware.com/thread/305792 https://www.v-front.de/2013/04/how-to-vmotion-from-intel-to-...

No, Intel FlexMigration and AMD-V Extended Migration architectures are different, and vMotion isn't supported across them.

EVC does not allow for migration with vMotion between Intel and AMD processors.[1]

This isn't really that surprising. x86-64 is standard, but extensions aren't. Eg, here's some of the new extrnsion in Zen vs Steamroller: Compared to the AMD Opteron™ "Steamroller" EVC mode, this EVC mode exposes additional CPU features including RDRAND, SMEP, AVX2, BMI2, MOVBE, ADX, RDSEED, SMAP, CLFLUSHOPT, XSAVES, XSAVEC, SHA, and CLZERO [1]

[1] https://kb.vmware.com/s/article/1003212

Indeed, AMD's best in would be at the larger scale end, where someone might add whole sites at a time and choose AMD for a new datacenter, since inter-site live vMotion is pretty unusual.

Reply on news.ycombinator.com