AMD EPYC 7713 'Milan' Zen 3 CPU Benchmarked

They “just” miniaturised the rack into a chip :)

We've had pretty high parallelism in server CPUs for a while, eg Ultrasparc T1 had 32 threads in 2005, later generations with 128 threads (T5). But a high number of threads and cores is a worse solution than being able to keep up high single threaded execution speedups, as most programs are not multithreaded because parallel programming is so hard.

I'm curious if they will keep up the increase in core counts. It might get more difficult to do that unless the new TSMC processes work very well. They probably would have to go to 16 core (2x 8 core CCX) chiplets as I don't think they can easily fit more chiplets into Epyc.

My surreal moment was reading the Genetic Programming series by John Koza. He discussed a 1000 socket Pentium II cluster which was used for a genetic search.

I did some back of the napkin math and realized that was roughly equivalent to my Zen 2 box.

So of course my failure to replicate the author's success was not the fault of my hardware.

I replaced 4 servers (doing computation work not serving things so no need for availability) at work with a single desktop computer with a ryzen 7... Between the sata vs nvme the RAM speed and the CPU speed, tasks finish faster on rhe Zen2... Oh and it is 1/8 of the power as well. My only problem is that I can't get WoL to reliably work, only the mode that wakes up with any packet works.

I recently had one of these realizations when looking at https://research.google.com/archive/mapreduce-osdi04-slides/...

I think the last digit is the generation number. No idea if the middle two digits have a rhyme or reason, other than bigger is more expensive.

I agree but I think that's on purpose, as part of their "finally get some share in server space", to avoid scaring any non-tech decision maker into thinking this is yet another new product or whatever that need to be thought about before signing on it.

In their other markets they have very clear generation naming, even going out of their way to give naming-room to their laptop chips.

AMD really want to be in server space because that's the safe spot (in revenue) for the next time Intel comes backs, and that's the one they missed last time they were ahead.

What's the workload if you don't mind me asking? Presumably the difference isn't just down to the smaller width of AVX2 but new instructions in the AVX-512 family?

Recompiling is not an option? Also seems weird that the binary only ships with a AVX512 codepath and not AVX2 as well.

I remember there being also an issue of linked MKL libraries shipped with various software (e.g. Matlab) not using the appropriate AVX2 codepaths on 'unsupported' AMD processors and isn't was falling back on slow non vectorized code.

If you can recompile but the code was written to only target AVX-512 you can use https://github.com/simd-everywhere/simde to near effortlessly map the intrinsics to AVX2 (or lower).

I think the key issue here is the use of the intel compiler on a non intel product. Can you not recompile your code with GCC or LLVM? are the performance metrics that bad that they are not options? AMD has made the system... Engineer your solution if you want to use it :)

compile it for avx2 and you might still win

Prices are so high because cloud vendors have no incentive to lower them. It’s even worse with bandwidth. Cloud bandwidth prices are rape.

I think you're exactly right. These cores are designed to be sold en masse to the cloud providers, who are looking for density in the rack, lowest total cost of operation (up front cost is nearly irrelevant), and above all power and thermal efficiency.

AMD knows they have some serious advantages there, especially on the efficiency front, with Zen3, and they're going pedal to the metal.

>and I don't understand what it is.

The prices felt off mostly because CPU are not the only cost, and in many cases it isn't even the most expensive cost on Server. Memory is, especially ECC Memory. Memory price per GB hasn't fall at all. And in many cases they are even more expensive. And it is only in recent few years you start seeing "High CPU Instances" because it is cheaper to include a Higher Core Count in the plan than to increase the Memory.

And because of the increase in VM / Core Count and Tenant, the server are now equipped with multiple High Speed Network connection. And if you look at it from a Cost Per Network Card / Port / Bandwidth / Server / Tenant. Price hasn't fallen much either. Adding in your cloud / VPS instance are paying Electricity, Co-Location and Networking charges, none of these have an Moore's Law and are part of the TCO. And if you include the continuous raise of wages, things doesn't look all so good.

But Zen 2 is barely deployed so its effect on the industry is not here yet. Linode already use Zen 2 and they are already offering better price plan than DO or other Medium Cloud vendors. I expect Zen 3 to be big with many of the Cloud Vendors. And you should see those reflected on pricing.

It's easier to scale to multiple cores than to multiple machines, and cloud services have been doing multiple machines for decades.

If this wasn't needed or wasn't useful then we also wouldn't ever see companies buying more than a single rack (still want redundancy and hot spares ofc), but they do.

So yes we know how to use all these cores. You take what's currently 4 machines and consolidate it to a single machine. Or similarly 40 machines to 10 machines, etc...

No we don’t know how to use them. The abilities of single systems have far outstripped our ability to manage all those resources. It takes a lot of work to drive a system like this to the rails, and very few people even bother trying. The closest most people come to exploiting a machine this large is by carving it up into normal-sized virtual machines with independent operating systems. This makes the utilization look acceptable but in reality much of the resources are just being squandered.

Disks have gotten ridiculously fast. One NVMe card can move 5GB of data or 1,000,000 IOPS over 4 PCIe lanes. With 128 cores, you need to parse 39MB/sec per core, or dispatch 7.8 thousand I/Os per second for each drive in the system.

This processor has 64 PCIe lanes, and a drive uses 4. If you use half the lanes for disks, you’d need to multiply my numbers by 8. So, in addition to running application logic and talking to the network, each core needs to parse 312MB or dispatch 62K IOs per second.

Those numbers are certainly achievable, but hitting them requires some care. So, this processor is fairly well-balanced for moderately compute intensive workloads, such as query processing, or maybe frontend logic (which would probably want a machine with more network and less disk).

Note that the 128 cores are just to keep up with storage that easily fits in 1U or a fairly compact desktop tower case.

If you measure speed as the IO to compute density, these CPUs are actually slow and bulky by historical standards.

VM data centers. One of these machines can be loaded up with ram and host tons and tons of instances without over provisioning. Some of those VMs will be mostly idle, but others will be at full workloads, depending on the tenants.

Also, there are sites like Source Hut or Travis that run thousands upon thousands of CI jobs every minute. I worked at a place where we were resizing thousands of photos and hundreds of incoming videos ever minute. One of these could easily replace 10~20 nodes in a transcoding farm for companies that do these types of workloads on physical servers.

There are tons of commercial applications that would keep these things busy during business hours, and some that would also keep them busy 24/7.

Some workloads can utilize almost all the power. Some workloads can not. Server workloads usually can scale almost infinitely. And we don't really have a choice, single-core workloads usually don't scale (most of the recent advancements are from implicit parallelization using speculative execution).

databases and webservers can use more cores no problemo

2x EPYC 7713 are faster than 4x Xeon Platiniums

The newer benchmark would be "how many hours does it compile Chromium?"