Do we need all those cores in a single machine? Do we know how to use them? Beside some scientific workload, how are they used?
I guess the standard answer is virtualized them and sell them in chunk as "cloud" but then why prices are still so high?
I am not complaining, I just feel that something is off, and I don't understand what it is.
The prices felt off mostly because CPU are not the only cost, and in many cases it isn't even the most expensive cost on Server. Memory is, especially ECC Memory. Memory price per GB hasn't fall at all. And in many cases they are even more expensive. And it is only in recent few years you start seeing "High CPU Instances" because it is cheaper to include a Higher Core Count in the plan than to increase the Memory.
And because of the increase in VM / Core Count and Tenant, the server are now equipped with multiple High Speed Network connection. And if you look at it from a Cost Per Network Card / Port / Bandwidth / Server / Tenant. Price hasn't fallen much either. Adding in your cloud / VPS instance are paying Electricity, Co-Location and Networking charges, none of these have an Moore's Law and are part of the TCO. And if you include the continuous raise of wages, things doesn't look all so good.
But Zen 2 is barely deployed so its effect on the industry is not here yet. Linode already use Zen 2 and they are already offering better price plan than DO or other Medium Cloud vendors. I expect Zen 3 to be big with many of the Cloud Vendors. And you should see those reflected on pricing.
This processor has 64 PCIe lanes, and a drive uses 4. If you use half the lanes for disks, you’d need to multiply my numbers by 8. So, in addition to running application logic and talking to the network, each core needs to parse 312MB or dispatch 62K IOs per second.
Those numbers are certainly achievable, but hitting them requires some care. So, this processor is fairly well-balanced for moderately compute intensive workloads, such as query processing, or maybe frontend logic (which would probably want a machine with more network and less disk).
Note that the 128 cores are just to keep up with storage that easily fits in 1U or a fairly compact desktop tower case.
If you measure speed as the IO to compute density, these CPUs are actually slow and bulky by historical standards.
Also, there are sites like Source Hut or Travis that run thousands upon thousands of CI jobs every minute. I worked at a place where we were resizing thousands of photos and hundreds of incoming videos ever minute. One of these could easily replace 10~20 nodes in a transcoding farm for companies that do these types of workloads on physical servers.
There are tons of commercial applications that would keep these things busy during business hours, and some that would also keep them busy 24/7.
If this wasn't needed or wasn't useful then we also wouldn't ever see companies buying more than a single rack (still want redundancy and hot spares ofc), but they do.
So yes we know how to use all these cores. You take what's currently 4 machines and consolidate it to a single machine. Or similarly 40 machines to 10 machines, etc...
AMD knows they have some serious advantages there, especially on the efficiency front, with Zen3, and they're going pedal to the metal.