Server Hardware Super-Cycle 2022
unum.cloud
unum.cloud
My previous comment
Not just 2022, after being stuck with an 14nm node and old uArch from Intel. The Super Cycle will go on till 2025+ once you factor in all the upgrade from previous hardware and new capacity. PCI-E 6.0 was 0.9 in Oct and should have been 1.0 by now. I guess a small delay. We are looking at PCI-E 7.0 by 2023/2024. Intel is working their ass off to get back on track to their somewhat original roadmap. Lots of competition from AMD and more importantly ARM. We finally have node improvement after stuck with Intel 14nm on servers for 4+ years. TSMC 3nm in 2023 and 2nm by 2025. ( Expect a year later on Server ). DDR5 and DDR6 as well. 1 PB of SSD in 1U with Ruler form factor. And 800Gbps Network possible.
The biggest problem with all of these is that they are not getting cheaper per unit. DRAM / GB hasn't dropped at all in the past decade. The price floor is pretty much the same at $2/GB and much higher for high capacity and ECC memory. ( Price excluding inflation so you could argue it is cheaper ) NAND wont get much cheaper even with higher stacking. So all in all, you may be putting double the Storage, DRAM or Networking in a Server put they will cost double as well. Only thing that is getting slightly cheaper is CPU Cost per Core. Edit: And that is why this decade may be all about Software optimisation. There are many inefficiency lying around in our software stacks. While we have peaks everywhere in our Hardware roadmap. I would be surprised if we could even get 1TB Memory and 128 Core Server at half of today's price by 2030.
The one thing that's relatively new-ish that I'd like to play with is persistent memory. But I'm curious about it for embedded applications where power is not assured. For servers? I don't understand the use case, because datacenters are where we have the most consistent power. Perhaps it'd be more interesting if I ran workloads on large datasets...
DDR4 CAS latency is 15 clocks, which is about 9ns for highest clock frequency. Add to that cache latencies and memory controller latencies and you easily can get about 25-50 ns of latency and more. And to overcome that you have to use stream-friendly algorithms just as with SSD and spinning disks.
The technology of persistent memory is no different than NAND or whateger SSD tech de jour is used. This means that wear will be same as with other SSD drives and may be even worse due to different abmient temperature conditions.
Persistent memory is also more expensive.
The conflicting transactions must be ordered as if they were processed serially, the actual processing order can be different and their committment to disk also can be different than "commit transaction1 and then commit transaction 2". Having these two transactions commit at once or none at all is also permitted by serializable isolation order. Which is heavily used in any semidecent transactional storage engine.
If you go all-in to persistent memory, you lose cheaper multi-tier storage opportunity. You can use heavily read-optimized storage structures on the spinning disks, for example, by rearranging writes to these disks using SSDs and just plain old RAM. If I remember correctly, currently spinning disks are 5 times cheaper than SSD and you can either increase bandwidth or storage volume by using them.
On my 3rd gen Xeon E5 NUMA machine (two nodes) accessing adjacent memory bank is roughly around 90ns under no load. Accessing non-adjacent memory was around 120ns.
> 300ns is a latency of PCIe, not DRAM.
I'd figure that this number improved across different PCIe generations or? Have you got any tool to recommend that can measure this kind of thing? I found https://github.com/andre-richter/pcie-lat but haven't tried it yet.
Transactions must be durable, but that doesn't mean they can't be committed to RAM initially. There are large-scale database systems you have probably heard of that commit to RAM and get their durability from committing to multiple replicas in distinct failure domains.
Databases seem like the obvious answer. Persistent memory / Optane is much faster than NAND Flash... but less dense than NAND Flash. Its also slower than DRAM, but more dense, so it sits in between.
Whether or not that's useful remains to be seen.
The problem with 1 is that you still need to periodically flush to disk (at least once persistent storage is full) which means it’s only an improvement in burst activity and not peak throughput.
The problem with both is complexity of implementation to deliver functionality that can only be used on a (currently) relatively uncommon subset of servers. Probably not something that Postgres would rush to implement, but potentially something that MSSQL or Oracle might consider as a premium feature.
Disclaimer: I haven’t looked into the details of optane and how you interface with that storage in software.
The hard part is developing software to use it. I don't think you can really port existing OSes and software to use a single-level store environment. It's a different language than the one we speak. Seems this has been a general problem for persistent memory, to really use it 100% you need new software designed for it.
How is this possible? Hot plug connectors have uneven edge connectors so ground pins contact before signal pins or, at the very least, initialize links with in a high-Z state or have protection diodes. Did PCIe add any of these features?
* https://www.fujitsu.com/downloads/SFTWR/manual/s_e/b23pav1h0...
Being able to hot-plug cards in/out of slots has been a thing for higher-end gear for a long time.
I'm sure the IBM Mainframe folks did it many decades ago like they've seem to have done for many things that the 'commodity' server folks are only now catching up to.
They could do that with CPU modules a couple generations back. I don't think the newer ones can do it though, at least not a CPU at a time. It's probably possible to power off a CPU drawer (4 CPUs on the Z15, IIRC) without too much hassle.
I remember seeing a colleague of mine doing it with memory and CPUs on a live Ultra Enterprise 4000, again, with memory and CPU. IIRC, he had to tell the OS the board was going to be pulled out before doing so and a light would signal it was safe to proceed. He was doing that because we used one machine to process a lot of logs overnight (in 1997).
A card cage design was used on some IBM machines, where you attach the card a slim sled with a lever attached, this allowed you to "slide in" the card into running computer then use the lever to push it into edge connector (this also handled powering up/down the connector, iirc).
Some servers have cases which instead implement that completely in the case itself, though usually that means you need more effort to do the replacement.
Then there are alternative PCI-E connectors - ExpressCard, U.2 & SATAExpress (used in SSDs), OcuLink and related (cabled PCI-E)
That said, hot plugging is definitely a "pets" thing and as for large scale data center computing, nobody cares. The machine fails as a unit and it's OK to take a machine offline to fix it.
Or I'm just paranoid. A ZIF PCI-E socket would be swell for this.
Real server hotswap hardware is not putting any force on the motherboard; it goes into a caged connector.
2U and especially 1U servers are much, much louder than normal computers. Shrinking parts means that the fans need to spin much faster to get sufficient airflow.
"Aside from DRAM expansion and smarter NICs, CXL also hopes to revolutionize how we view and share the memory pools already installed in your chips. Today, every device has its own address space. Transparently unified addressing is extremely slow for randomized memory patterns and causes unpredictable latencies. "
and never really finished it. does CXL include address mapping?