Xcena and Samsung's Near Memory Compute CXL Device

They've implemented mmap() in hardware. What a time to be alive?!

nb: Related discussion on Samsung's PIM from yesterday: https://news.ycombinator.com/item?id=49487341

o PCIe 6 attached card for PC's (Or, optional CXL 3.2 x8 device/connection)

o Part CXL Memory Expansion Device (up to 2TB DDR5 onboard)

o 3072 onboard RISC-V cores

o Cache Hierarchy onboard (L1, L2, L3, etc.)

o >"MX1’s programming model has parallels to OpenCL or CUDA" (= ability to write custom mathematical Kernels, etc., etc.)

o >"XCENA takes advantage of RISC-V’s extensibility to implement a custom Vector Processing Engine (VPE) at the subsystem level. Each RISC-V core gets a VPE command queue, and can ask the VPE to accelerate a variety of vector operations. Likely, XCENA uses special instructions to enqueue messages into VPE command queues, and expects code to treat it as a giant shared coprocessor. The VPE supports FP32 and FP16, and provides ~3 TFLOPS of dot product throughput across the chip."

(Repeated Vector Operations is what's necessary to implement Matrix Multiply (matmul), which is what's needed for AI training and inferencing to work.)

>"For perspective, an Nvidia GeForce GTX 1080 with similar onboard memory bandwidth has 8.8 TFLOPS of FP32 compute, compared to the 3 TFLOPS available from the MX1"

(Today's TFLOPS on an MX1 may be slow, but the orchestration, the integration of all of these components on a single device is what's important, because it creates a broader template pattern for future such devices originating not just from Samsung and/or XCENA -- but from the computer industry as a whole. Much like the original IBM PC architecture did for the PC industry, that was to follow...)

o >"MX1 can host SSDs, which are presented to the host as CXL memory. [...] However, SSDs have high latency compared to DRAM. MX1 can mitigate that latency by using its attached DDR5 to cache SSD contents. Caching works with 64 KB pages, with an on-chip 1024 entry map cache. The map cache acts like a TLB, and tracks DRAM pages mapped to SSD-backed addresses. If an access misses in the map cache, it causes a page fault that’s handled by firmware running on the MX1’s RISC-V cores. Firmware handles the cache miss by fetching data from the SSD and updating the mapping."

Observation: This is basically a hardware implementation of virtual memory / paging / swapping, that most OS's currently do, but if this trend continues, then OS's may not need to do this in the future... it'll be interesting to see where this goes...

Oh, last thing... would love to see RISC-V cores replaced, in whole or in part, with an FPGA or multiple FPGA's... Don't get me wrong, I love RISC-V over all other non-open, closed hardware ISA's, but I love FPGA's (since the user can define their own functionality) even more... ideally FPGA's with open toolchains...

Or maybe Samsung/XCENA could make several models of this... one with the RISC-V cores, another with an FPGA or FPGA's...

So to recap, from a "future technology" perspective, I like this architecture a lot!

(Also, technology-wise, this reminds me of Transputer from the 1980's, a device which despite its lack of commercial success was so ahead of its time that it's the stuff of legends among the tech-savvy: https://en.wikipedia.org/wiki/Transputer)