But off the shelf mini PCs are much more user friendly for existing software IME.
Thankfully ARM being so wide spread and continuing to grow this wont matter as much.
I'd love you to point me in the direction of an off-the-shelf mini PC that has 64gb of addressable memory and CUDA support.
On the other hand, if you force the CUDA support condition and any automatic translation of CUDA programs is not accepted as good enough, then this mandates the use of a discrete NVIDIA GPU, which can be provided only by a mini-ITX mini-PC.
There are mini-ITX boards with laptop Ryzen 7940HX or 7945HX CPUs, at prices between $400 and $550. To such a board you must add 64 GB of DRAM, e.g. @ $175, and a GPU, e.g. a RTX 4060 at slightly more than $300.
Without a discrete GPU, a case for a mini-ITX motherboard has a volume of only 2.5 liter. With a discrete GPU like RTX 4060, the volume of the case must increase to 5 liter (for cases with PCIe extenders, which allow a smaller volume than typical mini-ITX cases).
So your CUDA condition still allows what can be considered an off-the-shelf mini-PC, but mandating CUDA raises the volume from the 0.5 L of a NUC-like mini-PC to 5 L and the price is also raised 2 or 3 times.
This of course unless you choose an Orin for CUDA support, but that will not give you 64 GB of DRAM, because NVIDIA has never provided enough memory in any of their products, unless you accept to pay a huge overprice.