I have H100s to myself, and access to more GPUs than I know what to do with in national clusters.
The Spark is much more fun. And I’m more productive. With two of them, you can debug shallow NCCL/MPI problems before hitting a real cluster. I sincerely love Slurm, but nothing like a personal computer.
As for debugging, that's where you should be allowed to spin up a small testing cluster on-demand. Why can't you do that with your slurm access?
It's remarkable what can now be done on a whisper-quiet little box. I hope the Strix Halo's will be just as much fun, and they should be, so long as Flash Attention works.
Fair, thanks for the answer.
The bane of my existence...
salloc: Granted job allocation 1978
salloc: Waiting for resource configurationEven ignoring GPU details spark is an awesome little quiet powerhouse arm64 workstation that is 100% Linux first
Curious though how you offer idrac to customer, do you have another OOB BMC for the idrac? Or is this internal engineering context
We rent bare metal on-demand and our whole business is to be able to offer compute that you probably wouldn't be able to host in your house $, as if you own it yourself.
So, we made it so that users can get access into the BMC and modify the box however they want. When they are done, we've automated the reset as well. Fully self-service.
$ These boxes are very expensive, weigh 350lbs, sound like a jet engine and consume ~10kW.
That’s a big part of why we hot stage things for customers.
But it’s still not quite like exclusive access to resources when you want them. So I can see it from both ways.