I've used vast.ai (similar "Airbnb for GPUs" pitch) for years to spin up cheap test machines with GPUs you can't really find in the cloud (and especially consumer-grade GPUs like 4090s). Any insight into how this is different/better?
Also, don't know if vast.ai does this, but with us you can have 6 user sessions on your machine if you have six GPUs, so granular utilization is possible.
What would make you better than vast is extremely easy spot leasing and job prioritization.
I want to be able to have one of our training jobs finish, and then have the capacity immediately transition to a lease. With vast, we are renting in week long blocks.
(I'm the founder of vast btw - contact us for help on setting this up and/or any feedback on making it an easier/better process)
vast has a lot of bad machines with terrible PCIe lanes and architecture you have to learn the hard way. Someone on HN wrote a script to run a test docker image on every machine and auto-tagged the machines' quality using their API, which is what I'd do if I was going to use vast seriously for compute.
> vast has a lot of bad machines with terrible PCIe lanes and architecture you have to learn the hard way.
Wouldn't gpudeploy have exactly the same problem? How is it mitigated with gpudeploy?
I suspect it would be trivial for Vast or GPUDeploy to spin up a benchmarking job before allowing sales on that machine. I'm not an expert on PCIe lanes, but I would think the performance issues would be visible via bandwidth or latency on the lanes.
It kind of makes sense to me, though. If I were looking for absolute reliability and was willing to pay for it, I'd just go to one of the many GPU cloud vendors. Likewise, I suspect anyone willing to really work on getting good performance would rather be a real provider or sub-provider than being part of this nebulous C2C GPU cloud.