Introducing Preemptible GPUs
cloudplatform.googleblog.com
cloudplatform.googleblog.com
That was before preemptible GPUs: with the halved-cost, the cost-effectiveness of GPU instances now doubles, so they're a very good option for hobbyist deep learning. (I did test the preemptible-GPU instances recently; they work as you expect)
Whilst I would agree that university teams probably should use the resources the cloud providers make available freely, they should probably stay away from actually using capacity on the cloud and instead have their own hardware.
Besides, what I keep hearing from machine learning researchers is that no matter where you work, developing on your own machine ... there's no beating that, time and productivity wise.
Although the cost of a K80 preemptible instance on GCP is now close to the approximate cost of a K80 spot instance on AWS, so there's a bit of competition.
Note that our pricing is flat regardless of number of GPUs attached (and you don't need to buy as many cores to go with them). By comparison, Spot often charges greater than on-demand pricing for anything other than single GPU instances.
Thanks again for your write up, sorry about the confusion as we delayed announcement until the new year.
IANAL though so I might have interpreted this incorrectly.
http://www.nvidia.com/content/DriverDownload-March2009/licen...
And instead of restricting our rights, shouldn't we get a discount when buying multiple GPU cards?
Similar strategy of Adobe won't sue a single user for pirating Photoshop, but the second they have a successful business...that's a different story.
A great problem to have. Maybe first concentrate on creating a billion dollar business and by that time you can afford to get some 'approved' cards.. ;)
Unfortunately, using nVidia GPUs with a Mac is still fussy even in High Sierra. And with the GPU instance price drops making deep learning pay-as-you-go super affordable, it's no longer worth the physical investment in a card, especially because they depreciate quickly.
And you will still have pretty powerful PC at your home for everyday use/gaming
0.3 kW/h * 0.20 $/kW * 24 h * 360 days = $518.4 per year of electricity
In my experience, people always operate around break even. They only make good money if they held their coins and the price increased over time.
I'm not even being sarcastic, I'm thinking of building my first gaming PC this year.
TLDR: buy a used desktop business-class with a decent PSU for $300-$400, and stick in a GPU in the 1060 class.
Googles preemptible model achieves a lot more fairer distribution of GPU's as opposed to AWS's model. Though AWS ended up with spot market since you got charged for the entire hour up-front and if AWS evicted you, then you got the entire hour for free, this made pricing spot instances complicated for both users as well as AWS very difficult. With shit to per second billing this issue has been eliminated to a large extent since you only get first 10 minutes free if you get pre-empted within those 10 minutes.
tl,dr; GCP preemptible instances are superior to AWS along all possible dimensions, price, flexibility, IO etc.
While I appreciate the sentiment, there are certainly things some people prefer about Spot. For example, you basically get up to 4 minutes notice compared to the 30 seconds I chose (see the other thread). Similarly, until this change we didn't offer preemptible VMs with GPUs attached.
You're right about the complexity of Spot. Preemptible (and Azure's copy, Low-Priority VMs) is all about a fair, predictable price. Not everyone hates markets, but the number of companies and customers burned by the Spot market gave us the conviction to push for a simpler, fixed price.
Again, thanks for the praise (but there are pros and cons!).
I wish they put the minimum time you will be granted cycles though, as that information seems like an important design constraint when choosing the size of data chunks you design for.
That's a good start...looks like the model is "save your work often". I'm sure you could run a few instances for a while and model the distribution of time granted.
Disclosure: I used to work at Google on GCE, but not directly on preemptible VMs or this billing policy. I don't currently work or speak for Google.
Once you have a good pipeline and need to deploy it, sure. Then we can talk.
I don't think anyone has such a product at this point.
Some jobs have time constraints and need to be completed as quickly as possible.
And I have very bursty workloads.
I would love such a service.
Btw, I found some tasks (and some algos) don't parallelize well (e.g. would get any "faster" from employing 40 GPUs instead of 4), but it would definitely make sense to be able to run multiple models (or training) in parallel. Once your needs are well understood, it's almost always significantly cheaper to build your own GPU cluster.
No one does hourly billing anymore, thankfully. GCE and AWS do per second or per minute...
What I do now is autoscale a group of GPU instances in AWS based on the size of the jobs queue. I create as many instances as will reasonably help, sometimes up to 100. Work is distributed to them. They turn off immediately once there is no more work to do.
The code runs in Docker containers but I am forced to maintain the base linux system and nvidia drivers b/c no one provides a container or FAAS for nvidia gpu computation...
I get the sense that this is a common problem nowadays. The way NVIDIA manages software releases doesn't help anything. There's quite a bit of .. churn. They don't play nice with anybody else hence Linus's famous words.
There's totally a market for containers as a service with passthrough access to NVIDIA hardware. It best not be more than 30% more expensive than a raw instance though, or it won't be very exciting.
I'm still sad about the PiCloud folks. They were trying to do this (sort of), but it's a tough business to be in as you sit between the cloud provider and the customer needing to make money in between. IIRC, they were an early Spot customer and as the cost of their infrastructure shot up, they got squeezed out. You could imagine something similar again for running CUDA programs in a "serverless" fashion, but it would probably have to come from a provider.
Actually both Keras and raw TF have a check point thing built in (as does Torch). I believe it can be setup to do so every epoch, but unless you're talking about many many GBs of parameters, you can also stream lots of data to GCS in 30s.
I don't consider that a nontrivial amount of code to write. Am I missing something?
Performance wise that P100 is close to GeForce 1080. That GPU currently retails for $550.
For the price of 1 month of preemptive google’s IaaS you can buy the GPU and use it for as long as you want only paying very small amount (depending on where you are $0.02-$0.1/hour) for electricity.
However, IMO the price is just not OK.
Compare with traditional servers. For $0.7/hour you can use e.g. c4.4xlarge amazon instance. The hardware is much more expensive than $550, just the RAM is already around $400. You won’t be able to purchase an equally-performing server for 1 month of that IaaS rent, I think it’ll be like 3-6 months (which is IMO reasonable).
If you can find one.
The ones that aren't, are asking $1,200+ per.
https://www.newegg.com/Product/ProductList.aspx?Submit=ENE&N...
The real power of cloud is not a replacement of one instance but the ability to turn on tens, hundreds or even thousands of nodes.
Out of curiosity, in June or July I ran some Ethereum mining stuff on Azure's GPU instances.
The ROI was around -90% (negative 90%, to be clear: for $1 spent, I mined ~$0.10). Even with the higher price of Ethereum now, you'd probably be around that same ROI since the difficulty has also increased a fair amount.
Interestingly, you cannot run consumer GPUs in a datacenter, as Nvidia driver terms prohibit it.
Your account team can also advise you on what zones you should deploy preemptives in. Googlers internally can see current preemptive capacity and preemption rates on a per zone level. In certain zones you're more likely to have preemption due to large customer demand.