Next-Generation GPU-Powered EC2 Instances (G3) | Hacker News Reader

Next-Generation GPU-Powered EC2 Instances (G3)

aws.amazon.com

88 pointsjanober71 comments

They say "next-generation", but these are M60 GPUs, which are very much "previous-generation". Current generation would be P100 GPUs.

I am in the market for a cloud GPU offering, and I have to say the big cloud providers are very uncompetitive here, only offering these old, slow GPUs.

I was also surprised (and sent it around to our team internally last night). We're skipping Maxwell entirely as you can see from my previous comment threads.

For display it's still a fine part. The P100 is also a beast, so its overkill for most people just doing Remote Desktop. So perhaps the M60 (like with Azure) fills this market segment for them, and they don't mind the hardware diversity.

[Edit: Too sleepy. A post down below reminds us that these are G-series and G is for Graphics. So yeah, I assume they just didn't want to wait for enough P4 parts in volume or will quickly make another such announcement about the Pascals].

Disclosure: I work on Google Cloud.

There was an announcement last year that AMD FirePro™ S9300 x2 were coming to Google cloud. Any reason why those have not hit general release yet? I have some OpenCL workloads that I could put to good use on those.

Still on it, and now coming really soon (hopefully). There were a lot of interesting challenges (power, software, datacenter process) along the way, since this is very new hardware to Google.

Out of curiosity, why OpenCL on AMD rather than OpenCL on say a K80 or AWS's M60s or upcoming P100s? (That is, are you blocked on having an AMD part, or do you have a great reason to prefer AMD for compute)

Disclosure: I work on Google Cloud (and want to sell you GPUs!)

Because the code is already validated / tuned and deployed on AMD parts. For a while NVidia dragged their feet on OpenCL so validating the app on their hardware was not possible.

Great feedback, thanks.

FWIW, we haven't invested in opencl for anything due to the fragmentation. It's great to see google pushing an alternative.

Overall though, I would basically consider opencl "AMD's thing".

Not original poster, but for us is the lack of OpenCL2.0 on nVIDIA HW (mainly shared data structures) Only OCL 1.2 on Linux from nVIDIA

I think "generation" here is referring to EC2 generations and not GPU generations – AWS tends to use that term to refer to new instance types being released.

((Next Generation) (GPU EC2 Instances)) rather than ((Next Generation GPU) (EC2 Instances)) :)

"I am in the market for a cloud GPU offering, and I have to say the big cloud providers are very uncompetitive here, only offering these old, slow GPUs. "

It's one thing if one of them is like that, but if all of them are like that, maybe it's not because of the cloud providers?

Oh, there are smaller vendors renting out Pascal architecture GPUs. Yet Amazon and Azure are stuck at prev-gen (M60), while Google is all the way back at prev-prev-gen (K80).

Would you mind specifying these smaller vendors and a few details about your experiences?

I don't want to promote anyone in particular, but there are a few vendors here, for example:

https://www.ovh.ie/dedicated_servers/gpu/ https://www.exoscale.ch/gpu/ http://www.cirrascale.com/cloud/plans.aspx http://www.nvidia.com/object/gpu-cloud-computing.html

Last link has a list of vendors. The one I used gave you a bare bones machine running some Linux distro, with no platform bullshit around it. Exactly what you need for deep learning work.

Or GTX 1080 ti. Aren't the Tesla class like the P100 mostly super overpriced for deep learning because their only main advantage is Double (64 bit) float support, and no one really needs that? Plus half float (16 bit) support, which is not super widely used (but certainly more than double). Something like 95%+ of Deep Learning must be done with single floats (32 bits) right now afaik, making this a fairly dubious expense

> Aren't the Tesla class like the P100 mostly super overpriced for deep learning

Yes. Nvidia has real competition from AMD in consumer graphics but no competition in enterprise. Their consumer cards are sold at a competitive price while their enterprise cards are marked way up.

If cloud providers were really smart, they'd release cheaper/just-as-fast instances with consumer cards, but Nvidia probably doesn't want to do that deal; they'd prefer to push teslas as the standard for GPGPU, even if they have to mark them down for cloud providers.

As DannyBee said in a sibling comment:

> It's one thing if one of them is like that, but if all of them are like that, maybe it's not because of the cloud providers?

What is your point?

There may be non-technical reasons beyond "Those guys are just stupid" that we don't offer GTX 1080s.

Why don't you offer GTX 1080s?

I'm suggesting Nvidia is offering great deals on teslas to cloud providers but not offering those same discounts on 1080s; they're doing that because it lends support to their pitch to enterprises that they need Teslas instead of their competitively priced consumer line; if everyone was paying list price, cloud providers would be offering 1080s.

Even with all that, cloud GPU is not cost effective and end users are better off buying their own "consumer grade" stuff.

Deleted comment

Looks to be about twice the cost of the M60 on AWS, but you should get quite a bit more performance out of that system.

The g2s were Kepler, the g3s are Maxwell. 'Next generation' is technically correct.

Current gen is Pascal though, so 'Prev generation' is more correct.

In architectural terms perhaps. But the M60 is most definitely a current generation Tesla product.

These are G-instances and graphics is one of the primary use cases for this instance (and for the previous G2 instance). The M60 is the top part with GRID support built in. A GRID license that allows you to use the GRID driver is part of the offering.

Disclosure: I own HPC for AWS (among other things) and used to own instances

Not really. Typically there were reasons you would stick with older generations, such as M60s having very poor fp64 performance. If you want fp32, it gives a bump over the K80s currently in there. However, the P100 would have been the most logical path since it doesn't cripple fp16/32/64.

His point is:

They didn't say newest generation, only next and next may in this case refer to

- "next after kepler" (parent's assumptions) or

- "next after current" (other commenters' assumption)

Anyone care to chime in on why spot instances are now 10x on-demand instances? I've got a thread going here: https://news.ycombinator.com/item?id=14769026

That happens not infrequently. 1#x ondemand is the ceiling bid for spots. It's the result of a bid war amongst two or more big customers who really don't want to be evicted.

Lol we don't even have the P instances in Sydney yet, so I'm not holding my breath here.

Under what circumstances do you care about the location?

When I'm using cloud GPUs it's pretty much a batch job, and latency is the last thing I care about.

I'm not aware of any DL projects in Australia on health images which may have some legislative requirements about keeping data onshore.

Data transfer between EC2 and S3 in the same region is free but incurs costs between regions.

The projects I'm working on unfortunately do; less legislation directly though and more extremely exacting regulators.

More selfishly, I game using parsec.tv and would love a new rig, and it's pretty latency sensitive! :)

They definately do, only the p2.xlarge though I think.

If you want some current generation GPUs ;) check out https://paperspace.com

Once again, no instances with multiple powerful GPUs and like 1-2 CPU cores and 1GB RAM… Not doing them to discourage mining? :D

Far from it. For a long period the floor spot price of aws gpu instances tracked the bitcoin mining positive roi threshold.

Sad to see no fractional-GPU instances. A 4xlarge is massive overkill and unaffordable for our use case.

You can't give fractional GPU instances with this card. The K80 had two logically separate chips that were separately-addressable over PCIe. This allowed them to send two different PCIe devices to different VMs. The M60 doesn't have this. The V100 is supposed to allow time slicing to do this kind of thing, but that's not out, nor do we know how well it'll work.

So?

> up to 18 H.264 1080p30 streams

How is the quality compared to x264 with the default settings (preset medium, crf 23)?

A lot worse, but your CPU doesn't take a hit. NVENC doesn't have very good quality at low bitrates, but it's fine for local recording (1080p@15Mbps+) that will be transcoded later.

Here's a comparison video: https://www.youtube.com/watch?v=BV5btdqQfu4

According to this it's almost equivalent when you compare 720p@5Mbps and 1080p@12Mbps, which is way more than most streaming sites will do: http://on-demand.gputechconf.com/gtc/2014/presentations/S464...

NVENC[0] has some comparessions with x264.

In our use case(sports broadcasting, 720p) we found that in reasonable bitrate(>1Mb) NVIDIA HQ quality was virtually the same to x264 faster. (in newer versions of nvenc, they got amazingly better in the last couple of years) When the bandwidth drop you start to see x246 advantage.

[0] https://developer.nvidia.com/nvidia-video-codec-sdk

I'm taking this and Nvidia's announcement it was going to sell a mining-oriented GPU as the shot over the bow for cryptocoins. But then again, only market-makers get rich calling a top.

Has anyone done much Linux gaming on EC2? I want to be able to play xonotic again but I don't play it often enough to justify buying a high power desktop.

I did some mac gaming on it. Not terrible for certain games. I was mainly playing Rocket League multiplayer. There's a bunch of resources/experiences at https://www.reddit.com/r/cloudygamer/

Is it profitable to use these for mining cryptocurrency?

Nope. I have made profitability assessments on several different cloud GPU solutions having different hardware.

As a general rule, for every 100 USD you'd only mine about ~50 USD's worth of crypto-currencies.

Which is not surprising since on these products you get a fancy motherboard + high-end Intel CPUs + boatloads of RAM.

These are of little to no use when mining, and account for about half of the cost of this hardware. Also, the local cost of electricity is not the lowest price in the world (China having one of the lowest)

While the price of hardware is fixed, crypto-currencies possess a difficulty adjustment mechanism. This makes the whole system have an upper bound on mining profitability, and this bound converges on the profitability of the best-yield-hardware's. Which would be something to the tune of this [1]. Note that while having 6 GPUs, this system has 8 GB of RAM and an Intel Celeron.

[1] https://blockoperations.com/6-gpu-mining-rig-amd-rx580-intel...

----

EDIT: We're talking about IO-bandwidth-bound crypto-currencies here, like all the ones based on EThash[2][3], Ethereum being one of them. Bitcoin's upper bound on profitability is set by the best ASICs for SHA256 processing.

[2] https://github.com/ethereum/wiki/wiki/Ethash

[3] https://github.com/ethereum/wiki/wiki/Ethash-Design-Rational...

You probably didn't evaluate Spot prices. I wrote a tool once to evaluate current AWS spot prices against the Ethereum price and sometimes it was a low as losing 2 cents per hour and G2 instance.

AWS is by far the most profitable business unit at Amazon, with margins consistently in the 25% range.

https://www.recode.net/2017/4/27/15451726/amazon-q1-2017-ear...

if this is true:

> it was a low as losing 2 cents per hour

then it seems someone should be able to negotiate with you (even if not Amazon), especially if you're willing to use 1,000 or 10,000 instances - and don't need memory, etc.

it seems 2¢ should be within what you might be able to shave off of somewhere...

maybe not, though.

Funny side note: when I prepared for getting into all this mining and increasing G2 quota in N Virginia from 2 to 20 or so, customer support first needed to check "if this won't disrupt overall region stability". Don't know if this was bs or if scalability in Aws might be worse than we think.

Used to get told the same thing for GP2 EBS and R3.8xlarge limits in us-west-2 all the time. On a couple occasions was told no, that increasing the limit wasn't possible due to existing commitments in the region, so I'm not entirely sure if it's a canned response or actually a legit concern of theirs.

We frequently request account limit increases to 1000-2000 instance limits and get the same canned response. Though I bet in the earlier days capacity allowances for GPUs was probably more sensitive.

I have some free AWS credits and would like to use them to mine some cryptocurrencies. Anyone here able to help me out as I'm quite a noob in this field..? I thought if I can launch loads of instances at the same time to mine the same currency, I would burn all my credits in no time but have higher rewards, or am I wrong?

Love to hear your thoughts on this!

hey I could help, please email me at convnet@protonmail.com

It's not profitable to mine using your own money. But i am pretty sure those instances will be abused by carders. Like you invest 100$ in carded money and get 50$ back in cryptocurrency.

No. Nowadays the big players in mining cryptocurrency have datacenters full of ASIC's. The currencies that are resistant to ASIC mining (due to eg using memory-bound hashing functions), like Monero, are probably just as resistant to GPU mining as they are to ASIC mining, although if you were to investigate it, I'd look at one of those and not bitcoin.

No. Making a GPU resistant algorithm is hard. Ethereum is ASIC but not GPU resistant.

Why is that? I understand that GPU's have a ton more memory than an ASIC, but couldn't you attach a huge amount of RAM or something to your ASIC (kinda like how GPU's work)?

Those ASICs will cost as much as GPUs or more for the same amount of performance.

Thanks!

Although, since an ASIC is application specific, couldn't you pack more cores (since you can ignore the circuitry that you don't need) or otherwise optimise for the application? If you can do more work, or do the same amount of work with less energy, then it could still be worth it.

I am obviously ignoring a rather important factor: its unlikely a "small" player could make a more performant ASIC than a "big" player like the GPU vendors, who have invested billions into building high performance hardware in a cost effective way. I'm not really asking "why don't people do this", but rather asking if its possible at all given a big enough budget.

This does not apply to Equihash (ZCash) or DaggerHashimoto. NiceHash consistently mines those on my 2 rigs (3x 1070 GTX rigs.)

gaming GPU prices and availability would like to talk with you. where i live, 1070 price has gone up 50% in 6 months - if you manage to get one. it's crazy and was attributed to ethereum/zcash mining.

A similar thing happened when bitcoin GPU mining became big. I was watching an interview recently with one of the large mining operations where they basically bought all of a suppliers GPU stock (of whatever type of card was popular for mining at the time) and couldn't find any other suppliers who weren't out of stock.

Of course not. Then everyone would use it for that purpose alone.

well in 2011 i was using a Desktop PC + 3 GPUs to mine bitcoins which was barely profitable at a Bitcoin price of around $20 USD. Would i have kept them though and not sold at that price.... FML

I could reward if someone can help me out burning my AWS credits to mine lots of cryptocurrencies! ;)

Does anyone know what the difference between G and P is supposed to be, conceptually?

P are intended for general-purpose GPU compute applications (and have 1, 8 or 16 GPUs, more RAM and fewer CPUs). Typically you might use these for scientific computing / machine learning / anything CUDA intensive.

G are optimized for graphics-intensive applications (and have 1, 2 or 4 GPUs, less RAM and more CPUs) - you might use these for design work, gaming etc.

Reply on news.ycombinator.com