New – EC2 Instances (G5) with Nvidia A10G Tensor Core GPUs
aws.amazon.com
aws.amazon.com
Except when GPUs are involved. This just seems like something that dedicated hosting industry isn't on top of _at all_. I don't know if it's hardware availability, upfront costs, unreliable demand, an inability to compete on price, or if the usage really is so much more elastic.
The best I could [quickly] find was a 5600X w/32G Ram, 1TB SSD and 12GB 3060 for $189/m.
The gist of it was that these hosting providers exchange credit for cash-equivalents. For example, you can sign up with a credit card, and then immediately start using the compute that you have purchased to mine crypto.
Now, if you pay your bill, this ought not to matter. Who cares what you use your rented computer power for? (As long at is not hideously illegal child porn hosting or what have you.)
The problem was that the smaller providers were being mercilessly abused by criminals signing up with stolen credit card numbers. They would extract sufficient crypto coins before the provider shut them down for this to be worth doing over and over.
The providers could have simply requested up-front payment for compute until a "track record" was established, and this would have solved the problem.
Instead, they all opted to maximise the "sign ups per month" metric to get those sweet, sweet KPIs met, and then killed the crypto-mining accounts mercilessly with AI detection.
... which also killed several legitimate customer's businesses outright. Typically with no recourse.
Oops.
Edit. Found their message:
"At the following link you can upload the copy of your ID, or you can use our pre-paid option, where you pay €20 in advance via PayPal: https://accounts.hetzner.com"
I helped a niche API-as-a-service company go from -20% margins to being profitable. How? Moving away from AWS to Hetzner.
Yes your monthly bill is lower, but now the liability is all yours.
There are other options than the two that you've described - namely VPS providers and dedicated providers.
Can you help me understand why you thought those were the only two options? Maybe people who joined the industry after, say ~2010, were primarily exposed only to AWS?
Once more, it’s not like AWS is absolutely nothing. Chances are that any web-based startup would indeed be able to take advantage of the many autoscale-as-a-service features of AWS, enabling them to handle the next quarterly 100x increase in customers and throughout.
because triple++ digit margins enable cloud service providers to allocate vast amounts of money towards marketing.
My point is that you can't outsource reputational damage and blame.
My customers don't have a contract with AWS. A service they're paying for is down, and what am I going to do about it?
But they can sue you to bankruptcy if you mess up with their data. And these incidents happen way too frequently, even to big players, to ignore them.
You cannot casually say I am able to setup a Linux box myself, no need to hire experts / outsource this task to experts.
1 - Free credits
2 - No one ever got fired for buying IBM
3 - AWS is sold to CTO/CIOs. AWS is a massive marketing and sales success
4 - An entire job type has sprung up around cloud-complexity. These people have a vested interest is staying on cloud.
5 - Growing number of people think the choice is exclusively between cloud or colo.
6 - Entire generations brought up on cloud: a lot of people don't realize just how expensive cloud is.
Ah, so true. Heroku was my "solution" for so long, when its really so expensive on its own. All the pricing models are so messed up.
I have some Digital Ocean nodes at either $5 or $20/mo, I need to shut those down.
Deploying stuff that can communicate to other stuff is so easy right now and so cheap ... somewhere. Vercel to the rescue, for now.
The same people who know it’s expensive seem to not know how cheap it is.
It's literally pennies in some cases, vs. renting a server that sits idle 99% of the time.
Besides, you could either rent a $5/month VPS that you know will always be $5/month, or you can use AWS Lambda and never know what your bill is gonna be at the end. Your new blogpost hit frontpage of HN? Good news, your bill is now $40 for this month.
Most (85% maybe) of my clients I work with want consistent costs over cheaper costs.
You know whats expensive in AWS. S3 Storage. Got some odd ~500TB stored in there. But at 6k/m I can't imagine trying to keep that in dedicated hardware. Atleast in S3 I feel it's safe.
Not always. People on the ground push for cloud stuff because it makes their lives easier and they are used to it from their side projects (where they, more likely than not, used the generous free credits).
It's just like back in ye olde days when all you needed to do to get a copy of Photoshop was the official installer and a keygen... Adobe didn't invest much in marketing and didn't do much against piracy until the end of CS6/the rise of CC, simply to become the universal industry standard.
If you are at AWS scale, NVIDIA would definitely sue you ...
Despite supposedly being 4x faster and 4x cheaper, these dedicated hosting offer guys come and go like moths circling a burning flame.
Same thing with the s3 competitor file hosting companies. I haven't followed this space recently from hotfile / filesonic etc etc. All supposedly way cheaper than S3 - but boy do they disappear in the night at times.
For example, the quote that I found was from a provider I used over a decade ago (no affiliation), and who have been in business since 1999.
Processor 8 Cores HD 240GB SSD RAM 16GB $311 / month
This is not that competitive with m5a.4xlarge reserved for one year.
64GB / 16 core machines - good modern gen cores
And even if it wasn't, the existence of a single expensive dedicated server provider wouldn't disprove parent's claims.
My theory is that publicly advertising GPU instances tends to attract people who want to convert stolen credit cards into crypto mining power. I mean it's the same reason why the 3090 == GPU most suitable for crypto has been sold out for a year.
That's also in line with the fact that I had to sign a voucher stating that I will NOT do any kind of crypto hosting / mining / processing on any of their servers before I got that GPU instance.
I also asked Hetzner for GPU instances after they took down the ones from the website, and a price list was not available at the time, but for a sufficiently sized order they said they would provide me some.
A big part of the difference is in Nvidia's datacenter tax (The A10 is basically a 3090, just double the price). Hopefully AMD's new accelerators will bring some much needed competition to the market.
Cheapest reasonable GPU instance you can get on EC2 is P3.2xlarge with $3.06/h.
(The largest available compute-optimized instance is currently the c6i.32xlarge at 128 vCPUs, not counting the absurd u-* family of instances that mere mortals can't get. 192 vCPUs would be a substantial upgrade; 256 would be even better.)
And no, GPUs aren't appropriate for this workload type. We're bound on IO and memory more than raw compute.
> Unlike the fully unlocked GeForce RTX 3090 Ti, which uses the same GPU but has all 10752 shaders enabled, NVIDIA has disabled some shading units on the A10G to reach the product's target shader count. It features 9216 shading units, 288 texture mapping units, and 96 ROPs. Also included are 288 tensor cores which help improve the speed of machine learning applications. The card also has 72 raytracing acceleration cores.
As CTO from two unicorns, I’d say that it’s pretty much wasted money.