GCE vs. AWS: Why You Should Never Use Amazon | Hacker News Reader

GCE vs. AWS: Why You Should Never Use Amazon

thehftguy.wordpress.com

64 pointsayberk51 comments

I'm always extra-skeptical about devs who choose to use click-baity titles to explain why a widely used piece infrastructure is useless. The merits and demerits of AWS have been discussed in depth by folks with way more experience and knowledge than this author, and in most cases "Never Use Amazon" would be found as a laughable conclusion.

Nevertheless, I hope AWS reads this to get an idea about how they still need to improve their accessibility and image as a company overall. It's completely true that the transparency and accuracy of the "status page" is terrible because no product team wants to admit something went wrong. It's also true that their Premium Support is constantly overwhelmed and often not incredibly useful but still essential for many cases (This basically boils down to them having huge turnover rate, a low number of people who actually have skills to do the high-level support, and having too many ways to "game" the system where customers who spend nearly nothing get away with taking way too much support time and substituting paying actual architects with constant support cases. Increasing cost of support would actually be the best move here.) Regardless, this plays into the narrative that I've always observed. AWS, despite being easy to get going with, is also incredibly easy to mess up. The fact that there is a certification system spanning so many different parts of AWS should go to show the disconnect between how much AWS actually expects you to master before using it, and how much the average dev really wants to learn before jumping in (reading a blog post is not gonna cut it.)

Yes the title is click baity to the extreme. The author hasn't used GCE extensively (has anyone?), so recommending GCE is disingenuous.

Despite that, I found the article to be a good summary of some of AWS's worse sore points. EBS performance really is a dog. He failed to mention c3 instance types though, which also have locally attached SSDs.

Why is bare metal so "uncool"? I am certain that a lot of companies don't need the flexibility provided by AWS. I can understand that you don't need additional staff by going with the AWS route, but in the end you get a bare vm which is kinda similar to a bare metal server – you have to manage it yourself. Things like RDS isn't good idea anyways, let it be in the startup phase (expensive) or later on (inflexible, etc.)... The only service I could think of which is really helpful is SES, because handling email is very stressful.

And even if you need the flexibility you can still run your predictable stuff on bare metal, while, when the traffic spikes, spin on some cloud servers as you need them. But things like load balancing are always better handled on own servers, where you actually know what the fuck is happening.

Bare metal isn't uncool .. it just requires someone who is willing to manage it. That includes rushing to the data centre at 3 a.m. to fix a failed power supply or a hard drive. In comparison, with a cloud provider and a proper architecture, a huge number of problems just disappear: if your server stops responding, it is automatically killed off and replaced; there is no need to worry about the operational overhead of managing a DB (backups, clustering, recovery etc) when you just consume it as a service; you don't really need to worry about maintaining a separate disaster recovery site when it is actually easier just to run in active-active mode; when your box is no longer adequately sized, you just stop it and restart it as something larger. Cloud computing provides a huge number of benefits - once you have used it, it is pretty hard to go back to managing your own hardware. That said, it isn't always the best option and it certainly isn't always the cheapest.

there is no need to worry about the operational overhead of managing a DB (backups, clustering, recovery etc) when you just consume it as a service;

Except that that is technically impossible! Even if it were somehow magically true, all of that magical infrastructure would be completely useless without an application which could run her own instances in parallel, and do synchronous updates of global data state from all nodes at once. This is a non-trivial computer science problem which has not been reliably solved to this present day, so the entire premise is flawed.

There is no getting away from good old fashioned UNIX system and database administration, and in this day and age of distributed applications, the stakes are even higher, as now there is no getting away from system engineering. Incidentally, that includes administering and servicing hardware, but even more importantly, it includes engineering fault tolerant, distributed infrastructure. There is no getting away from that: you learned how to program, now in order to use that knowledge, sooner or later, you will have to learn how to administer systems and databases, and then you will have to learn how to engineer entire systems.

Computers used to be fun when you had a Commodore 64, Commodore Amiga, or an Atari ST in front of you and you were a single user on a single system; now they are just complex and hard, and if you want to write a highly available, fault tolerant application which continues to operate correctly even in the face of severe hardware and software failures, there is no getting around learning how to administer and engineer systems.

There's something called "remote hands" and the big DC operators like Equnix offer it. I haven't been in a DC in years. Also just because its bare metal doesn't mean you don't need to design for failure just the same as do with AWS.

And those problems don't just disappear by using AWS they just turn into different problems, certainly more opaque, like IP addresses that disappear from underneath you and the noisy neighbor problem.

Its interesting to me that there is now a generation that isn't familiar with hardware. Knowing hardware will never count against you, especially if you want to do consulting or go work for a cloud provider or need to run a hybrid cloud.

You're right, remote hands is available, but in my experience (7+ years managing a DC in another country), it is expensive, inexperienced and slow to respond. If an issue occurs in a DC that's a drive away, I'll usually send a staff member because I know that the job will be done correctly and (usually) quicker.

You're also right that the problems don't disappear, but they are certainly easier to resolve in a cloud environment. I have had boxes fail and get replaced in the time that it would normally take me to wake up and logon to work out what the issue is.

I also agree that knowing hardware 'doesn't hurt' (it's where I started), but it also comes at a cost - the time required to learn and understand it. If you were starting out in IT today, would it make sense to spend time on that, or maybe learn about programming or architecture or something else. I don't know the answer, but it's certainly something to think about.

It's expensive but its also lot cheaper than paying someone full time that might not be needed full time. It depends.

The difference in metal vs a cloud provider is that with bare metal you can dive deeper into why things are failing beyond what you might with a cloud provider. That may or may not be important to you. It depends. With a cloud provider you often just spin up new instances and hope they land somewhere better.

If you were starting out in IT today it would absolutely make sense to learn hardware. The cloud is just somebody else's hardware.

You can rent bare metal with the hardware managed by the hosting center. Hetzner.de provides this. If there's a problem with the software, you fix it, if there's a problem with the hardware, you tell them and they fix it.

Speaking as someone who works at a small startup that transitioned from a hosting centre to AWS a couple years ago, we sliced our monthly hosting costs by about 1/3 immediately, and gained a massive amount of flexibility. Maybe GCE is better is some respects, but we're certainly not regretting the choice.

I have no idea, every time someone in the company talks about moving anything to the cloud except our off-site backups and MAYBE a DR plan I cringe hard. The vast majority of our infrastructure is fixed, we have a bunch of MSSQL databases, Exchange, AD, Laserfiche, VMWare VDI, our IBM i running our billing software, and a bunch of production application servers. We don't have any benefit to the "flexibility" of the cloud, we rarely spin up new servers as our load is very static, and it's all internal. We have millions of dollars invested in our current IT infrastructure and that is chump change compared to what we'd pay to move to Azure/AWS/Google for no real reason.

Hell, I have physical hardware in my home (a Lenovo TD340 running XenServer and a HP ProLiant ML10 running FreeNAS) for my personal / small project use because for the upfront investment cost me as much as 1 month of the same capacity on the cloud, especially factoring in my 14TB RAW / 7TB available storage sitting in my SA120 enclosure and having 72GB of RAM between them.

The cloud is great for really small things where a $5-20/mo VPS from Linode or DigitalOcean will do you just fine, but once you get past that if you aren't running a small steady-state with the need to burst on-demand it's just a money-pit.

For startup phase, its all about speed, I can provision new machines, databases in minutes which have sane defaults and predictable behavior. I do not have to worry about configuration, backups etc. And, its not about difficulty, its about speed, if I have to recruit one less person (which is ungodly difficult tasks), or spend more time on product, I'll go for cloud.

For large companies its again about flexibility, while they can manage/own data center. Core products with predictable load are good for data center. But in SOA or microservices, where a team is responsible end to end for service, cloud is godsend. Getting a new bare metal server is slow and bureaucratic. Boxes are assigned to cost center and getting them moving around is difficult. Getting temporary servers for an experiment is difficult. So the company has to go for internal cloud or external like AWS

this is the same conclusion that I came to as soon as the article started listing specs and mentioning SSD.

> "Unfortunately, our infrastructure on AWS is working "

> "I learned recently that we are a profitable company, more so than I thought. Looking at the top 10 companies by revenue per employee, we’d be in the top 10."

Congratulations. You just convinced me to go with AWS

As somebody who is stuck with AWS for now, I wish you luck.

You would have real fun when one of your running instances shuts down on it's own (and possibly comes back after some time), and would be pointed to the AWS SLA upon asking them what had happened.

You would have more fun when rebooting an instance would take 2 hours to be told that there was some underlying issue with the hardware.

For samples of the responses that you get from AWS:

https://news.ycombinator.com/item?id=11822298

https://news.ycombinator.com/item?id=11822544

> You would have real fun when one of your running instances shuts down on it's own

Or you could author software and systems that aren't so brittle that the loss of a single machine is problematic for you. Seriously, your engineering practices need some love if this is a problem.

I fail to understand such arguments.

Does that imply that every single webpage in the world needs to be behind a load balancer with unwarranted amounts of redundancy in place?

And if one doesn't do that, it's ok if the site is down.

Well, yes, that's exactly the point - a single site without redundancy does mean that you will get some hours of downtime every now and then; either it's ok if the site is down for such time, or extra redundancy is actually warranted.

If redundancy is unwarranted, then the site can't possibly be mission critical?

Because these types of problems never occur when you are running on bare metal?

In fact it is worse: typically you are resource constrained as the budget cycle to buy new hardware doesn't start up till next quarter so you literally don't have another machine to run on so you are screwed.

That isn't to say that AWS is without fault, but I find most comments about unresponsiveness of systems or the inabaility to find a route cause (i.e. A bad disk) would happen anyway with another system designed in the same fashion.

I feel like it would be easier to find staff that knows or is interested in using AWS over GCE, and that can be more important than $X extra dollars a month. Sometimes, the right solution is the most practical solution, not the most optimized solution. Once AWS becomes a problem, you're probably well off enough to look into other solutions. But I'd think you could just tweak your set-up.

I feel that I it would be bad hire if staff is uninterested or unable to learn GCE.

Let me tell you the different between use bare metal and use AWS, as current my position in a startup, I am the only developer, our system includes more than three services for two different type of users and admin. I coded all these services and put them online within 2.5 months, and all these possible is because I use AWS. There is no way if I use bare metal. (I have been there before)

If you truly understand how to design your system in cloud infrastructure fashion rather than try to manage the way to use a cloud infrastructure as a bare metal, then, you can have a really long term benefits.

Most instance types you can launch with ephemeral disk that is local SSD. All recent generation servers are like this. m3s, etc. You don't need to launch an i2. There are a lot, lot of problems with this article.

If you want a BIG local SSD, no; I2 (still at the same price that it launched with almost 3 years ago!) and X1 are your only options. For fast, cheap networking and large local SSD GCE is much better (for now at least)

You are probably right for big, cheap SSDs and fast networking GCE likely is the place to be. It's a newer stack. But this article fails to prove that, and it provides a lot of FUD.

"Local SSD storage is only available via the i2 instances family which are the most expensive instances on AWS" - He didn't say BIG. And gp2 is actually pretty good in my opinion. (Also, you missed the now legacy hi1 instance in your list.)

Just for completeness I give you other problems in this article. I run a moderately large AWS workload, and there are definitely problems with AWS, but this article missed many of them and feels like a rant more than something with veracity.

1. "Add 10% for dedicated instances" - but he never brings up a competing product on GCE. There is another simple way to getting a dedicated instance: launch the largest instance type in a class, and you have dedicated hardware.

2. "Add 10% on top of everything for Premium Support (mandatory)" - Simply wrong.

3. "We are forced to pay for Provisioned-IOPS whenever we need dependable IO." - This was true 5 years ago. I'm sorry, gp2 is a fine service which runs great.

4. "Local SSD storage is only available via the i2 instances family which are the most expensive instances on AWS" - Wrong as stated above which you didn't even refute. You just put parameters on a use case.

5. "An unplanned event is a guaranteed 5 minutes of unreachable site with 503 errors." - This is certainly hyperbolic. I've sat down with the ELB team, and we've hashed through this. We had an ELB that ramped up to 600,000 rps everyday with massive spikes in that ELB's performance. They offer pre-warming as a convenience, but it's hardly necessary. The worst case scenario would be some number of 503s for some amount of time up to a maximum of 5 minutes. How does GCE perform? No answers given.

6. "All resources are artificially limited by a hardcoded quota, which is very low by default. Limits can only be increased manually, one by one, by sending a ticket to the support." - This isn't true in GCE? Of course it is. There isn't infinite infrastructure. The AWS limits are published, and they are kept low so, for instance, it would be difficult for a developer or a malicious user to run up a giant bill for you. Learn to plan.

7. "There is NO log and NO indication of what’s going on in the infrastructure. The support is required whenever something wrong happens." - Cloudwatch anyone? It gives pretty decent instrumentation of, for instance, your ELB. What does GCE give him? Not mentioned.

8. "We have to comply to a few regulations so we have a few dedicated options here and there. It’s 10% on top of the instance price (plus a $1500 fixed monthly fee per region)." - Amazon was a first mover in this space. I think this is a shoehorning of something that is complicated and manual on their side. What does google offer, I repeat?

9. "A reservation is attached to a specific region, an availability zone, an instance type, a tenancy, and more." - This is garbage. You can change reserves between availability zone and instance type (within a family of instances.) Moreover, you can sell your reserves. I agree vaguely this is effort, but it's minor effort that's generally solvable with a mild amount of planning.

10. "The discount is small." - It's like 40-70%, he quotes 30% on GCE.

11. "AWS Networking is sub-par" - I actually agree with this, it feels like an aging infrastructure, but I imagine it's something they will address.

I didn't read all the points carefully, but regarding #9 one needs to see https://aws.amazon.com/ec2/faqs/

You can modify existing Reserved Instances via the Amazon EC2 Console or by using the ModifyReservedInstances API call.

In both cases, the reservation must match the following attributes of the running instance you want to cover:

    Availability Zone (e.g., us-east-1a)
    Instance type (e.g., m3.large)
    Platform (e.g., Linux/UNIX (Amazon VPC))
    Tenancy (e.g., default)

>You are probably right for big, cheap SSDs and fast networking GCE likely is the place to be. It's a newer stack. But this article fails to prove that, and it provides a lot of FUD. "Local SSD storage is only available via the i2 instances family which are the most expensive instances on AWS" - He didn't say BIG. And gp2 is actually pretty good in my opinion. (Also, you missed the now legacy hi1 instance in your list.)

Not sure what you mean by "prove"...for instances with "large" (say over 250 GB) local SSD you have I2 and X1 on AWS, both very expensive (and I don't see H1 on https://aws.amazon.com/ec2/instance-types/ any more). He specifically gave what SSD sizes he uses (800 and 375 GB) so the implication is that's what he needs. It is a bit sad that to get the equivalent storage performance of a $200 Samsung 850 Pro SSD you need to spend thousands a year on AWS, and for the equivalent of a higher end PCI-e enterprise SSD, I think only the X1 is competitive (which appears to be only available at a $3.970 3-Year Partial Upfront RI Price per Hour..wow!)

As for prove, I mean you can't just make crap up and pretend it's true. Your points are more logical than his for your use cases.

http://aws.amazon.com/ec2/previous-generation/ -- you have to use a different page. I guess in SSD-land commodity drives versus normal drives don't change much since you're trapped by the number of writes. -- My point about disk sizes is he says something unequivocally: that only i2s have SSDs. He doesn't qualify it. And that's bad netizenship and wrong or whatever. :)

Not that AWS isn't great overall--just that compared to https://cloudplatform.googleblog.com/2016/02/Compute-Engine-... they are not competive for low cost and high performance storage. (And their internal networking isn't so great either for smaller instances)

I don't disagree for those use cases that's very impressive. I just don't require a lot of local disk space.

> 10. "The discount is small." - It's like 40-70%, he quotes 30% on GCE.

It's 26% if you don't pay upfront. So yes, less than 30% that google gives you.

> AWS Premium Support is mandatory

This is simply not true. We only pay 49/mo for support. Sure, you can pay for the top-level which starts at 15k/month but it is not mandatory.

The biggest item I'd mention to you is learn to use spot instances. I doubt Google can beat my 2 cents an hour for m3.large servers.

We do offer preemptible VMs (https://cloud.google.com/preemptible-vms), which for our n1-highcpu-2 (a 2 vcpu Haswell) comes in at 2c/hr; when you toss in our per-minute instead of hourly billing, we're often 2x cheaper and all without any guesswork or bidding.

Disclaimer: I work on Compute Engine, and specifically preemptible VMs.

In all the AWS workloads I've dealt with none was ever suited to spot instances

Huh. Do you not use say EMR? We've got lots of folks running Hadoop/Spark either directly or through Dataproc (https://cloud.google.com/dataproc/).

All of our Monte Carlo finance customers as well as the genomics folks at the Broad Institute seem pretty happy.

If you're saying they're a bad fit for stateful applications like databases and web backends, I agree ;).

We run a bunch of production workloads on them. Just look at the spot pricing history in US-East-1A and 1B for a good idea at the stability.

This is over a year now and only once did I have any sort of interruption and even there it was in only one AZ so of course I had other instances up and able to take the load.

You do know that AWS AZs are different per customer? Your us-east-1a isn't the same as my us-east-1a.

All the test infrastructure can easily go on test if you have auto-scaling. Some percentage of production, you can move to Spot.

spot instances can be interrupted anytime, there is very limited usage out of those.

True, but you receive a two minute warning. Your app should be performing frequent checkpointing, and be ready to wrap up its work in two minutes. That's plenty of time for most workloads to end gracefully and notify your command and control.

https://aws.amazon.com/blogs/aws/new-ec2-spot-instance-termi...

Agreed. On our 8 vcpu and higher instances, you can do ~16 Gbps. Even on AWS you can do upwards of 10 Gbps. So worst case scenario, you stream a ton of data at a recovery box.

More likely, you just checkpoint along the way and save to PD / EBS (and then reattach the disk post preemption) or even do ~X Gbps to GCS / S3 and pickup whenever with a fresh VM.

Disclaimer: I work on Preemptible VMs at Google.

I have not used AWS a LOT, but I have used it some, and did run into issues with throttling CPU and what at least seemed like random times (after some quota of cycles reached or something?). That did not happen on Linode and Digital Ocean. I cannot verify that this type of thing happens for network and larger instances but I believe him if he says it does.

I'm guessing their philosophy is to be more strict so they can predict and ensure they will have available resources to spread around.

For me, AWS has some things like VPCs, S3, DynamoDB, etc. that are hard to get unless you go with something like GCE and not available in VPS provider like Linode and Digital Ocean. So far I have found the smaller simpler providers to be a much better deal and generally to perform better for the same or less money. If I need those other features though then I would have to use an AWS or GCE.

Which instance type were you using when you experienced throttling? This is a documented "feature" on a few of the instances. They give you the ability to burst for a certain amount of time but you can't burst forever.

You'd very often see it on t2 instances. To make things worse, it isn't obvious that throttling is causing things not to work as expected.

Then you have noisy neighbors that affect you.

To me, the difference between GCE and AWS really has to do with "openness": while I do admit GCE machines can be faster and cheaper, you only feel like you are treated somewhat equally as the big corps when using AWS.

For example, Lambda has been released to the general public for quite some time, while on the other hand, Google Cloud Function is still in invite-only phase.

I do love Google, they are building many awesome products(personally Chromebook is my favourite computing device now, if only I don't have the tech requirements that is only doable on a Mac -_-), but when it comes to GCE, it really gives one the sense that they are only aiming at big corporations right now, for small startups or individuals doing side project, AWS is significantly better. For me, this is an advantage even tho I have to sacrifice a little performance

Google Cloud Functions is in Alpha. It's not being held back for "the big corps", in fact most big companies won't touch any product that isn't Generally Available (and therefore backed by an SLA).

It's just a product that's still in development, and changing a lot! They'll be in Beta soon enough, and then anyone can use it. Until then, they're following our usual launch phases (https://cloud.google.com/terms/launch-stages).

If you want to try it out (and provide feedback, and deal with potentially large changes), I'm sure they'd be happy to whitelist you: send me your info (address in my profile).

Disclosure: I work at Google on Compute Engine (but I know the folks on Cloud Functions).

Awesome, thanks for the answer! In that case I will wait a bit(already signed up for the form but didn't got a reply).

The article conveniently ignores things like the number of services available, the ability to find staff who are familiar with the platform, the availability of data centres in your location, the depth/breadth of the available APIs, market share and so on ... Sure, if the only thing that you care about is cost, maybe you should use GCE. Alternatively, if you're really serious about saving money, then perhaps you should buy some second hand servers and host it yourself.

You would think that the author would have learned his lesson after all the bad experiences with the (Amazon) cloud, but no: his solution is to go to another cloud provider. And even then, GCE? If the author requires insight into system performance and the ability to debug, then Joyent is the place to go to, not GCE or anyone else.

I do not understand these people, nor do I purport to: a half-decent, 32GB of memory, 4 TB internal disk, octo core intel system (made by intel) costs around $1,800 USD, that is a one time cost, plus recurring electricity costs. Meanwhile, the author's company pays $1,500 just in monthly reservation costs. What ever happened to "keep it simple, stupid!" UNIX principle?

From the article:

"We have to comply to a few regulations so we have a few dedicated options here and there. It’s 10% on top of the instance price (plus a $1500 fixed monthly fee per region)."

"Paying guarantees a 24h SLA to get a reply to a limit ticket. The free tiers might have to wait for a week (maybe more), being unable to work in the meantime. It is an absurd yet very real reason pushing for premium support."

Not my experience.

Reply on news.ycombinator.com