VMs don't have a mess of instance types, just standard but customizable cpu/ram with any disks and local SSDs attached. Live-migrated, billed to the second, and automatically grouped per cpu/ram increments for billing discounts.
Networking is 2gbps/core upto 16gpbs/instance with low latency and high throughput regardless of zone and doesn't need placement groups. VPCs are globally connected and can be peered and shared easily across projects so that it's maintained in one place across multiple teams. Fast global load balancing with a single IP across many protocols and immediate scaling.
Storage is fast with uncapped bandwidth and has strongly consistent listings. BigQuery, BigTable and lots of other services have "no-ops" so there's no overhead other than getting your work done. Support is also flat rate and not a ripoff percentage.
The major downsides to GCP are the lack of managed services, limited and outdated versions for what they do offer, poor documentation and broken SDKs, and dealing with the opinionated approach of their core in-house products. This usually means as a startup that you are more productive on AWS or Azure because you can get going in a few clicks with a vast ecosystem, while GCP is a better home for larger companies that have everything built and just need strong core offerings with operational simplicity.
We hate Oracle like the next person, but we're locked in. And there's no way we're going to the "Oracle Cloud"
I can understand why this is a no-go for the GP.
This is the cost of lock-in.
Personally I'd rather have the expertise on staff than pay oracle (and microsoft) for this shitty behaviour.
In my experience, most startups just want managed options they can run themselves rather than engaging partners but if that's what you need then AWS will have more companies to offer, although GCP does have qualified partners. I recommend contacting one of the GCP developer advocates either in this thread or on twitter for help, or email me separately and I'll put you in touch.
Also I havent worked with them but https://shinesolutions.com/ has put out plenty of articles and case studies that seems to show they're pretty capable, that might work for you.
For an early stage startup/Indie developer, GCP support model is not appeasing. $100 Per user for development role is unacceptable. (at least to us, the stage where we are now)
If you're very early then you can also try the older support pricing: https://cloud.google.com/support/premium/
If that's still too expensive then you can probably rely on the free support and forums until you have more spend and revenue.
https://cloud.google.com/solutions/migration-center/ is a reasonable jumping off point. Sorry if that didn't come up more clearly.
Edit: I'm being downvoted but it's true, if you need enterprise support you're paying the same percentages you would on any other cloud service.
The role-based support is flat-rate but you're right that the Enterprise tier is $15k or percentage spend. From my experience, most companies are fine with the 1-hour production tier.
I haven't worked with GCP beyond 6 figure scale but I dont think that matters. If you need the 15-min response times and the TAM guidance then that's the fee but at least you can opt-out if you don't.
That is spot on. After a year of research and testing on GCP and AWS, I concluded with the same thing. AWS is much more startup/indie-friendly.
Gcp was always exactly the same perf, on the button exactly.
Azure was all over the place, fast then slow then fast then slow.
Aws was in the middle.
If you want guaranteed perf then go with gcp 100%
So, obviously my experience is based on my use-case so things that are important to me are predictability of resources and consistency of data.
From a resource perspective (GCP:Compute) there seems to be a tendency of setting CPU affinity for certain cores, often in the same NUMA zone on the host. This means my tickrate doesn't get hijacked on certain cores randomly. That's quite nice honestly and it's something that on-prem hyperconverged VM solutions don't manage. The NUMA zone affinity / samezone affects our performance quite drastically as certain applications (such as in-memory databases or gameworlds) allocate large chunks of memory and then do little updates millions of times a second. This means memory bandwidth is very important.
GCP also has live migration of instances which is 100% transparent to the application (even the CPU clock seems to be moved over), unlike my colleagues on AWS who get 24h notifications of host machine maintenances I've never had to "deal" with even a single instance outage since starting up on GCP 1y ago. But I would assume if AWS hasn't moved on solving this problem they will soon.
Things like VPC networking being inter-zone by default is something that just makes sense and solves some headaches that I hear other teams having.
From a storage performance perspective, on AWS we were hard-capped to 5GBps per instance to S3, such a hard limit does not exist for GCP instances to cloud storage so we're able to make use of it. (in fact, I can often saturate my 10GBit interface limit even to non-google instances). A n AWS sales rep that this hard-limit can't be removed no matter how much we pay.
I can't attest to the difference in APIs, since I use terraform and that abstracts away anything I would be using.
Regarding storage again; we had reps from AWS talking to us at length and they would not guarantee that fsync() would be honoured on elastic storage. They stated that "it should never be needed" but honestly I'm not comfortable with that answer. I mean, my bare-metal database instances haven't gone down in 3 years but it doesn't mean I'm going to start decreasing the consistency for raw throughput.
There's a lot more but these are the most important things that I can think of right now.
However it may depend on instance type and other factors. This complexity is a problem with AWS.
Like the parent said; we used to receive these notices quite frequently and the amount of time varies wildly. I should ask the teams who currently/primarily use AWS perhaps the situation has changed?
But it was not uncommon to have 24hrs of notice, especially for instances in US-EAST2 for some reason.
"We have important news about your account (AWS Account ID: XXXX). EC2 has detected degradation of the underlying hardware hosting your Amazon EC2 instance (instance-ID: i-XXXX) in the us-east-1 region. Due to this degradation, your instance could already be unreachable. After 2018-12-28 19:00 UTC your instance, which has an EBS volume as the root device, will be stopped."
The stop date is 2 weeks after the email is sent, but as the message states, the instance is likely already dead. It was already dead in this case. GCP will live migrate your instance to another host.
Live migrations is amazing.
Now if only Google paid more attention to certain features of their GLB... amazing tech with showstopper bugs means amazing tech I can’t use. (Also means it’s not so amazing)
In hyper converged environments the storage and the compute live together on a service mesh and thus some processing is done on the hypervisor to manage storage, those processes don't have an affinity (in the case of vmware for example) so a guest VM core gets suspended for a couple of hundred clock ticks sometimes.
This is a super challenging problem for general purpose virtualization stacks. EC2 has been working to avoid any guest VM interruptions. With the Nitro hypervisor, there are no management tasks that share CPUs with guest workloads.
See more here, including data from a real customer with real-time requirements running on EC2 under the Nitro hypervisor: https://youtu.be/e8DVmwj3OEs?t=1796
This is a great tool to help uncover if CPU and memory NUMA within a VM aligns to the physical host or not:
https://www.cl.cam.ac.uk/research/srg/netos/projects/ipc-ben...
Paper from the USENIX 2012 conference: http://anil.recoil.org/papers/drafts/2012-usenix-ipc-draft1....
EC2 instances have been NUMA optimized since we launched our CC1 instances in 2010. I encourage you to try ipc-bench on your cloud provider of choice, if CPU and NUMA affinity is important to your workload.
https://aws.amazon.com/blogs/aws/the-floodgates-are-open-inc...
The fixed performance EC2 instances (in contrast to instances with burstable performance) all have dedicated physical CPU allocations, and 1:1 affinity from vCPU to the underlying CPU. Similarly, memory allocations are fixed, pre-allocated, and are aligned to the NUMA properties of the host system. When combined with both our Xen hypervisor and Nitro hypervisor, this provides practically all of the performance of the host system with regard to memory latency and bandwidth.
See: https://twitter.com/_msw_/status/1045032259189760000
With our latest 25 Gbps and 100 Gbps capable instances, EC2 to S3 traffic can make use of all of the bandwith provided to the instance.
Please send me more information about the interaction you had on the EBS side, msw at amazon dot com. All writes to EBS are durability recorded to nonvolatile storage before the write is acknowledged to the operating system running in the EC2 instance. fsync() does not necessarily force unit access, and explicit FUA / flush / barriers are not required for data durability (unlike some storage devices or cloud storage systems). Perhaps there was confusion about the question that was asked.
GCP has much more modern APIs (they got to do "right" the first time around, while AWS had to learn from their "mistakes")
GCP is less mature with documentation, but still excellent (AWS is first class). I've found bugs, and others I know have run into straight up wrong docs - a quick message to support yielded the correct information and docs update within the week.
GCP is far more "opinionated" about how you should run a service. AWS is opinionated as well, but less so. What this means is that while GCP will sell you resources in traditional way, to go GCP native or from scratch really requires buying into the GCP "way" of doing things more than AWS does. Basically, K8S or go home.
We use GCP and I enjoy it quite a bit. I have extensive AWS experience as well, but no strong preference.