Hacker News Reader

Comment by panarky | Hacker News Reader

"I guess you are essentially confirming that the instance maintenance was likely to be the reason for the major change to cpu usage, and that the only solution now is to change instance type."

"Of course I am not entirely happy about this. I bought a 3-year reserved instance 2 years ago, and now have to hope I can sell the remaining year for a reasonable amount (which may be a stretch given that I am apparently using legacy instance type), and then purchase new reserved instance after upgrading."

View on HN

Sounds like all instance types can now be run in HVM mode (news to me, but the AWS tech in the thread made the claim). Converting to HVM is annoying but ultimately not that hard or risky. Since HVM mode appears to resolve much of the performance issue, this seems like a reasonable workaround.

>Sounds like all instance types can now be run in HVM mode (news to me, but the AWS tech in the thread made the claim)

https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/virtuali...

"All instance types support HVM AMIs."

This is a recent change.

(I work at AWS, not directly involved with this change and not speaking in any sort of official capacity, but can confirm you can use HVM on m1, etc, now)

Or migrate to google since they don’t have a broken discounting billing system

I'll stick with the vastly superior offering and swallow the occasional hiccup, but thanks.

"Vastly superior" could use some qualification. I find the gcloud tooling to he superior to awscli.

I would describe much of AWS and the supporting tooling as "ugly", sometimes hideously ugly, but the services tend to work well.

They clearly prioritize making things work over making them pleasant to use.

AWS Lambda Mapping Templates and S3 Bucket Policy are two things that come to mind as hideously ugly. But they work!

I have used Amazon for many years happily, but I'm struggling to understand what you consider to be vastly superior? Google has some very solid product offerings.

Google Container Engine for instance is amazing. Granted Amazon has recently launched a couple of products that offer similar features, but as of mid last year running containers on Google felt like it was ahead of Amazon.

running containers on google compute or ec2 has always been the same.

gke is great, however. if you need to run kubernetes in production, today, i would not even slightly hesitate to yell to the corners of the earth to run gke. it makes me excited for eks!

i would also say that the new c5 instance type w/ ena is extremely nice. if they are unaffected by this, they're probably the fastest cpu-based instances you can get anywhere (admittedly i haven't checked... really anything else, but they're fast).

One thing is s3/gcs latency. On my last job we had a Go server running on ec2/gce and do some occasional read from s3/gcs (same, single region). We used to be running on ec2+s3 but later switched to gce+gcs for cost reasons (gcp only costs ~60% of aws' cost for our setup). But the thing we noticed was that when we were on aws reading ~10KB single file from s3 usually takes <100ms, occasionally hit 100ms+ but rarely hit >200ms. On gcp that's usually >100ms, sometimes hikes to >1s and rarely goes <100ms.

Disclosure: I work on Google Cloud.

Yeah, the underlying system is optimized for throughput (though we're working on small file latency). AWS clearly has a small file optimization and caching that we don't (yet).

I often point people at Zach Bjornson's blogpost [1] since he compared all three providers and is a neutral third party.

[1] http://blog.zachbjornson.com/2015/12/29/cloud-storage-perfor...

I'd encourage you to take a(nother) look at Google Cloud. I've been using it for the past few months and have been very impressed.

There are a few things missing compared to AWS, but all the main stuff is there (and to my mind a lot more logical and integrated). Some key features blow AWS out of the water.

They've come a long way in the past few years.

Having used both, I'd say that AWS has a lot more "stuff" (services) and they tend to be more scalable and practical. Google tends to create fewer services that are (generally) rock-solid with some gems like BigQuery.

Fun Anecdote time:

6 months ago I needed 100 GPU machines for a bunch of batch processing.

I called my Google rep and contacted AWS and Azure at the same time.

Google struggled to give me enough instances and was only able to provide 10 at the time, with more on the way in other regions...

However, their GCE wouldn't let me create an autoscaling group with GPU instances. Their tooling simply didn't support it. After much hassling their support centers, my rep informed me they simply didn't support it yet, and he recommended I use AWS.

Azure never responded to my inquiries. Their UI and tooling is pretty lame too, and the service is expensive.

AWS increased my quota quickly and it worked great. Their support tends to be responsive if brief and biased towards making the problem go away by giving you what you want.

Disclosure: I work on Google Cloud.

If you had no history, our folks were probably being cautious in granting quota (you may have noticed that cryptocurrency has been going crazy). Feel free to request another increase, but I understand if you're not interested.

I don't understand your managed instance group thing though... Is that perhaps because you tried to create a regional instance group, and we only had GPUs in particular zones? (I saw a bug like that, conveyed an answer, and never heard back).

I got 10 or so in one region, 10 or so in another region, and 32 or so in asia. It was all spread out. It really seemed like a capacity issue. This was in March 2017.

In fairness, Google was pretty transparent about the whole thing. It had a lot more personal touch than the other providers. Google set me up on a call with some of their engineers to discuss autoscaling. But at some point the communication broke down a bit or maybe interest was lost in helping with my issue. Concluding with the realization that they didn't support managed instance groups with GPUs yet, and that was that.

Please please please convince someone to invest in your documentation. Your services could be great but people will never know because the documentation is either poor, spread out across multiple seemingly unrelated pages, or non-existent.

Already in process, but we hear you. The most helpful thing is to actually use the Send Feedback features throughout the docs and console (you need to be signed in) to highlight things that you find unclear. Often the initial documentation is written by the product manager in concert with our docs team, but it's easy to gloss over "obvious" things when you know it in your head.

Google offer exactly the same reservations mechanism

https://cloud.google.com/compute/docs/instances/signing-up-c...

"Discounts apply to the aggregate number of vCPUs or memory within a region so they are not affected by changes to your instance's machine type."

Sure, if you're happy with global outages.

https://status.cloud.google.com/incident/compute/17003 - On Monday 30 January 2017, newly created Google Compute Engine instances, Cloud VPNs and network load balancers were unavailable for a duration of 2 hours 8 minutes.

https://status.cloud.google.com/incident/compute/16007 - On Monday, 11 April, 2016, Google Compute Engine instances in all regions lost external connectivity for a total of 18 minutes, from 19:09 to 19:27 Pacific Time.

Or, complete outages for a region: https://status.cloud.google.com/incident/compute/17008 - On Thursday 8 June 2017, from 08:24 to 09:26 US/Pacific Time, datacenters in the asia-northeast1 region experienced a loss of network connectivity for a total of 62 minutes.

AWS guarantees regional independence for all services, and EC2 guarantees availability-zone independence.

Reply on news.ycombinator.com