Hacker News Reader

Comment by yebyen | Hacker News Reader

> You pay $0.20 per hour for the EKS Control Plane, and usual EC2, EBS, and Load Balancing prices for resources that run in your account

Objectively that's not bad for HA masters in separate AZs, but I think for those who have been using Kubernetes on the Google cloud it's certainly going to have a hard time competing with "you don't pay anything for HA masters at all."

> ingress

from the Twitch stream, it sounds like they have not worked out ingress with ALBs. No mention of Ingress on the announcement page. Twitch stream is here[1]. (it's over now) [2]

This is going to be super expensive to use in the near term.

Just now Nishi Davidson has just mentioned ingress/ALB is a focus of the sig-aws, so hopefully we can expect another announcement soon.

[1]: https://www.twitch.tv/aws

[2]: https://www.twitch.tv/videos/269722012

View on HN

HA seems like a poor choice as a default. I don't want to pay ~$150/month for dev and staging clusters, and I certainly don't need them to be HA.

IIUC, Google/GKE and Microsoft/AKS are single-zone by default... free for masters, but not free for HA.

Google and Amazon are enterprise services. If you consider $150 too expensive, you are not part of the target customers.

The Google GKE offering can operate with a single g1-small worker.

That's $0.0257/hr or $0.0070/hr preemptible. $5/mo minimum*

This is not even the same ballpark.

*(to be fair, the $5/mo minimum does not include a private network and a load balancer, and you will almost definitely need both in order to actually do anything with a cluster.)

You're thinking like a human. There are plenty of organizations where $5/mo and $150/mo are essentially indistinguishable and would not affect decision making in any way.

That's $150/mo before you even launch a single worker tho.

I don't know about you, but I'd personally rather pay the $5 if I don't need a serious cluster yet. How much in actual absolute dollars do you think this is going to cost you, just in order to figure out exactly how it works, and what other AWS widgets you'll also need to pay for in order to use it effectively?

I am a human, basically looking for a stepping stone on some actual cloud to land on that is comparable to Minikube, and this sure isn't it.

I'm pretty clearly in the minority with my opinion here, and I recognize that.

Amazon has made a substantial investment in Kubernetes here and that is nothing to shrug at. They are entitled (and justified) to recoup it. It's fair to say that K8S is not for lightweights, too.

AWS provides a fairly generous free tier and they are also justified to extract some payment from every user of their services that derives some non-negligible value. If you're using Kubernetes and you're not getting any value out of it, I think it's also fair to say that you're doing it wrong.

You're not in the minority. It's well understood that there are amateurs trying to use the expensive clouds and shave off every penny, while the main customers are enterprise customers who couldn't notice a thousand dollars more or less.

It's just pointless for small users to argue about AWS costs. The service is not meant for them and it cannot fulfill their requirements.

You're obviously no slouch, and I respect your opinion, but I think Amazon is all about the long tail and without speculating too much about what they are thinking, I'll say they might disagree. They are obviously smart to go for the big money dollars first, but everything I know about "Long Tail" says you're wrong.

I absolutely agree with everything you said, if you only added "at this time" to the end of that last sentence.

What long tail do you mean? I would say the small tech companies with tens of developers and tens of servers are the long tail for AWS.

The service is too expensive and too complicated to be used by very small companies or individual users. There are also more appropriate competitors in that space.

You told me you thought I'm not in the minority, and others have come forward to agree. I think we are the long tail.

It is exactly as you say, companies that don't want to spend more than they absolutely need to on infrastructure. Shaving pennies to save money. We are each too small to make any significant money on us, taken individually. That's what makes us the long tail.

Do you think in 6 months, you will be able to get a non-HA Kubernetes master on Amazon's free tier? Almost surely within 12 months. At that time, they will have addressed the long tail. Today's announcement is not for us. It's for large enterprise customers that settled on Kubernetes (and 57% of Kubernetes is already on AWS, so many of them are likely not new customers.) We're both right, from opposite perspectives.

Sorry, there might be a misunderstanding. I meant that you were not in the minority, in terms of user count. There are people trying out AWS/Google, riding on the free tier, or just running a small site for the experience. People spending less than a hundred dollars a month could very well be the majority of the user count.

It doesn't mean that they bring any significant revenues or that the distribution follows a long tail. I would actually bet that amateur users are not forming a long tail.

I believe the whole kubernetes is designed with the assumption that you are an enterprise customer.

I serve a couple small websites from a very very small GKE cluster that costs well under $100/month. I don't worry about OS updates or Kubernetes upgrades. I don't worry about it going down.

I don't think my needs are that unique.

Not unique, but certainly low value to Amazon in terms of expected revenue.

And plenty of startups that choose between AWS and GCP at a point when those numbers are plenty distinguishable, even though later on they'll become those organizations.

And they will all die pretty quickly for running their infrastructure on a pre emptible instance to save a few dollars.

By the way, startups get $100k free credits on each of the major clouds.

You can afford ~5 preemptible instances for the cost of one full-time instance. I think you are severely overestimating the harm that is done by preempting/terminating instances in a prod Kubernetes cluster that is capably configured for full HA and distributed across zones. That's what it's made for – nodes are as disposable as pods. They may come and go as they please, and they'll be replaced as needed by the self-healing nature of the cluster and the scaling group.

(Why save a few dollars when you can get extra capacity instead? Especially when you have $100k of someone else's money to burn... so, for real though, anyone who has actually tried this configuration can chime in and confirm that it works as well as I imagine.)

I'm pretty sure that periodically shutting down some nodes is a boon for cluster utilization, too. One of the things that Kubernetes does not do on its own, is load rebalancing. You can configure the autoscaler to recognize when nodes are overprovisioned, and let it drain a few and shut them down... or you can let the preemptible nature of (some/all of your) nodes do it for you.

(Why not both? Some nodes are getting killed, or you're paying for resources you don't use, so... one way or another.)

Your message obviously implies that you have no experience with what you are suggesting, hence why you don't realize the downsides.

"No experience" is a stretch. I'll concede, no significant Prod experience to speak of.

I got introduced to the Kubernetes ecosystem when Deis Workflow was rebuilt for it. Today, I'm a core contributor on the team that is building the fork of that project.

(It is a side thing for me. I'm very interested but I'm not spending hundreds on infrastructure every month. I have an infrastructure team at $dayJob, and they are not doing K8S at all, if you are having a hard time understanding how exactly I got here.)

You can mix & match preemtibles with normal VMs in a cluster. Aka 10% of your cluster's nodes can be preemtible, and 90% normal VMs.

This is great cost cutting measure for services where loosing some percentage of compute only induces lag vs taking down the whole service.

How? How can I get that free $100k credit

Here's the scoop[1]

It sounds like you must have taken some investment to qualify. But I believe my friend received the credit, and I have no idea if his business has received any investments or participates in any accelerator program.

It is a program (several different programs actually), you apply for it. This looks like a good place to start.[2]

If you don't think you qualify after reading that, might be worth talking to Arun Gupta anyway. He is the Principal Open Source Technologist. He was all over the Reddit thread answering questions about EKS today.[3] Super classy.

It looks like they offer $1000-$100000 depending on how fast you want to spend it and what stage you're at.[4] "AWS Activate - Portfolio Plus" is the program talked about, that offers $100k credit that expires in a year. If you don't qualify for that, you probably qualify for either the $1000 or the $15000.

[1]: https://www.quora.com/How-can-I-get-100-000-credit-on-Amazon...

[2]: https://aws.amazon.com/startups/

[3]: https://www.reddit.com/r/aws/comments/8osr8c/amazon_eks_is_n...

[4]: https://aws.amazon.com/activate/

I second this. This means you don't have to setup and manage those master nodes. AWS will do this for you. A developer fixing an issue with your self-managed master nodes will be much more expensive than 150$/month.

As an individual this may look expensive, but as a company this is much cheaper than paying the loan of a developer.

I suspect you missed the point earlier: GKE offers a free managed master, even if your cluster is a single node costing you $5/month.

Disclaimer: I work at GCP, but not on GKE.

Yes it is nice for experiments, but if you already invest thousands of dollars a month for cloud resources, an additional 150$ are for sure not a problem.

Exactly. We spend hundreds of thousands a month with AWS.

$144/month is nothing if it means someone else can manage some of the parts of Kubernetes.

You can set up the nginx controller with NodePorts to get rid of the load balancer if it's just a side-project cluster. (Otherwise load balancers will be the majority of your cost)

Thanks for that!

GKE is free for HA as well: https://cloudplatform.googleblog.com/2018/06/Regional-cluste...

Looks like they want companies to pay to avoid the platform lock-in that k8s helps you avoid...

Until recently gke was charging 0.15/ hour too, so aws is just a little behind

https://cloudplatform.googleblog.com/2017/11/Cutting-Cluster...

Not only recently, but actually this was the cost until yesterday for GKE with HA masters distributed across AZs[1]!

I only heard that this announcement from AWS was likely coming this morning, over here in the thread about upgrading GKE clusters[2]. Given how long we've waited since the announcement at November's re:Invent, there's honestly not a lot that seems terribly rushed about this news, but I bet that AWS really would have liked to have Ingress controllers that are integrated with ALB ready for this announcement to go with their CNI plugins.

"Why am I waiting for Amazon to get out of preview when Google's been giving it away this whole time?" For AWS customers that aren't locked in, that seems like a reasonable train of thought.

[1]: https://cloudplatform.googleblog.com/2018/06/Regional-cluste...

[2]: https://news.ycombinator.com/item?id=17236794

I look at this more as a product catering to existing AWS customers (and not have them move to GC) rather than trying to lure GC customers over to AWS.

Ouch, no option to schedule pods onto the control plane for small clusters?

AFAIK, none of the big three cloud providers (GKE,AKS,EKS) lets you do this.

Yeah, none of the rest of the big three are asking you to pay anything for those masters at this point, though. "They're your nodes, you paid for them" seems like a reasonable position to take here.

> "They're your nodes, you paid for them" seems like a reasonable position to take here.

If that were a reasonable position, you'd think AWS RDS and Google Cloud SQL would give you superuser access to your database instances to do things like installing Postgres extensions.

It seems a lot of people are happy to pay for instances they can't even SSH into. :/

Sure, I guess so, it's a value add. Your engineers can't mess anything up if they have no access at all.

You underestimate our engineers :-)

I don't even want to SSH to instances that I control and have provisioned, why would I want to SSH into instances managed by AWS?

Unless the point of your server is to provide SSH (you are using it as a development box, maybe?), having to SSH means that you are lacking in the tooling department.

At work we are guilty of that. We are actively trying to improve on this.

In this case, it's not so much "SSH" as the ability to install files, as root, onto the server. For RDS/Cloud SQL, the inability to do that both restricts you from installing your own extensions; and restricts you from being able to use the Postgres COPY command with local/network-mounted (rather than network-streamed-to-STDIN) CSV files, majorly increasing the overhead of the operation and preventing parallelism.

Agree, really confused why AWS didn't follow Azure and GCP here.

(Disclaimer: I work for GCP, sorry for not making that clear in the parent comment)

When you deploy Kubernetes "add-ons" like Helm's tillerd or https://github.com/jetstack/cert-manager/, the active containers of those get deployed to the management node, no?

No.

Reply on news.ycombinator.com