Could you elaborate a little on this part? Our team is looking at ECS/Fargate as a possible container solution and I am curious what you felt was missing from it.
> Could you elaborate a little on this part? Our team is looking at ECS/Fargate as a possible container solution and I am curious what you felt was missing from it.
Some typical examples:
- Kubernetes allows you to run a monitoring container on every single node using something called a "DaemonSet". On ECS, you'll have to build all your monitoring tools into your base image, or use cloud-init to spawn an ECS task on each machine.
- You're probably going to end up writing a bunch of scripts to generate ECS task definition JSON and to update running services, and you'll need to integrate this into your CI system somehow. With Kubernetes, you can get away with "kubectl apply -f" for a fairly long time.
- Kubernetes makes it relatively easy to allocate and manage persistent disk volumes. I wouldn't necessarily use them for a production database, but they're great for smaller things.
- Kubernetes has autoscaling support, plus the ability to control which containers run on which types of servers.
- Kubernetes has basic secret management built-in. It's nothing as nice as Vault, but it's good enough to get started.
- Kubernetes has support for a whole bunch of useful minor things that would typically wind up as Terraform scripts on AWS.
And so on. None of these is very hard individually, but there's a ton of things like this. So we're slowly migrating pieces of ECS infrastructure over to Kubernetes so that we can stop reinventing so many wheels.
Again, for those things which can run on Heroku, either choice is overkill. And I have to admit that ECS is very reliable at the things it does. If you do decide to look at Kubernetes, I highly recommend skimming the O'Reilly books, which provide a solid overview of how it all fits together.
- https://aws.amazon.com/about-aws/whats-new/2018/06/amazon-ec...
- https://docs.aws.amazon.com/AmazonECS/latest/developerguide/...
- ECS runs on ASG / EC2 so there is auto scalling
- https://aws.amazon.com/blogs/compute/managing-secrets-for-am...
I think you don't understand all the glue between AWS services, off course ECS doesn't have everything that's why there is EKS, but all the thing above exists on ECS.
I've looked at various guides for setting up cluster autoscaling on ECS, and so far, everything I've found looks far more complicated than Kubernetes cluster autoscaling on Google. Is there a nice guide?
https://docs.aws.amazon.com/AmazonECS/latest/developerguide/...
But I'm using ECS and Fargate right now (orchestrated with Terraform). https://rivethealth.com
Probably the biggest thing I would like is native secrets management. "Security groups" are limited as it depends on having an AWS interface per container.
ANd I don't know how much longer our time on Fargate will last. It's expensive, but more significantly there are limitations in not having access to the host. Having to install an SSH server in every docker container to be able to debug (simple things, like top) is annoying.