How Far Out Is AWS Fargate?
read.iopipe.com
read.iopipe.com
take elastic beanstalk, its meant to be a competitor to heroku(thats how it was pitched to us in $big company), but my word it misses the mark.
Lambda and API gateway is a massive faff to setup manually, but serverless and zappa make it really really simple to do. That and its cheapness is why its caught on, its fast(to iterate), simple enough and super cheap.
Fargate is kinda aimed at people who have out grown lambda, but then you will be evaluation the whole hosting ecosystem. But if you are evaluating hosted K8s, I don't know why you wouldn't just plump for GKE.
Mind you, avoiding K8s and sticing with lambda + ECS for long running stuff is far simpler to understand, even if updating it with CF is a massive ball ache.
actually with GKE you can run your "AppEngine" apps directly on GKE: https://cloud.google.com/appengine/docs/flexible/python/run-...
Better yet there is the "next gen" stuff from AppEngine (Cloud Run) which is like Fargate, but can ALSO run on your own GKE: https://cloud.google.com/run/
the only thing which I still think is way more expensive/worse than the aws counterpart is RDS. (Google has a hosted db service, but that feels worse than rds)
Its definitely more involved than just a git push, you'll have to deal with RDS for databases for example, and get credentials for that into your application. Having said that I think the extra effort needed for ECS is worth it, you end up paying less for the resources your consuming (in some cases significantly less), and you're sitting within AWS so integrating with other products they provide is a lot smoother.
Now that I’m knee deep in the world of Kubernetes I actually find it easier to use and run. CF is an absolute abomination in comparison. God help you if you need to rollback in CF or debug something. I have none of those problems in Kubernetes.
Kubeless is a lambda replacement that is easy to setup and run. Apache Openwhisk is another alternative.
Why would I ever want to lock my company down to a specific vendor and their tooling when I don’t have to? Eventually cloud providers will need to contend with the idea that most of the stuff they’re doing will be replicated in Kubernetes. Kubeless and projects like Cert-Manger are perfect examples.
However for a small team, I don't think its a great fit, yet.
Salesforce purchased Heroku in 2010 for $212MM in cash before Beanstalk was even announced.
When Fargate was released I was very curious about it as it seemed like AWS was moving up the ladder towards PaaS and I wanted to know how it compares to Elastic Beanstalk and also Heroku.
I started going through this AWS written tutorial:
https://aws.amazon.com/getting-started/projects/build-modern...
The app they use as an example is of course meant to demonstrate how a plethora of AWS services can work together.
I still couldn’t help but be surprised at the sheer number of different things I had to learn and configure, sometimes involving editing yaml.
For long time AWS users: is that tutorial representative of how you really build apps in AWS?
I legitimately believe there's a checklist at the bottom of every new AWS service launch, and among their internal requirements includes a line like "Is this product so amazingly incomprehensible that customers will take weeks to integrate with it and be forced to upgrade to a technical support plan?"
The closest any major cloud player has gotten is App Engine. And its pretty close; it has issues, and some inherent complexity, but generally I recommend any new startups look to either App Engine or Heroku for hosting. Avoid AWS like the plague.
Autoscaling: Heroku - 2017, AWS - 2009.
Heroku also really needs to wake up to the world outside the US and Europe. Just enabling the other regions (for general use, not the super expensive private spaces) would be a step on the right direction.
I've often assumed that no one's KPIs are tied to making things more usable or docs more readable, and everyone's KPIs are tied to shipping new features.
In general I'd say that AWS takes the approach of offering a lot of configurability if you need to tweak things. You can go into that low level JSON or YAML if you want, but we are also providing higher level tools like AWS CDK which do the work for you, and let you just focus on building your app with just a few lines of code.
I'd say that the referenced tutorial is aimed at being "mid level". It abstracts some things but still shows a lot of the lower level settings you can tweak directly. If you want a more abstracted, higher level getting started experience you should start with a tutorial for a higher level tool like AWS Cloud Development Kit, and use that to automatically setup and deploy to AWS Fargate under the hood.
If you (or anyone else reading this) have any specific feedback on things you want to see made more usable, or docs that aren't readable feel free to email me using the address in my HN profile.
Everything in the Architecture Diagram of Module 2C are definitely necessary pieces of modern software development, you could easily replace as follows:
AWS Cloud9 --> VSCode
AWS CodeCommit --> GitHub
AWS CodePipeline + AWS CodeBuild --> CircleCI
So in any case you are configuring _something_.
Most people don't write good tutorials. Often what they're trying to teach you is ambiguous, they don't clearly define what the dependencies are, and they don't provide clear steps. AWS's docs are not great, but same goes for everybody else.
AWS is complex just enough for production infra requirements. They don't make some stuff up just for fun :) Heroku is more for prototyping or personal blogs, imho.
I agree Heroku has the perception of being only for prototyping projects or small projects. It does work great for those.
But also, there are lots of large companies (with non-trivial and large scale deployments) using Heroku, such as Macy's and Toyota and many others I can't mention without their permission.
I don't want to hear that 'it's a bit like kubernetes but not quite'. I want to try it inside 30 seconds and see for myself.
So let me get this straight, you want to try something as complicated as Kubernetes, inside of 30 seconds?
I'm thinking you might have some unrealistic expectations.
I would love to see the same done for Kubernetes. What I want is a kubeconfig file that links me to a paid account somewhere, and whenever I run `kubectl apply -f foo.yml`, I pay by the millisecond for whatever resources get created. Zero ops for me the customer, and all the complexity will be on the side of the company offering this service.
okteto.com
I'm not sure this is relevant, but it's a great example of a service that has a tutorial, that covers multiple use cases, and lasts less than about 2 minutes, leaving you with a pretty clear understanding of what else you're meant to do.
This is how all elevator demos should be.
Since it was already easy to get a cloud machine on the Internet!
Have you tried digitalocean's kubernetes offering? It takes about 10 clicks to create a cluster and download a config.
Two or three clicks to kubeconfig, then your cluster is deleted in about 4 days. I call it Hephynator and it's not open source yet, but I would definitely consider it. (This model works for me, because I received Open Source credits from DO. :thanks:)
I don't know how much that helps, but DigitalOcean's built-in interface to creating clusters is about that easy. It's nicer than my stripped-down version. Things like "how long until your cluster is ready" -- I didn't take care of that in my DropletKit client, but DOK8s does in their web interface.
My driver to build this little widget was the fact that their Kubeconfig files that are issued by DigitalOcean's interface by default expire after 7 days, so I either needed a way to be sure that my OSS contributors who I hand these clusters out to, could get another kubeconfig when it expired, but not wait for me... or, their cluster would not live as long as the expiration date, which seemed to be a more reasonable economy-driven decision.
I decided to make the clusters last 4 days and then delete themselves. It was a fun project, and now we can use it to make more fun Open Source.
I should open source it. It's a very simple rails app. It does exactly what you describe, I just push "Create cluster" and then confirm some parameters, then get a "Download Kubeconfig" button which is the last step where you have to interact with anyone other than K8s API. It needs to be made pretty, before I'd consider publishing it. But for now, it does the job and my team is using it fruitfully :)
EKS and GKE are the "managed" k8 solutions offered by Amazon and Google. Not sure how much of the management they do here, never researched the subject, the advantage being that your automation that uses the k8 "language", can work with both. I repeat, I don't know the low level details, eg how do you get some resources in such a cluster.
ECS is the pure AWS alternative for K8. Eg, is a software solution to manage docker containers. Is not compatible "language" wise with K8 tools afaik. But, surprise surprise, is very well integrated with a host of AWS services.
ECS can work with two different types of resources, EC2 clusters and Fargate clusters.
EC2 Clusters are sets of EC2 instances, you start them in whatever way you want, make them as big or small as you want, you just need to run the ECS Agent on them. If you use the Amazon AMI optimised for ECS, this is trivial. The drawback is that you need to manage these instances, software update, you need to start new ones if you need more capacity, stop them when you are wasting resource etc. Obviously, AWS has a bunch of other services you can use to make this easier (CloudFormation etc). The advantage is that you can customise the AMI if you need to and you can leave data behind, after a container terminated. For example, all my tasks in this particular ECS Cluster need access to the same S3 bucket of data. So I'm precaching it on the machines and then simply mount a volume in each docker container that needs that.
A Fargate cluster takes away all of this complexity, you just need to specify a name. Then you tell ECS to start tasks/services (services are just tasks that need to run in perpetuity) on the Fargate cluster and the thing will scale up and down, and scale your price up and down, as you need it. Two drawbacks: you can't customise the underlying host and you can't have persistent data left on the host after a container exits. Therefore, the project I'm working on atm would end up costing more to run, since I can't cache the s3 data and every container that needs that data will end up re-reading it from s3.
Best I could do :)
/edit: GKS -> GKE
I have a different question, if you don't mind.
Under what conditions or use cases, do you think it may make sense to migrate from ec2 to ecs? What are the advantages or disadvantages that one needs to take into consideration?
Thanks
I guess what I am asking is that what points should I take care of or give thought to if I decide to migrate from ec2 instances to ec2 instances inside of ecs?
Does it clarify now?
-----
I have just a very very shallow knowledge about these two services, so I'm not going to comment on them. Also, to make sure there are no missunderstandings, clusters of EC2 Instances and Fargate clusters are the things you put stuff in, like barrels, ecs is the tool you use to get water out of the river and put it in the barrels.
At this point, I don't really see any benefit on running 3rd party apps, like I assume logstash and the other one are for you, directly on the ec2 instance.
As a note, I would not run my own db server/cluster or redis or what have you, if I can avoid it. I would pay for it (and do atm, we are using RDS dbs).
The question is if you can benefit from using ECS to start docker containers or not.
You don't need ECS to run a docker container on an ec2 instance. All you need to do is ssh on the instance, install docker, docker pull and docker run. There, assuming you are building your own docker images with all the configs required, this is all it takes to start a new instance of your app. You can even automate some of this with a Launch Template. Two big advantages of doing this: fairly simple setup and might get cheap - if you prepay for your instance, say for 1 year or more, you can get huge discounts.
There is a drawback to this method, but it depends on your use case: you can't react quickly on spikes. The way to increase capacity, for this setup, is to simply replace the ec2 instance with a bigger one. Then put back the smaller one. Since you have to manually do this, I think is fairly obvious why this is a drawback.
So far, the assumption is that your app can only scale vertically, eg to get moar power you need a bigger server. There is the case when you can scale horizontally, eg if you need more power, you just spin up another instace of the same app and share the load.
If you can do this, eg scale horizontally - instead of replacing a t2.medium with a t2.2xlarge, you simply start 3 more t2.medium and all 4 can cope with the load, then shut the extra 3 off when the spike is over (I have no idea if 4 x t2.mediums can do the job of a t2.2xlarge, just giving an example). You can manually do this or automate it via autoscale/CloudFormation or even your own scripts that read crap from CloudWatch and use the api to do things.
You noticed that I didn't say when to use ECS yet and this is because ECS is the tool you use to automate most of the above. The truth is you can live a happy ops guy life without ever touching ECS, or EKS, or GKE, or K8, or chef or ansible or what have you. But knowing how these tools work, this is going to make your life soo much easier.
We are in a very messy infrastructure stage at the moment, which really just revolves around 3 concepts- being able to run code on a lot of machines as "easily" as one machine (kubernetes), being able to support in that context multiple runtime languages consistently (docker), while also protecting the machines the code is running on from malicious and/or poorly behaved code (kubernetes and docker together). So maybe we will soon get back to a place of "simplicity" with a stateless function-oriented programming model that looks a lot like COM/DCOM, hopefully with better ergonomics.
But the benefits aren't really reaped by the app developers - they are reaped by operations. These technologies save money by reducing operational complexity. This happens by decoupling the infrastructure from the app. For MTS you still had to configure and manage your server or pay someone a lot to do it. I guess we have come full circle but it's probably more flexible and cheaper this time around.
Here is my 2 cents.
Instead of your team managing a fleet of servers with operating operating systems (which includes security patching, user management, log rotation, etc) you move to raw containers running on a self managed server fleet. Now you have (largely) decoupled the app from the infrastructure it's deployed to via the container interface. Remember CodeDeploy scripts? You don't need that any more since devs are delivering immutable images. That code is moved into the docker file.
Ok so you've gotten this far and it's great, but how do I do autoscaling, failover, etc. Well you can have your ops team do a bunch of work to make it happen on your self managed container host or you can run your containers on a managed Kubernetes fleet. Bam, now you have autoscaling, failover, etc. And the best part is that the Kubernetes API is platform-independent so it's much easier to move it to a different cloud. Or you do Fargate here if Kubernetes is too complex for your needs.
But your app is dead simple and you don't care about configuring all this crap. So you ditch Kubernetes (or you jump straight here) and you write serverless. Now the ops work is dead simple: deploy serverless app with 10 configuration options. No autoscaling to manage, way less security to manage, no user accounts to worry about. Just make sure it's written correctly and runs on the specified interpreter.
IMO there is no way that these technologies are going away. They provide way too much convenience. If you're a small startup (maybe no ops specialists?) and need to run a self-managed off the shelf OSS server, do you want to waste time dealing with ssh keys, choosing the OS, configuration, etc? Or "just" grab the container and throw it into your EKS cluster and forget about it (This step will get easier). Need to deploy a stateless app? Throw it into Lambda and don't even worry about EKS.
Disclaimer: Yes each benefit that I mentioned is possible as you go down layers. That's because the higher layers are built on the lower ones ;-) This post is about how much work it takes to go from 0 to live and manage the live system indefinitely.
Yes, all these solutions are gradually converging back to CGI.
I'm going to simplify some of the things, please don't get your pitchfork out if I gloss over some aspects.
Consider a web app, you most likely store your data in database, have some code for the business logic, say php and a bunch of front end stuff, html and js. You set up a web server, say apache, with and interpreter for your business logic code, say mod_php and a db server, mysql, on the same physical server (or virtual server). Your www.something.com is up and running.
Fast forward some time and your business grows. You notice that access to your website is very slow, since there are so many people trying to use your website. You move the db on another physical server to free more resources for the business logic. This doesn't last for long and your website is slow again. You also notice that your db server is not doing that much, yet the app server is being hammered.
You set up another app server, call it ww2.something.com, rename your original one to ww1.something.com and connect both of them to your db server. You also set up a third server, way smaller than the app servers, that on requests to www.something.com, based on some heuristic, will http redirect the user to either ww1. or ww2. [glossing over the db details here]. You see this works and you keep adding wwxxx.something.com. One day, on 20th of December, after you finished deploying ww1337.something.com, you realize you have another problem. How on earth are you going to deploy the brand new version of your webapp, that your team worked hard to finish before Christmas, in time for 25th? And there was a new CVE published, apache ver 1234 is vulnerable to the Grinch exploit. Not to mention that you expect a huge spike in traffic from 25th to 10th of January. So you need to add more servers. You also need to pay for these extra servers all the way to 25th of January and you start to wonder if this is worth it. And ww1256 has a hardware failure. Or was it ws1265?
Some time after, you had a chat with the CEO of this hosting company, he wants you to stop using bare metal and instead, move your stuff on virtual machines and they will make sure that all our vms will run, with an uptime of 99.99%, on their cluster. They also give you some tooling to build virtual machine images, thus a new deploy takes hours now, instead of days. No more worries about hardware failures [glossing over], if you find vulnerabilities, you can react on hours.
With the extra money made by being able to quickly deploy new code, hence develop new features, you hire more people and grow your business even more. One of the new people pokes around for some time and comes up with a list of things she thinks can get better, like your web app has powerusers and the http redirect load balancing makes everybody that happens to end up sharing a server with such an user, terrible miserrable, you should load balance based on request, not session. Most of the features are fairly light, resources wise, just a handfull are heavier. You should offload these requests to another set of bifier, more expensive vms, so while the main app servers wait for this forwarded request to happen, they can asnwer a bunch of the ligher requests. You agree and add the final piece in the puzzle, the application load balancer and move some of the features to a different set of resources.
Hope this clarifies what some of these pieces do. Next, a brief list of real life equivalents for the above:
1) vms: any technology that allows you to have a piece of code run in isolation. VMWare images, Virtual Private Servers, docker containers, this is NOT a new technology.
2) cluster: any group of CPU/Memory/Storage you can use to run vms. AWS Fargate is one, the entire EC2 is another one. An ECS Cluster of EC2 instances is yet another one.
3) tooling: whatever you use to create vms and instruct the cluster to run your vms. ECS is the pure AWS solution. Docker compose is another one. Kubernetes is another one. EKS is the aws offering for managed kubernetes. GKE is the google version.
/edit: GKS -> GKE
That $144 a month is a deal considering it's just the price of a few instances that you'd normally provision anyway if you were rolling your own k8s cluster, and it manages the control plane administration for you. The moment your hand-rolled k8s cluster goes the way of so many horror stories [1] that savings evaporates into hours of expensive engineering labor hours wasted.
[1] https://k8s.af
- Service cluster IP range is not definable, making it difficult to integrate with an existing network topology [1]
- Limited choice of CNIs, e.g. Calico can not be used
- No accessible etcd snapshots for recovery
I tend to avoid EKS and prefer to create most clusters by using kubeadm as the foundation.
Service IP configurability is a very common ask, and as you’ve linked, is on our roadmap along with a slew of other control plane configuration options.
You can delete the AWS VPC CNI DaemonSet and install any CNI plugin you’d like.
EKS regularly backs up etcd and has automatic restore in the case of a failure. Manually restoring to an old snapshot would be quite disruptive. What is your use case, and what would be the interface you’d like to see?
Managing worker nodes is also a huge pain. They recently released a tool that manages some of it, but it doesn't work with older clusters. To do something simple like fix a vulnerability in the linux kernel, you have to make a new CloudFormation stack (involving cutting and pasting a ton of random stuff; the template ID, the AMI ID, etc.), edit several security groups, start the new nodes, make sure they join, drain the old nodes, delete the old stack, edit security groups. Upgrading the Kubernetes version in the cluster is a similar story, especially if you skipped an intermediate version. (That said, incidental changes like adding more nodes is easy. It's a lot rarer than upgrading Linux or the k8s version, however.)
Their load balancers other than Classic don't work every well either. I would prefer to use a NLB for all incoming traffic (terminated by Envoy in the cluster which does the complicated routing) but that apparently results in kube-proxy not working right... and every time you change a set of pods whose selector matches the NLB service, it changes all the IP addresses in DNS, and traffic just stops arriving until the DNS TTL changes. It is very broken and the docs should say "never even think about using this" instead of "it's beta and here are some weird caveats that probably won't dissuade you from using it".
Anyway, my overall impression of EKS is that Jeff Bezos walked into someone's office, said "you're all fired if we don't have some half-ass Kubernetes service in the next 3 months" and walked out. The result is EKS. It's wayyyyyy better than being locked into Amazon's crazy stuff (CloudFormation, etc.) but it's not as good as what other managed k8s providers offer.
If I were to setup k8s again, I would just buy my own servers and self-host everything. Dealing with Things That Can Go Wrong With Servers is much easier than dealing with Things That Can Go Wrong With AWS. (But I already have a datacenter, can get the machines a 100Gbps connection to the Internet for free, and have a team of network engineers to tell me which CNI to use. If I didn't have that I would just use GCP.)
Calico policy can be used with the AWS VPC CNI, but you can remove the default CNI and install Calico or any other CNI plugin you’d like.
EKS is priced competitively, with Amazon's other offerings. I was a part of the chorus of voices saying "wtf Amazon, control planes ought not cost, nobody else is charging for them" but I think this is fairly priced for what it is... it only takes four reserved M4.large instances to eclipse the cost of an EKS cluster, and you will likely want more than that if you are aiming for real High Availability and trying to build it on your own EC2 nodes.
The default configuration of EKS is fully HA, AIUI is built to be resilient to faults like failure of an entire region.
If you don't want to pay for the Kubernetes API, the interop chances for engaging outside support that it gives you, and the associated complexity required to support it and keep pace with the release cadence, then there is also Fargate which is cheaper at low scale, (or my preference, go with the competition who have all agreed to undercut AWS, so why not take advantage as it's clearly favorable to run at least some of those workloads elsewhere.)
And hey, there's even Virtual Kubelet on Fargate if you want to get the best of both worlds, right? Of course that one comes with a big warning, DO NOT run any Production...
It's good to have properly defined, stable, open interface for compute workloads (containers as defined by OCI), so we don't have to lock ourselves into whatever shape of runtime environment our current cloud provides for their flavor of FaaS. So we don't have to learn cloud-provider-specific tooling, getting certifications for handling various tasks at AWS, becoming 2010s variants of Cisco-certified network engineers of the previous age.
But it's even better to have a properly defined, stable, open interface for orchestration too. To be able to run stuff locally. To be able to extend things. To ease the lock-in cloud providers currently have. To be able to actually understand what happens under the hood. And last but not least: to enable rise of open source solutions for higher-level abstractions, like KubeDB.
For a serverless proponent like me it's still too much config.
For a no lock-in proponent like you it's not enough config.
Here's a simple use case: say you have a simple banking website with a frontend and an API and whatever. People log in and check their balances, etc. Typically, this requires requests that take on the order of milliseconds, so lambda works for this [0]. However, every month, you want to go in and calculate what you owe in interest for everyone's account. Say that you have 100,000 users and it takes 0.01 seconds to do each calculation, so this job will take 1,000 seconds, or 16 minutes to run. Lambda functions are automatically killed after 15 seconds, so that won't work.
That's where Fargate comes in. You dockerize your environment and then run a command in Fargate. Now, it runs for 16 minutes for the task itself + 1 or 2 minutes to wind up and then it automatically kills itself when it's done. You pay for the 16 minutes of run time and call it a day.
[0] One issue with lambda/serverless is dealing with cold starts when your application hasn't been used in a while, or you need more concurrent instances running, so it takes some time to "warm up". This is a good reason not to use serverless frameworks in many cases, but I won't get into that.
> One issue with lambda/serverless is... when you need more concurrent instances running...
Lambda can have 1,000-3,000 concurrent burst invocations for most of the popular regions and 500 for others[1]. Are you saying that's not enough?
[0] https://aws.amazon.com/about-aws/whats-new/2018/10/aws-lambd...
[1] https://docs.aws.amazon.com/lambda/latest/dg/scaling.html
Whoops that's what I meant
> Lambda can have 1,000-3,000 concurrent burst invocations for most of the popular regions and 500 for others[1]. Are you saying that's not enough?
No, that's plenty for most use cases. I'm referring to "cold starts". When you haven't used a lambda function in a while, it takes a few seconds for it to get going. But when you hit it a second time, it responds more quickly.
* Sometimes it's not efficient to break a task down or it simply can't be done. * Sometimes it just makes the code hard to maintain if breaking it down into different components is a difficult task
An example I can think of is loading a very large CSV file into a database. Lambda doesn't have enough memory to read the whole file and even if it did, it might take more than 16 minutes to run. I can try to break it out into multiple tasks and it'll probably be faster, but reading that code is (generally) gonna be much harder.
So what are you suggesting here?
It took me all of 4 days to come up with a CI/CD pipeline/production environment using ECS and fargate for the startup I was the cto for.
We are now looking at kuberneties or reserved instances to save some money as we have scaled to angel->seed->series A. Though tbh its looking like the extra cost is worth it given the out of the box 1 click scaling, blue/green deployments and "conatiner as a server" level monitoring and logging.
We have, no joke, Lambdas that kick off Fargate tasks to do "actual work". Why do we have Lambdas? Because that's the integration point AWS has everywhere. "Run code when X happens" on AWS means a Lambda, so that's what we use.
Again, I hope I am just stupid and that there is some obvious way with which I CAN see the logs and someone will point out to me how.
[1]: `command: ["/bin/sh", "-c", "sleep 36000"]`
If your containerized process does start and output to stdout/stderr, but then crashes for some reason you can see those stdout/stderr logs in CloudWatch, via the awslogs logging driver.
In general I'd say the docker engine logs are the wrong place to try to diagnose an app running AWS Fargate. The point of AWS Fargate is to abstract away the Docker daemon, and just run your containers for you. So if the Docker engine itself was failing for some reason that is an issue for the AWS team on call to fix. Your area of responsibility starts with the application code that is running inside your container. The bits outside the container are the responsibility of an AWS employee.
So we do validation in the task definition you supply via API to make sure that things check out before AWS Fargate launches the container. So you shouldn't see a "container configuration" based error because our API catches config issues first and won't let you create a task definition that is invalid. If there was such an error that caused Fargate to construct a bad container run command and fail to launch the container that would be a missing validation or bug inside our code and we'd fix that.
If you do find something like this please reach out to me or open an AWS support ticket for the oncall team to look at, and we'll fix it!
(/s...)
> Under the hood, they share the same virtualization technology called Firecrracker. Firecracker is a KVM-based virtualization layer that creates and manages minimalistic container-like virtual machines called “microVMs”. Firecrracker is still relatively new and was designed from the ground up to address issues identified with the previous generation of AWS Lambda.
I had a Node/Express microservice running in lambda using the lambda proxy integration to send and receive files from/to S3.
I knew in advance that the 6MB request limit for lambda was going to hit us, but it was good enough for what we needed right then.
When it came time, I knew I was going to have to convert to Fargate but I didn’t know anything about Docker or Fargate.
It took less than a day for me to get the API up and running in Fargate following this tutorial.
https://medium.com/@ariklevliber/aws-fargate-from-start-to-f...
And automating it with Cloud Formation
https://github.com/1Strategy/fargate-cloudformation-example
To be fair, I did have experience with the other fiddly bits of AWS and CloudFormation so I knew how to modify the template to work in our own environment.
If anyone has any feedback or advice I'd happily hear it.
Feels like if AWS started with dedicated boxes, they'd now be calling EC2 the "Elastic VM Service Fargate".
This was the major reason we wanted to move to Fargate in our org.
Is anyone aware of this tool? Is it something I can check out now, or do I have to wait for the next installment? This sounds compelling, and even if I don't want to use Fargate, it will help me to relate with people that have used Fargate, and give us more common ground so we can talk on equal footing.
(Subscribe me to your newsletter!)
Part 2 is still planned and is now being fast tracked since there's interest in Fargate.
I am not a fan of Fargate based on personal bias mostly towards K8s and a sour taste I got when Fargate and EKS were first announced together, (with Fargate "available now" and EKS coming soon, it felt very desperate, but that's neither here nor there...)
Was very impressed by this screencast and will definitely try Fargate now thanks to your work here.
When I got to the point in the screencast where you build the image locally, I was on the edge of my seat hoping it would include some CodeBuild setup automation too; maybe you have already planned that, (or maybe that would ruin its simplicity.) That was a great demo, even without this part.
AMA!