Serverless CI/CD on the AWS Cloud | Hacker News Reader

Serverless CI/CD on the AWS Cloud

caylent.com

36 pointslucyinkedup19 comments

Personally I would not use CodeCommit, CodeDeploy, or Lambda for my CI/CD. I'm very much a fan of Lambda, but it just isn't the right platform for a lot of CI/CD.

GitHub Actions with self-hosted runners (https://help.github.com/en/actions/hosting-your-own-runners/...) offers a much more user-friendly and complete product. It's a shame that GitHub does not publish an AMI, but installing the runner is straightforward enough. A T3 instance is all that you need.

In theory I like Github actions, in practice it was a nightmare when I tried it.

Discoverability and modification was a nightmare. Solving any task basically required that I pick a community written extension and trust it with github credentials.

Their documentation also managed to both be too long and devoid of the details I needed answering, which is a neat trick.

In the end I went back to CircleCI, since writing a yaml file was much clearer.

You can always fork an action-repository to your own account, and failing that pin to a specific version.

Github actions I find pretty awesome, allowing your configuration file to just be glue between well-written, well-maintained actions for doing "stuff".

I've written a couple of my own, and since they're docker based they're very simple to get running, and very easily composable.

For example this one is trivial, but it lets you run project-specific test-cases on PR/push/whatever:

https://github.com/skx/github-action-tester

Just add `.github/run-tests.sh` to your repository and enable the action, suddenly your test-cases run. And since the test-cases are in your repository you can do anything you like, including run them locally!

I don’t want to have to fork and maintain a repo to run tests or push to a docker repository. Things that take a single line of bash should not require an entire repository to implement.

I have to agree with this. Recently, I started moving a few of my personal projects to GitHub, and decided to embrace Actions, since there is plenty of free minutes available each month.

I can say that I do like extensions to a point, and it seems that there is a lot more traction from the community, than for example for CircleCI Orbs, which are more or less the same thing.

However, CircleCI or GitLab CI (I'd reckon they are quite close in the syntax and features they offer) configuration seems much more straightforward, and is easier to write and understand.

Also, documentation really does leave a lot to be desired.

GitHub Actions are pretty powerful actually. I wrote https://github.com/pullpreview/action, which allows to continuously deploy branches and PRs to AWS instances and it works great.

The trick is to understand that for complex behaviours you can't just glue different actions together, and instead you may have to build your own action and use the GitHub API token provided in the context to possibly perform additional actions.

Except for CodeBuild which is decent; CodeCommit, CodeDeploy, and CodePipeline are severely underwhelming to embarrassing. As if they were written by people who had never experienced what modern CI/CD and dev tools looks like (and interns/college grads). And this is coming from someone who's had the "pleasure" of setting up and maintaining Jenkins.

If Amazon's internal tooling is like that, they should've gone ahead and bought GitHub. Would've probably paid off in terms of developer productivity very quickly.

As a community Im surprised there arent more open source configurations to do this type of thing. We should probably be assembling distibutions containing best practices or at least highly opinionated tools with well documented trade offs.

The entire “heres a parts bin go bananas” thing cant be the final state of devops. Simultaneously building profitable businesses on top of the three is a terrifying prospect.

Isn’t that what a PaaS is? Lambda, beanstalk, app engine, heroic, etc?

Sort of. But those are hosted highly opinionated systems. The option / flexibility of self hosting a system like that using open source wiring is whats missing.

So Amazon ECS? Bring the docker container but put it behind a cloud loadbalancer? Or App Engine Custom environment. (not tried to be confrontation just generally feel like a lot of these problem sets have been experimented against and provided for).

Ah I meant more the glue. Like open source cloudformation configs. Or terraform configs. The building blocks are all there to be sure.

If you are using the Serverless Framework to build your serverless application, we actually provide a full CI/CD solution as a part of the Serverless Framework Pro Dashboard that is free to use. You can easily sign up at dashboard.serverless.com and it includes a lot more than just CI/CD; monitoring, debugging, local development via the cloud and a ton more!!

We have had awesome success with Buildkite [0] and self hosted build agents on EC2. We use the provided autoscale lambda and it’s pretty hands off system [1]. You also get the upside of controlling how much local caching you want to do.

0 - http://www.buildkite.com

1 - https://buildkite.com/docs/tutorials/elastic-ci-stack-aws

Wow thanks for recommending this! I'm currently using Google Cloud Build and it's been great to get a CI/CD system up and running quick. But it does add about an extra 4-5 minutes of startup time due to limited caching ability. Buildkite looks like a great next step up while still being able to utilize GCP.

Interested to hear any feedback from those running Buildkite with GCP.

Seems marketing gig, how's it different than using Gitlab?

Probably. Amazon is equally guilty disguising "training" as a foot in the door for helping you solve ($$) your code pipeline issues. I'm not surprised if partner vendors do the same.

It's much worse; it's very difficult to debug complex pipelines (you can never get "inside" the runner), there are no services or reuse etc.

Due to lack of features you can really only use codepipeline in very simple cases for builds. Just too painful.

So it's better to stick to codepipeline for deployments only; but if you do, your deployment ci is now different than your build ci.

Really wanted to like it (the security benefits are real) but now avoid and use "full" ci for everything.

Obviously org, team and situation dependent, but we got the most velocity benefit from using 1 tool everywhere. If you had to use 1 tool I can almost guarantee you would not choose codepipeline after using it.

Highly recommend that anyone looking for an OSS tool to build serverless apps on AWS check out Architect [1].

The folks behind Architect built a fully hosted service that includes CI/CD called Begin [2].

I’m happily using both for a few different projects.

[1] http://arc.codes

[2] http://begin.com

Reply on news.ycombinator.com