Hacker News Reader

Comment by STRML | Hacker News Reader

While this looks nice, part of me can't help but be annoyed by yet another deployment option on AWS. We now have CloudFormation, Elastic Beanstalk (which can take many forms, including Docker), CodeDeploy, and Opsworks.

I can imagine how, for a new user, it's utterly baffling which of these options is the best for the longterm, with the least friction. I use OpsWorks quite a bit but have found it very challenging, and the feedback cycle when attempting to develop new cookbooks is excruciatingly slow.

All I want, personally, is a system that uses a set of interchangeable scripts that represent dependencies, so my server configs can live in version control. It doesn't even need to run on multiple OSs (which seems to be a central tenet of Chef). It just needs to deploy/rollback with zero downtime, and ideally autoscale as quickly as possible. Is this it? Is there any way to know without spending weeks fleshing out how it works?

View on HN

CloudFormation is really a different beast. It's focused on creating and managing related collections of AWS resources, not on application deployment. Anyone doing, well, pretty much anything on AWS should probably be using Cloudformation.

http://aws.amazon.com/cloudformation/

I've found developing OpsWorks recipes directly on the instance to be the best remedy for the slow feedback cycle.

Your cookbook is located /opt/aws/opsworks/current/site-cookbooks. If you run a command from the web console, you can repeat the command with a tailed log output using "opsworks-agent-cli run_command".

> I use OpsWorks quite a bit but have found it very challenging, and the feedback cycle when attempting to develop new cookbooks is excruciatingly slow.

Yes. As a frequent Chef user, I was really excited about OpsWorks, but it feels quite beta, and as you said the build/test cycle is even worse than usual for config management.

Have you had any success developing your Chef scripts on Vagrant and then using them on OpsWorks? It seems like the real thing is just too different. The open source OpsWorks cookbooks don't match what's actually running, you've got a VPC environment around you, there are extra tweaks to the AMIs, etc.

For new projects I think CloudFormation + regular Chef is a better way to go.

I tried getting a Vagrant environment up but I'm not convinced it's worth it. There is at least one repo with what looks like a working config [1].

I was far enough along in development by the time I started messing with Vagrant that I never finished it. I started debugging using the opsworks-agent-cli as mentioned in zackangelo's comment. I'd just been `git push`ing changes, updating cookbooks and testing again - his method of editing cookbooks [2] directly actually sounds rather workable. I imagine in the future I will mostly edit and test cookbooks this way.

1. https://github.com/wwestenbrink/vagrant-opsworks 2. https://news.ycombinator.com/item?id=8598255

I used this development flow regularly. You had to account for opsworks nodes settings not to be there. There's discussion where I currently work that there really should be a plugin to simulate metadata and perhaps the same or another to simulate opsworks.

How about an Amazon-managed Docker (not shit ElasticBeanstalk) infrastructure/config management system?

If they handled things like EBS integration, orchestration, ELB integration, and all of the usual stuff, I'd be sold in a second.

From my understanding, after talking to them at Web Summit, something like this is in the works. And they're aware of how incredibly complicated the AWS console is becoming. I don't know how or when they plan to address it, however.

They promise lots of things (like cross region VPC peering) to person to person at Summits...

Yes, I am still waiting for something prettier than software VPN or OpenS/WAN. Their docs show 5 ways to connect VPCs, all of them complex, all of them with significant downsides. One would think this sort of thing would be easier.

On Google Cloud Platform, there is a new service called Google Container Engine (currently in alpha). It uses Kubernetes to manage Docker containers, running on Google Compute Engine. You could also run Kubernetes on EC2 if you wanted to.

It takes a couple of days to get Ansible playbooks set up to deploy CoreOS VMs onto EC2 with a decent Docker setup and EBS. I know, I've just done it. And I don't like Ansible, and haven't used it seriously before (I use it for project because it's on a contract and it's easy-ish for them to find people who understands Ansible if/when it's needed) so it took longer than I'd have liked.

Frankly, I don't understand why people are so happy to lock themselves tighter into AWS.

I'm a big fan of Elastic Beanstalk. AWS with Elastic Beanstalk just makes things super easy for my deployment. And today Amazon announced they have over 1 million active users. This just shows how powerful their offerings are.

Or it shows that AWS doesn't have any other simple deployment service which doesn't tie you into using particular technologies. Hopefully this is where CodeDeploy will come in.

Wow STRML - thats exactly how i felt and why i created CloudCoreo.com... you are literally describing what we do (plus we handle services other than ec2 as well). it's not technically open to the public until January, but after seeing what you want i would love your feedback on the system. Let me know if you are interested and i can walk you through how it works.

also check out http://cloudnative.io

instead of using rolling updates, it's a blue-green deploy -- essentially what netflix does. it also automates ami building, triggered by a jenkins plugin after a successful test run.

Thanks for the link. Yes I was excited to see AWS moving higher up the stack and offer code pipeline and deployment options, but CodeDeploy is exactly what would have been designed (and in fact was) if EC2, ELBs and AutoScaling did not exist. It was born 18 years ago. It suffers from all the same issues that any rolling update with mutable infrastructure does.

Blue/Green, while harder to use without tooling like CloudNative adds, has numerous benefits including near instant rollback to a previously known good state.

Netflix figured this out 5 years ago.

Reply on news.ycombinator.com