A simple DIY Heroku replacement to keep your hosting costs down | Hacker News Reader

A simple DIY Heroku replacement to keep your hosting costs down

github.com

209 pointspritambarhate74 comments

This, Dokku, Flynn, that 1000+ line bash/chef/ansible script that the guy who used to do your job before he left are all legitimate solutions for deploying apps (and all better than FTP'ing up whatever the developer happens to have on their laptop).

But, I feel like the comparison to Heroku isn't great.

Heroku saves you from having to do and worry about significant amounts of (I would argue) extremely important admin and security work.

How are datastore backups handled in this scenario? How do you handle updating redundancy? Can you in a couple clicks add a secure Postgres, MongoDB, Memcached or Redis server to your app? Setup CI?

Or even just secure these things properly. This stuff gets complex and it's extremely easy to accidently set up something that's much more open to the world than you would want.

To sum up:

Heroku -> A way for developers to put up much more robust sites than they could otherwise.

Scripted Setups -> a (very positive and cool thing) for those with devops/sysadmin skills to more rapidly setup a site.

Full disclosure: I do a lot of work in the Heroku ecosystem and could well be defending the platform out of greed. But, I've also seen first hand some of the wild misconceptions and mistakes that were caught by the built in systems and limits to the platform.

While I agree that a script can't match what Heroku does, in the end Heroku shouldn't be used for anything serious.

I love Heroku for it's simplicity. They give you very few levers, making it less likely for you to screw up.

When we had a production add we faced following issues with Heroku:

1. Frequent downtimes. Heroku would have a 3-4 hour downtime every 2 months. In some cases, site would be down. In most cases you couldn't deploy and add machines.

2. There is an end user impact in terms of performance. After moving to Elastic Beanstalk, we had a 20% decrease in response time and 5% increase in revenue.

3. In the end Heroku is very simplistic. They don't have features like rolling deployment or Blue-green deployment. They had some beta version of rolling deployment, however that was buggy and caused version issues with deployment. With Elastic Beanstalk we have a health check based rolling deploy, which has caught production issues many times.

4. To do custom things, you need to deal with buildpacks etc, which is not very convenient.

5. Thought it wasn't an issue for us, Heroku can be expensive. Our server costs went down 80% while giving better performance.

So, in summary - Heroku is great to get you started quickly. However if you are doing anything serious but do not want to get deep into DevOps, consider something like Elastic Beanstalk.

Eventually we managed our environments using CloudFormation. That allows us to create repeatable Elastic Beanstalk instances. Once we got the CloudFormation template right, it was easier to maintain Elastic Beanstalk vs Heroku.

> in the end Heroku shouldn't be used for anything serious.

This is a provably false statement. There are thousands of companies running very serious things on Heroku handling load that would exceed the needs of the majority of audience here. All with near zero effort from a developer perspective.

They've also always been transparent about downtime, retrospectively creating/amending cases if required. The only multi-hour outages in the past 60 days afaict affected the ability to launch new processes & use the API but didn't actually cause downtime for apps that were already running (https://status.heroku.com/). That said, uptime for the past 60 days for the US region is > 4 nines, and EU is 5 nines. Better many ops teams achieve.

I was using rolling deploys (and preboot before that) on Heroku without issue back in 2013. I can't speak to the issues you had, but there were (is) a number of documented caveats to be aware of when using it.

You get all of that simply by doing `heroku create; git push heroku master`. Plus a broad range of automatically provisioned, configured, and fully-managed services you can use with a single click.

There's plenty of valid reasons to not use Heroku, claiming one if them is it's unsuitable for anything serious is disingenuous.

> All with near zero effort from a developer perspective

For a very small subset of developers.

The last time I was tethered to a Heroku installation, I spend 60% of my time coding, and 40% of my time working around Heroku's shortcomings.

It doesn't matter how many times you reply to people on HN stating that their position is "provably false," that doesn't make it true.

I've been using Heroku for many serious projects since its inception. It's been nearly a zero effort, especially compared to doing it all myself. This alone is enough to prove the statement false (counterexample).

A little nitpicky here, but the original claim is "shouldn't be used for anything serious" which is an entirely subjective statement. I don't know if proving something subjective is false is really possible.

That's the fault of the original claimer. They shouldn't be making subjective claims and trying to pass it off as objective.

Just for the other perspective...

1.) I’ve been running production apps on Heroku for 7 years and we’ve had about 3-4 hours of downtime total in that time with the exception of the major AWS outage a few years ago that was a full day. Maybe US-East is more robust than other regions?

2. It’s true. Request queuing specifically can be a killer.

3.) Heroku has had rolling deployments for a long time. 5 or 6 years at least. It was a beta feature for a lon time, but has worked perfectly since the day they opened the beta for my apps.

4.) sure, but there are a ton of build packs out there and even if they aren’t a little scripting makes everything pretty straightforward. Especially compared to scripting everything yourself.

5.) definitely true. But, if you have a team less experience or less interested in devops it can be a great platform to trade salary for ease.

I think we're agreeing more than disagreeing here.

But I think you're not properly valuing yourself in this scenario. People that can utilize AWS and navigate through all the services, use CloudFormation, etc. are in a high demand and at some level in the cost analysis you need to factor that in.

In terms of my time, it has been very valuable.

I had to spend quite some time working around Heroku's limitations. We have had 100% uptime (based on external monitoring) since we moved to Elastic Beanstalk.

Health check based rolling deploys are awesome. Caught production bugs 3-4 times.

I agree that it was easier for me to do, because I have extensive DevOps experience. However if someone is getting to anything more than a hobby project, consider investing in someone to build a basic CloudFormation script for ElasticBeanstalk. I will open source ours in a couple of months (things really hectic right now).

My company hosts a whole bunch of critical software on Heroku. It generally works fantastic. We do use relatively less "batteries infra for some of our services, and we'll probably shift further in that direction, but I think you're underselling just how effective Heroku is. It goes a long way towards letting your team worry more about the business domain and less about the infrastructure concerns that aren't at all unique.

My last job we were 100% hosted on Heroku and coming from managing hundreds of EC2 instances and databases via autoscaling, CF, Puppet and home-grown scripts, it was a dream to use Heroku. We simply didn't need any operations staff to manage infrastructure, maybe a few hours a week for 1 developer out of 11 to adjust or verify a few things.

Need a bigger database? Just a few Heroku CLI commands. Bigger web servers? 2 clicks. Want a test database to run queries against? 1 Heroku CLI command. Want to test a branch of your code against the staging database? Automatically done via GitHub PRs, with DNS setup too.

We had 0 downtime in a year due to actual Heroku issues, developers could push code easily (git push), databases were backed up automatically, servers auto-scaled, etc.

I was genuinely amazed at how well it all worked. The amount of operations work was easily 10x less than if we had to run our own infrastructure on AWS due to not just the hosting, but the tooling. Deployment, rollbacks, slave database setup, configuration management, access control, log aggregation, 3rd party integrations and more.

Heroku does have some drawbacks, request queuing is one of them, mostly due to lack of clear docs and information. Even with its flaws it still saves a massive amount of time and money. For nearly all startups to medium sized companies I'd highly recommend using Heroku.

OTOH, the Heroku people aren't personally managing each instance you create; their work is writing the automation code that turns AWS VMs into these more manageable, higher-level pods.

Maybe Dokku and Flynn aren't there yet, but conceptually, is there a real difference between the two? I don't see it.

It's that whole "as a Service" part people tend to forget when making overly generous analogies.

Well, Dokku [1] is already a good solution to the heroku cost problem. All the power and convenience of heroku commands with the freedom and low cost of docker deployment. It has lets-encrypt ssl support too!

[1] https://github.com/dokku/dokku

I'm using Dokku to serve a bunch of sites and love it, it has made deployments a breeze. I did run into some Docker problems (used too much CPU and couldn't connect to its socket, etc), but I don't think it was Dokku's fault, but Docker's.

I want to see if I can use Kubernetes as a Dokku replacement (hopefully a more stable one), but I don't think Kubernetes has all the niceties Dokku has (automatic reverse proxying, Let's Encrypt support, etc).

You might look at Red Hat's OpenShift, which builds on top of Kubernetes. Or Deis, which has adopted buildpacks.

Cloud Foundry has buildpacks too, but it's usually seen as a heavyweight solution -- the light footprint variant is 6 VMs, the standard variant is 20-something.

Disclosure: I work for Pivotal, we contribute a lot to Cloud Foundry. Red Hat is a competitor.

SUSE's Cloud Application Platform is already available to selected customers. It is based Cloud Foundry running on top of Kubernetes, all with a very slick UI. No VMs.

https://www.suse.com/products/cloud-application-platform/ https://www.suse.com/communities/blog/take-look-suse-cloud-a...

I tried it recently with an internal app, and the experience was very similar to Heroku. cf push and my app was running.

Disclosure: I work for SUSE, we contribute to Cloud Foundry and the containers ecosystem. Pivotal is a competitor.

I am actually super pleased that SUSE have picked up where HPE left off. I am hopeful it will avoid too many distro-level monoculturisms.

The link to the UI blog post has turned up in our private chat channels :)

btw, here is an article about the management UI:

https://www.suse.com/communities/blog/introducing-stratos-ui...

Ah, I've heard about Deis, I'll check that one out, thanks. My main concern is the fact that Kubernetes recommends running more than one server, which is a luxury my personal projects cannot afford. Does anyone know how well it runs on a single server?

It is possible to run Kubernetes on a single machine combining master and worker nodes but there are few instructions for it. Really only makes sense for single bare metal machine. For production, better to run separate VMs. The hosted Kubernetes, GKE and Azure, can run clusters without paying for master nodes and can have pretty small nodes.

There's minikube, but so far as I'm aware, it's not really meant for production use; it exists mostly for local development.

My own company has Pivotal Web Services[0], which gives a Heroku-like experience using Cloud Foundry. Pricing is per-megabyte[1], plus services. I expect we'll introduce a container service before long, because we like to dogfood our stuff with real production workloads.

[0] http://run.pivotal.io/

[1] http://run.pivotal.io/pricing/

Yeah, minikube is just for trying K8s out. PWS looks interesting, but, as with other managed/cloud offerings, it's much more expensive than a dedicated server (50ish EUR/mo for a 64 GB RAM i7).

For my side-projects, where I am cost-sensitive, I think the best solution would be a dedicated server with something like Dokku. It would just be nice if the "something" could be an industry standard.

I think it's hard to split the difference. If there was a service using RackHD to spit out cheap bare metal it'd be more doable to run on top of that, I think.

It's easy enough to set that up (though it's not the recommended use case) using kubeadm.

I use Deis on a single server. It works great.

There are a few things you need to be aware of, but the tl;dr is that it works great for us. We'd be using it more, except that our org hasn't adopted containers in any meaningful way yet, and Deis is large and complicated, and a little too much for them to bite off and swallow all at once. They want to do a pilot on ECS so that we can say we're using more AWS services (yay lock-in, we love vendor lock-in.)

So, those things you need to be aware of:

#1 - In the default mode, Minio is used instead of a real Object Storage backend like S3 or Swift, and Minio is configured to keep all objects in RAM by default. Use S3 instead. This is an option to configure in values.yaml.

#2 - builder really needs to be on an SSD, but if you're not backing your Kube node with SSD, you can at least prevent a lot of builder crashes by making sure there isn't much swap to fill and bog down the disk with swapped pages. This may be counter-intuitive (don't things crash when they run out of RAM? Isn't that why you have swap...) until you realize that Builder has to respond to health checks by default once per second. So if the node that builder runs on is getting even remotely bogged down by swappiness and lots of swappy pages, it will be killed and restarted, which takes some time and is noticeable and annoying.

#3 - The remaining cloud services that Deis can optionally depend on IMHO are take-or-leave, depending on your cloud provider. Deis can do everything in-cluster. I actually use this configuration with Minikube and Deis, and it works great. I let Deis spin up its Postgres database on-cluster and restore from S3 WAL backups each day, it just works.

I'd be using Deis more heavily, except that my app development has a production database attached. (Our infosec department is leary about letting me put backups of sensitive data into S3. So we really have to use RDS for our app, even though Deis is creating a Postgres database that nothing else is stopping you from using with your app. It's for Deis controller operation though.)

#4 - This one may be a dealbreaker if your company is very serious, Deis is now "End-of-Life." This means that Microsoft has committed to provide break fixes for Deis until March only, and by middle of next year, any break fixes will have to come from the community. There is AFAIK nowhere to buy Deis support, although there is a new group that has formed to commit to support Deis into the future, it's not clear what form that support will take, or whether you'll be able to pay money for an SLA, or anything like that. It is open-source.

This may be exactly what you'd prefer, or it may be a deal breaker. I like Open Source, but my group likes support.

I do not work for Deis or any other cloud PaaS provider, although I have evaluated a lot of them including OpenShift (and I consider Deis to be technologically superior in some ways to OpenShift.) Because of my ECS requirement, I'm looking at Convox now. If you have to use ECS, so far I would definitely recommend Convox. If you're like the rest of the world and using K8S, I can't think of any really good reason not to recommend Deis.

That's very helpful, thank you! I was with you up until "Deis is EOL", I only want to use it for my sideprojects, but I still wouldn't want to be out of luck if something breaks on the next Ubuntu update and I can't get a fix anywhere :/

Maybe I should look into OpenShift instead, as that's more active?

I will tell you this: there is a group of people who are committed to keeping Deis alive, called Team Hephy and the Deis slack is still active. I've used OpenShift and it is certainly more active, but I also have the very strong feeling that Deis is "done" where many parts of OpenShift look like they are still evolving. (And I mean "done" in a good way.)

If you are not sure, feel free to drop in and ask any and all questions: https://slack.deis.io/

The announcement of EOL came in July, and the announcement that Team Hephy would take over was almost one month to the day later, in August. I have a lot of faith in the Deis codebase and I'm on Team Hephy now.

We have yet to put out a fully e2e acceptance test passing and providence-verified signed, versioned release of our own, but we have and know of a number of production users that still use Deis in Production.

It definitely does not feel like a ghost town on the Deis slack. Questions get answered. Neither does OpenShift (it is definitely not deserted, and it does not seem to suffer from "abandoned k8s fork" syndrome, they do keep up with the release cadence, because they have to.)

I would go with the one that solves your problem better. I'm personally still recommending Deis, as I like the built-in support for Heroku buildpacks that OpenShift lacks. (Convox also has support for this, on ECS.) I'm not aware of any K8S-native Deis competitor (other than Hephy) at least for now.

Edit:

The really nice thing about working in Kubernetes is that you have the conformance suite[1]. You shouldn't worry about breaking something with the next Ubuntu update, because if Ubuntu update has broken Kubernetes in a way that will cause Deis to fail, then it will not be a Kubernetes-conformant distro anymore (and there is a large list[2] of K8S-conformant distros and platform to choose from, so you should pick a conformant one[3] instead.)

So, your question should be whether Deis will continue to work on Kubernetes distributions that are conformant, and with some minimal research, I think you should be able to say that with a high degree of certainty it will, regardless of ongoing vendor support from either side.

Then, you can ask whether your K8S distro is likely to remain conformant. (This should be a no-brainer.) Finally, is there anything that can break within Deis internally (not related to how it interacts with Kubernetes) that would mean you'd have to stop using it?

For me, the big reveal / final test will be, can we upgrade from Cedar stack to Heroku 16, which is on Team Hephy's planned roadmap for the future.

If that goes smoothly, then I'm not sure what else there is to worry about.

[1]: https://www.cncf.io/certification/software-conformance/

[2]: https://www.cncf.io/certification/software-conformance/#logo...

[3]: https://docs.google.com/spreadsheets/d/1LxSqBzjOxfGx3cmtZ4Eb...

I used to work on buildpacks -- can I suggest getting in touch with the Cloud Foundry Buildpacks team? They also do a lot of work around the Ubuntu base image (cflinuxfs2) and there's been work towards cflinuxfs3.

They also used to use soft-forked Heroku buildpack code. Almost all of the Cloud Foundry buildpacks have now been rewritten on top of a single shared library. The strength of Heroku's code was that each buildpack has a maintainer familiar with the relevant ecosystem. The weakness was that they were all different. Often forked from a previous one and then tinkered into submission.

That's really cool and good to know. I will keep that in mind! Thanks

Thank you, the fact that there's a team that will take over puts my mind at ease somewhat. I'll give Deis a shot, thanks again!

For the Kubernetes + Dokku replacement, GitLab's Auto DevOps https://docs.gitlab.com/ce/topics/autodevops/ can do a lot of stuff. It's included in CE, thus it's free.

That's a layer over Kubernetes and Prometheus, no? It looks good, but it assumes I've already installed the former.

In the next few months, GitLab supports GKE Cluster Integration. https://docs.gitlab.com/ce/user/project/clusters/index.html

You can set up your Kubernetes, Helm, Tiller, Runner, Prometheus by one click.

Gitlab now bundles Prometheus, but you would need to install Kubernetes.

Kubernetes can have automatic reverse proxying and let's encrypt via an Ingress controller. I don't doubt Dokku has some nice plugins that Kubernetes doesn't though, especially for developers.

Ah, thanks, I'll try that. Dokku does have nice plugins, but they tend to be things I can write Dockerfiles for. The other big thing Dokku does is that it builds the image on the server, so it doesn't need a Docker registry, you just push the code directly to Dokku and the image gets built and deployed.

I haven't used kubernetes yet, but https://traefik.io/ was a great reverse proxy in combination with docker swarm. I imagine it works great with kubernetes, too.

Putting a few labels on the containers you want to expose with your reverse proxy is all you need to do. Traefik continuously monitors docker to react to changes. Traefik handles load balancing and automatic certificate generation with ACME, too

That looks great, thank you. I'll try it with my K8s install.

Shameless self plug: we are also building an alternative[1] that enables you to run tons of services on a single node (and to call them easily from front end projects - with the help of autogenerated type safe clients).

[1] https://github.com/1backend/1backend

I used to love dokku until I had 7-minute deployment times for an average application in production. Has that been fixed? Was around 0.6.0 I think.

Can you give more info on that 7-minute deployment time?

Dokku is basically a wrapper around Herokuish which is just a docker image with Heroku buildpacks pre-installed on them to build an image of your app to be dployed to docker instance...

Source: Am a maintainer of Dokku.

Mine takes 30 seconds for a sizable rails app

It's awesome. Even lets liberal arts majors set up servers and R, Python, etc. apps with easy ;-)

As a relatively late Heroku convert (we moved to it maybe 18 months ago from managing everything ourselves) all of these DIY Heroku replacements may cover deployment and scalability well but they miss two very important aspects of using Heroku.

1. Heroku will manage the underlying os and software, ensuring its all update and patched. We don't need to worry about the security of the bottom of the stack, that is their responsibility.

2. Backups, Heroku have a fantastic backup system for Postgres, it has already paid for itself. I will find it hard to move to any other system after they saved me after accidentally deleting half our db...

My only "complaint" about Heroku is cost, it is quite expensive for memory heavy processes (we have a few of these), jumps from $50 for 1gb to $250 for 2.5gb?! But the ease of use far outweighs that. Just hoping that there is a pricing update with more ram coming soon, I think its very overdue...

I think that products like this occupy a tricky space. Heroku won't win any contests for price, but we know what they do offer - a simple setup, and you can trust that they have experts behind the scenes working to keep everything up-to-date and secure. That's one extreme.

The other extreme is, I set up my own servers by hand or with a handful of custom scripts. Cheap in price, but time-consuming, and I might miss something. The advantage is that I know how every bit of it works if I need to add a feature or set up something weird.

Either extreme makes sense when used for the right purpose, but I feel like it's harder to find a place for things kind of in the middle like this. It has a bunch of nice, handy features, but how much can I trust the maintenance on it? If some sub-item has a security alert, how quickly will it get updated? Will I have to dig into the guts of this thing to fix it? Will that still be true 3 years from now? If I want to do something weird and custom, how hard is that?

This but based on Kubernetes/Deis would be a huge thing. This project feels like may be fun, but not industrial strength. A bit like Dokku and Dokku-alt some times back.

There something called Minikube also right? Maybe that would also be great to integrate.

Maybe OpenStack's interface could even be used as a starting point.

Oh, and I frown at the use of JS where it is not mandated. Looking at the K8s/Deis community it seems Go is the favorite, and I find it a much more defendable choice for serious infra projects. (sorry will stop about JS now)

Reading back I gave some comments (and even a JS nag), but I mostly want to say that I love the project from what I see on the README. Very well done REAMDE. And I love this LetsEncrypt feature (Deis has is identified as cool feature as well -- https://github.com/deis/workflow/issues/708 )

How does this compare with Flynn ?

https://flynn.io/

The first thing that comes to mind is that captainduckduck is free and opensource and the product you linked to is a closed source service you pay ($3,499/month) for.

Flynn is also free and open source, the pricing is for their management services -- that's explicitly stated in the first paragraph of their pricing page

It's been said already, but the idea that Heroku is expensive is provably incorrect.

They're not providing hosting, they're providing a service.

If I want self-hosted platforms, I'll manage my own docker containers on a GKE cluster or something similar. As it turns out, I want to write code and scale it to millions of people. Heroku lets me do that, and I don't need to hire an SRE team.

Well, maybe provably incorrect for you but I can see that if you're a company outside the Silicon Valley bubble, Heroku would be way too expensive compared to other solutions. Even for me personally (living in Barcelona) I think it would just be too expensive to use for anything without feeling like I'm bleeding money.

I am not (and have never been) inside the Silicon Valley bubble. I live in the UK and work outside of London. I'm as far from the SV bubble as you can get and still work in tech.

Some of the big sells about Heroku for me are:

- Heroku takes care of patching the OS for you.

- You can scale easily to more than one server.

- If a server goes down or becomes unresponsive, Heroku will essentially destroy and recreate it.

- There's lots of simple options for backups, rollbacks, logging and alerts.

How does this and other solutions like Dokku compare? If your project isn't making much money I can understand wanting to keep costs low. If you're building something for a client or are making a profit though, trying to save something like $1K (a few days of a developers time) a year doesn't make a lot of sense. To get all of the above, you're going to have to spend significantly more in developer time and end up with something less robust which will likely have lots of growing pains. Heroku lets you just get on with the coding.

A blog post by the author of the project is here: https://medium.com/@kasra85/how-i-cut-my-heroku-cost-by-400-...

I haven't used this, but if you're considering this I would highly recommend taking a look at https://flynn.io.

The usual counterargument is that DIY keeps your costs down if your time is free.

Disclosure: I work for Pivotal, we have some overlap with what Heroku does.

The great thing about CaptainDuckDuck is that it expressly aims to address your argument. I haven't used it myself, but from reading his blog post (linked below), his goal is to build something 75% cheaper, but just as easy-to-use.

> My goal was to enable a typical web app developer create a Heroku-like server instance in less than 10 minutes. I am happy to say, I did it!

I think I worked out what my actual problem is here: pride.

I work on a platform partly inspired by Heroku and which consequently deals with many of the same problems. I am intimately aware of how much work has been required to build that platform and expand its capabilities.

It is hard to read someone saying they have replaced Heroku and not feel annoyance, because it downplays the enormous amount of careful engineering that Heroku and its competitors represent.

Also, infrastructure and support.

another option that is a pretty good Heroku clone is to deploy to a Deis stack. https://deis.com I set up a deploy on kubernetes a year ago and it was very close to Heroku -- impressive.

Yeah, Deis!

I have been a big proponent of Deis even as Microsoft has acquired the team and they've gone more hands-off with the project. It's harder to make the case for Deis when there is nobody you can buy support from...

(Is there anyone left that you can buy support from? I heard the easiest way to get an answer on the internet is to start by "being wrong"... secretly hoping there's still someone out there in the business of providing support to Deis)

I'd also settle for someone who has continued to use Deis in production after the end-of-life date was set, and currently has no plans to move away. (I know you're out there...)

I've been looking at Convox the last few days, because my team actually wants to insist on doing something with ECS. We've used Deis with our developer laptops with a middling degree of success.

The best part of using Deis now IMHO is that when they ask "what are you going to do when you need support and you can't get it," ... or "what are we going to do if you leave, and nobody knows how it works?"

Well there are a lot of 12-factor PaaS and hosts. Pick another one and move on. Or you can roll your own. It's not that hard, there is even a step-by-step guide (12factor.net)!

I think the two major platforms for PaaS into the foreseeable future are going to be OpenShift and Cloud Foundry.

Both have large engineering professional organisations involved, large community footprints and a lot of sales to pay for all of the above.

There is what I call the "kambrian explosion" of PaaSes being built on top of kubernetes. Most of these will, I think, shake out over the coming years and again, the heavy-duty PaaS market will largely be dominated by the two existing platforms.

Disclosure: I work for Pivotal, we're the leading contributor to Cloud Foundry. Red Hat is a competitor.

Thanks for chiming in, Jacques from Pivotal!

I don't know which teams have the most meat behind them, but I know my team (or, the part of my team that deals with infrastructure and platforms, at least...) has taken a guarded approach while embracing as much as possible everything that AWS does.

So while I've recommended Deis and OpenShift I've only really been able to see them start to take a serious look at Kubernetes now that Amazon says it's a thing to look at. We don't have any K8S or containers at all, except for our Jenkins server. We knew this announcement (EKS) was probably coming, so we started to look at it again weeks and months ago, and we still haven't really made any progress. The working group is formed and we're committed... to looking at it. Starting now.

We haven't really even looked at either OpenShift or CloudFoundry. So I'm asking you, in the hopes that you'll give me something I can relay to my team, why do you think CloudFoundry will win? What are the major advantages of CF?

Large engineering effort, community footprint, and sales are all good indicators, but from a technical perspective, why do you think we should be looking at CF?

Disclosure: I work in University IT, where we have a majority of legacy projects and large databases to support, as well as incidental development efforts that we'd love to support with a platform (like the project I'm working on, a Rails app that provides a suite of workflow actions for Human Resources and Payroll to obviate their antique paper-esque forms processes)

> Large engineering effort, community footprint, and sales are all good indicators, but from a technical perspective, why do you think we should be looking at CF?

Broadly it's that a good platform gets everyone out of everyone's way.

Cloud Foundry (these days, it's called the Application Runtime or CFAR) has made it possible for small teams of operators to support thousands of developers and applications. Similarly, it means developers can deploy rapidly, get databases and integrations on a self-serve basis, enjoy uniform platform features and so on.

There's now a lot more interest in the container as the unit of deployment, which is where kubernetes has become the winning orchestrator. So there's also a Cloud Foundry Container Runtime (CFCR), which is based on kubernetes, designed to use the same low-level operator tooling, BOSH, and to integrate smoothly with the Application Runtime.

Both the Application Runtime and Container Runtime are intended to run on any major IaaS; BOSH provides a uniform layer that abstracts away the raw interfaces. So you can deploy to AWS, GCP, Azure, OpenStack, vSphere, RackHD and there are others I forget.

Last week we had our flagship conference, Spring One Platform, which had a parade of customers talking about how much faster they move. Months per deployment turning into deployments per day. Thousands of developers running tens of thousands of apps. I am aware of customers handling billions of requests per day and handling hundreds of millions of business events using CF. Going through black friday effortlessly without having to massively overprovision.

tl;dr we build installable superpowers.

> developers can deploy rapidly, get databases and integrations on a self-serve basis, enjoy uniform platform features

That's what I'm looking for, personally. It's a real headache that in our environment, we usually have exactly one each: dev, test, prod. Sometimes you need a couple more dev environments for a few days, but it's totally not worth it to go through a RFC or REQ process to get it. Instead, you tend to get developers merging features before they're ready, just to be able to get them in front of customers at the same time.

> tl;dr we build installable superpowers.

I was assuming this, but it's good to hear it out loud ;)

Honestly don't know what my teams will go with, but we're starting a trial of ECS for the organization's first formal foray into containers. Convox looks to me like it might actually make using ECS palatable. I was frankly not at all optimistic about ECS until I found it.

The scope of the pilot is written to deliberately exclude K8S, which I'm not sure I agree with, but there are also a lot of people saying they believe we will fail on our first attempt to implement containers as an organization, and I'm not sure there is any choice we can make (K8S or otherwise) that would surely make those people wrong (to make the first attempt succeed.) Solely due to institutional inertia, and also difficulty integrating with people who are slow to adopt new things. It's tough working in a group where everyone must insist on moving together.

As a developer, I've been using containers seriously in various shapes and sizes since probably well before 2013; here it's still considered new (and scary.)

So I almost prefer that we fail hard, at least once, so that we're not stuck carrying legacy garbage along with us forever. Failed launches may leave orbital debris to burn up on reentry. But a successful launch could remain in orbit with its components, for better or worse, basically forever...

The biggest risk is going to be the temptation to roll your own PaaS on top of ECS. Later on it might be to roll your own on top of Kubernetes.

It just doesn't seem like a lot of work. Assemble this logging system, that proxy ... before long you have a working PaaS and a source of pride and joy.

And a future millstone around your neck.

A lot of people who work at Pivotal have worked at other places that tried to DIY. Usually this was a long-running bonfire of cash, during which any potential competitive advantage was lost. No matter how technically capable your team is, there is just a hell of a lot of engineering to be done to build systems of this depth and breadth, especially when they are distributed systems.

Software generally is a sharp illustration of comparative advantage. Even if you can do everything better than anyone else, it does not make sense to do everything. Focus on the area where you can add the most value. It is almost certainly not in rolling your own cloud application platform.

Shouldn't something like this be deliverable with a set of Fabric or similar scripts?

I wrote a thing that uses fabric to do docker-compose based deployments. It also handles static sites.

I've used it internally at a couple of companies and it works well for us.

https://github.com/mikew/b3cmd https://github.com/mikew/b3cmd-server

  vi ~/.bashrc
  alias cdd=CaptainDuckDuck

So cool, but the name makes it needlessly hard to adopt it.

So, I have to tell my client I'll move his apps from Heroku to a CaptainDuckDuck install..?

NodeChef is a Heroku alternative which is cheaper that DIY https://www.nodechef.com/

Just playing with the sliders on the pricing page show I'd pay almost double (~130/m) what I pay now for Heroku (~70/m).

I see you're the cofounder of NodeChef. You need to rethink either your pricing or your market position. Also, it's customary to be upfront about conflicts of interest, by declaring "I am the cofounder of NodeChef". Better than trying to hide it (which comes off as scummy), or expecting people to dig through your comment history, which is just lazy.

Reply on news.ycombinator.com