Please. You seem to be making the same mistake, that many make on blogs and news websites, of exclaiming "containers are the future!" while both VMs and containers have their place and use-cases.
I'd rather suggest you read the below. (Since I'm linking to a blog, quick background about the author: Rich is one of the long-time contributors of KVM-based Virtualization stack and the lead author / maintainer of libguestfs.org)
https://rwmj.wordpress.com/2013/06/19/the-boring-truth-full-...
The thing is, though, they are at odds. Enterprises don't have nearly the dynamic workload set that people thought but where they legitimately have some is in devops style development. And if you have container management, which you do, then highly dynamic VM management is much less valuable.
Most of the customer sites I interact with on the high end just end up baking their own system images tied to a SSO solution and just manage it with VMWare/Ansible/something homegrown. On the low end people just don't deal with OpenStack at all and just do it the hard way with a VMWare or KVM style solution. For either side of that equation containerization is a faster value-add.
From the other side what people ask me about cloud integration is more feature-driven of particular clouds than anything else. I'm just not seeing the momentum in the field.
From that perspective, container solutions are delivering a better developer proposition than OpenStack has yet managed. There are ways now to build container clusters that you can ship in parallel to AWS and Azure with very little code difference.
In that earlier discussion I was skeptical of OpenStack precisely because of its focus on infrastructure first. Without the buy in of being a clone for a specific cloud structure (AWS compatibility over anything else, for instance) or the backing of traditional datacenter/server vendors (IBM who eventually started into BlueMix; Microsoft whose "on premises Azure" is now firing on most cylinders but was announced as a plan early in OpenStack's history), OpenStack didn't seem to have an obvious niche in the infrastructure world. The closest to a niche it might have had in its early life was the promise of application portability between clouds and that never quite seemed to be delivered.
I can tell it frustrates infrastructure folks to hear that containers have been eating OpenStack's lunch, but that is the very real case from the developer perspective. As a developer today, I go for containers and OpenStack is no longer relevant on my radar. Sure I can run containers on OpenStack, but containers abstract away more of the infrastructure and I have less and less care what cloud(s) is underneath the container cluster. When I asked OpenStack people what OpenStack might deliver to me that vision of application portability was tantalizing but never seemed quite finished; container technologies have actually delivered that.
> There are ways now to build container clusters that you can ship in parallel to AWS and Azure with very little code difference.
OpenStack is an IaaS, so just like AWS. Focus on the context. Do you want your own private cloud? Yes or not?
If no, then this discussion can end, because AWS and Azure run on their proprietary IaaS code. As a customer, you request resources from the IaaS layer, and you build your server/platform from that point.
So arguing that container clusters can ship to other clouds with very little change (almost likely writing the APIs to create a container) is unfair in the context of why one would choose container over OpenStack. The purpose is different.
If your answer is yes I am building a private cloud, how are you going to do that with Docker alone? Can you build a software-defined network with Docker? Absolutely not with Docker since Docker is a host-based deployment solution.
What you are looking for is ability to create OpenStack the same way CloudFoundry / Kubernetes are created. You write up a manifest, and the necessary databases and services are deployed to some EC2 machines. In the case of CloudFoundry, you write up a manifest file, describes number of instances, types, credentials, what not, then call Bosh to create Cloud Foundry (will create a pool of app machines, router servers, UAA, etcd etc). Machines are created based on a stemcell, basically an image. You want to create your IaaS based on images. You want to be able to script up a manifest and deploy your IaaS. You want a lift and drop IaaS infrastructure. Container can do that, but it cannot be done simply with Docker. You need that infrastructure abstraction layer to cover up. That's probably what Docker Enterprise Edition might do, but I have not really dig into it yet.
As a software developer, do I ever want a private cloud? No. Does my employer? Maybe. Is it my job to tell them how to invest their infrastructure dollars? Quite possibly no, because software development and infrastructure are typically held at arms length. But even when they are not in a "proper" DevOps shop, the ballgame of which cloud is then subservient to developer convenience and how easy it is to deploy software to a cloud and how productive developers are writing software for that cloud.
So yes, the purpose of OpenStack and Container technologies are very different and I appreciate that technically. In terms of real world value to me as a software developer, however, I have platform problems not infrastructure problems. I don't care what the infrastructure is under the service so long as it provides a stable, reliable platform for me to build upon. Containers abstract that for me in a way that solves real platform problems that OpenStack was only ever relevant to me in so far as its ability to once hint at a possible solution to. That's not fair and that was expecting too much from OpenStack at the time, but that's life.
Of course, I would advise against running a private cloud unless there is a dedicated team of at least a dozen or so. I applaud Digital Ocean for able to survive and make good business from their private cloud. As a developer I totally agree I just want my code to be deployed and that all the appendices are deployed and configured.
OpenStack is at the same layer as AWS, GCP or Azure. It's not an abstraction over them -- that's a PaaS. I'm not sure OpenStack was ever seriously intended to work at that level.
I work on a platform, Cloud Foundry, which provides the kind of portability you're talking about. Red Hat have OpenShift, which is another such platform.
Disclosure: I work on Cloud Foundry on behalf of Pivotal.
Not that asking AWS+GCP+Azure to commoditize themselves in such a way makes much economic sense, but that's the proposition.
Cloud Foundry gets around this by using BOSH for deployment; it essentially creates the kind of layer you're describing.
There was a hint at some possibility that it might, especially in some of the hype circles early on. It is of course unfair to blame OpenStack for never actually fulfilling that dream, when that was not its intention per se, but that is part of why today people continue to compare what OpenStack promised versus what containers delivered.
It's frustrating to read people comparing container technologies or worse container schedulers with OpenStack. Apples and oranges.
If you are planning on running containers on premise you need something from managing the IAAS side of things. OpenStack is excellent at IAAS and is the foundation for running higher level abstractions and services on top.
I disagree with you that you need an IaaS solution to run containers on-prem, but realistically, yes you do need an IaaS solution