Kubernetes Founders Launched Startup Heptio to Bring Containers to Enterprise
blog.calsoftinc.com
blog.calsoftinc.com
This happens quite frequently with open source infrastructure. Keep an eye on them to push back against improvements that directly conflict with their company's "enterprise sauce".
What Brendan, Craig and I did was (a) meld those ideas with the external-to-Google world (including Docker), (b) motivate releasing it as Open Source (not the default model for Google TI) and (c) work to seed a really open community that we wanted to participate in.
Soon after we started some other folks from around Google and beyond joined in. From Google, those include Ville Aikas (not super active these days, doing other stuff at Google), Tim Hockin, Brian Grant, Dawn Chen, Daniel Smith. Most of these folks had worked on Borg and/or Omega. From outside Kelsey Hightower got involved very early (long before Kubernetes was a thing for CoreOS) and the folks from RedHat (led by Clayton Coleman). I'm sure I'm forgetting folks but hopefully that gives some context.
I'm hopeful k8 won't go the way of openstack because the problem domain is simpler and more well defined, but I'll be damned if I ever believe a "diverse" group of enterprises with shared interests can reliably produce good software.
Fool me once, shame on me. Fool me twice, shame on the standards committee.
It is a funny you say this. I spent a lot of time looking around the community at what existed before starting CNCF, and agonized over this. We needed to take K8s to foundation so that it wouldn't be a 'Google project'. Google was actually the best steward of the tech you could imagine, because the plan was always to make k8s ubiquitous and just win on quality of infrastructure but the community had no way to know that.
I looked at OpenStack hard, and like the energy and enthusiasm but really worried about (1) balkanization that was emerging with no 'true north' -- it just didn't have technical taste, (2) the tragedy of the commons -- most vendors were focused on their own interests and neglected the end users, (3) lack of coherence.
When designing CNCF I tried hard to work through this by creating a better foundation structure. (1) the business board has very limited authority over projects, hopefully making sure that we avoid it being a pay-for-play affair. (2) we made provisions for little companies to get top level seats based on community contributions (ditto) (3) we created an empowered end user group that would have equal authority to any other affair to make sure real users interest are promoted (4) we added a TOC (technical oversight committee) that was the most empowered group to establish true north that is community elected -- the idea is they need to champion the projects and establish technical 'taste' (e.g. Brian Grant from Google -- the guy who drives consistency sat on this group, not me the guy who had access to the purse strings and who was focused on the business).
(side note: i picked this structure because i was geeking on government structures at the time, and figured that the separation of powers yields more sustainable administration)
In terms of looking at OpenStack hard; and reaching decisions based on various <things> did you do any reach out to the OpenStack community to actually communicate the things you found or heard or concluded so that the group there (including myself) can actually work on improving itself (or perhaps some of the reasons you stated aren't even correct and the community could have helped you clarify those)? If not then it concerns me that you may have reached conclusions without actually talking with that community (but I don't want to jump to any conclusions without getting your thoughts/input).
So far from looking outwards in on the CNCF and seeing how it compares to the OpenStack community (which I am more involved with, including other small side-communities that I also work in) I've yet to understand what exactly the CNCF is targeting. It seems to be a body that is just adopting various projects that align to some mission (?); I have personally a hard time understanding the reasoning some of the projects have been adopted, maybe you can shed some light on that (what is true north for the CNCF, where is it written down, what is the TOC actually making adoption yes/no decisions on? what criteria? what is the technical taste you talk about, where is it written down?)
The nice thing about OpenStack is that they are writing most/all of this down and agreeing on those kinds of questions in public:
https://github.com/openstack/governance/tree/master/referenc... (github is a mirror, not the source of this repo, but easier for browsing purposes).
The mission of CNCF is the promotion of 'cloud native technologies' -- specifically container packaged, dynamically scheduled, micro-services oriented workloads. It isn't about picking winners, it is about establishing a safe space for innovation and bringing to bear the collective communities. We have legitimately taken some time in getting the identity of the foundation established, but I feel like Dan Kohn (our new ED) is doing super work in creating a collaborative space for new projects.
For me that would have repercussions for what other tech a CNCF project supports (e.g. is all stats monitoring based on Prometheus, or can 2 projects in the CNCF support different technologies)
Of course there are people who want a KubeStack that is like OpenStack, for better or worse. That's fine too! We just don't want that to be the ONLY choice for customers.
You said it yourself in another comment on here: "It's never blatant, it's always calls for seemingly good things like extra pluggable points to make sure we don't favor particular solutions. Then it's making sure that any decision is brought to a huge vote by a giant committee that spends weeks arguing about if it's something they should even decide on, etc."
This kind of premature generalisation by committee is what has pulled OpenStack down; a situation from which it is now apparently recovering. CNCF seeks to avoid this, by encouraging projects towards interop but not in mandated ways.
Do you ask projects to support all the implementations or just choose one?
1 - the openstack board has 0 technical input to projects - the way to get things done in OpenStack is still to throw developers at it.
This does somewhat push the balance of power to larger companies - they have the money to employ developers.
2 - we have "community directors" who are elected by the people who actually commit code 3 - definite improvement over the initial setup of openstack, but that is currently changing with the User Committee
One question I would have about this - how are the end user groups requirements put forward? what mechanisms is there to ensure developers work on the defined priorities?
4 - Yup - we have the equivalent with the Technical Committee (the TC in OpenStack slang)
Separation of powers is ++ - but how does that play out when the TOC decides that they want to do something that does not mesh with the boards plans?
The underlying idea here is that a well-run open source project gets plenty of strong direction from actual users, who must be interacted with directly.
There is a still-forming End User Board designed to create a strong forum for some types of User-Project discussion. But overall CNCF will lean towards "voluntary" and not "mandatory" requirements.
When integrating something like Kubernetes with existing systems and processes there will always be opportunity for additional systems and support that aren't easily built in the open. Often times these aren't fun things and so are harder to get critical mass in the open.
The first part of our plan is to help fill usability gaps for Kubernetes in the open. This will grow the pie and expand our standing in the community.
Commercially we will filling gaps unique to large companies around integration with existing systems and managing ops and dev relationships. Specifics here will have to wait until product is ready.
In general we have a value around "Honest Technology". We want to build products that just work and do what it says on the tin.
2. There is usually not a "natural" conflict for companies that started out of open source projects
3. A full kubernetes deploy is already hard
I'm actually quite happy that we will now see Heptio and CoreOS pushing for boxed kubernetes deploys. Multiple entities pushing an open source core is usually a good thing.
No, it's not that cut and dry. Depending on their product, they end up pushing the core to be more pluggable and extensible to make it work better with their product and have places to add secret sauce.
Any company that sells support benefits from more complexity because they compete by gaining expertise. Any company that makes a product for it benefits from fewer features in the open source version.
It is unlike, for example, Influxdata that they own InfluxDB and dont integrate clustering capability into open source edition.
It isn't quite there yet but things are improving on that front all the time and will be significantly better for newbies in a few more versions. In other words, it is getting there, don't worry about it.
It's never blatant, it's always calls for seemingly good things like extra pluggable points to make sure we don't favor particular solutions. Then it's making sure that any decision is brought to a huge vote by a giant committee that spends weeks arguing about if it's something they should even decide on, etc.
Ultimately there are ways to do immense damage to open source communities with small well placed "concerns" that seed doubt. Usually it's a mailing list post (or github issue) with some grand question like "Is X something Kubernetes should really be doing?" followed by bullet points outlining other unrelated issues that "we should be focusing on" and to "leave X up to other tools because the solutions are a matter of taste and therefore there is no right solution."
That is, me with eventually a junior that I'm getting up to speed so he can take my place.
Furthermore, the "run" part is only a fraction of the job, the whole pilgrimage from developers' code to a production-ready container is most of the work, and that's very client-specific.
For example, I will always have a high-paying job at making workflows that help ensuring average developers cannot send bad code in production, that they are reviewed etc.
Kubernetes is great because the "run" part was really the low-hanging fruit. A container is a container, and setting up routing, provisioning etc is very generic and boring.
You can clearly see the diverse number of contributors/companies
Please, have in mind it is (for now) just a proof of concept.
I'll try to publish a more detailed analysis soon. My hypothesis is that the K8s influencer ecosystem is not dominated by any one group or any one company, which is not surprising at this stage.
Some other pointers that give context here:
My blog post on my motivations: https://www.eightypercent.net/post/another-leap-heptio.html
The press release: https://blog.heptio.com/founding-members-of-kubernetes-team-...
Blog post: "Cloud Native Part 1: Definition" https://blog.heptio.com/cloud-native-part-1-definition-716ed...
Guest article (from Craig): "Cloud Native: Service-driven Operations that Save Money, Increase IT Flexibility " http://thenewstack.io/towards-cloud-native-operations/
Just one simple example of many: think of all the software services that Tesla needs to do the fancy stuff that they are doing.
There are many car companies right now trying to catch up, building scalable and resilient services, doing analytics, navigation, M2M stuff, emergency services, car rental, lots of stuff.
Weekly timesheets of course is definitely not the use case for this.
To raise awareness of Docker in Google, I asked Brendan to pull together a demo for our all hands. In a nutshell what he produced was the bones of Kubernetes. I remember looking at it and having a moment, he had built a mini borg cell on VMs. Basically made borg a devops accessible tool, not just a monolithic clustering tool (like Mesos was). When I saw that the product ramifications were obvious, I called Joe over to look and the rest is history.
Well but why not having some more diversity in that area.
Just the title sounds like no one would be working on this angle.
I would love your feedback on the mailing lists or twitter: https://coreso.com/community
Google Cloud does offer hosted k8s though.
As to Heptio’s name, Beda explained: “When we were helping create Kubernetes, we pitched it as ‘Seven of Nine,’ a ‘Star Trek: Voyager’ character who’s a former Borg drone. That was a reference to the Borg, a code name for Google’s internal version of Kubernetes, the thing that runs its search and apps and ads. It’s total geek culture. We wanted a friendlier Borg. That name turned into ‘Project Seven.’ When we went public with Kubernetes, we didn’t want to lose track of the ‘seven.’ The Kubernetes logo has seven sides. ‘Hept’ is the Greek prefix for ‘seven,’ and it’s a way to pull the ‘seven’ through.”
[1] http://www.geekwire.com/2016/ever-come-kooky-kubernetes-name...
I would describe this as a case of 'domain based naming' (i.e. we could get the domain name, it was uncontested space).
I hope to create something positive out of it, if we do it up right hopefully the company character will dominate the name.