Thanks Brian! Feel free to drop some notes our way. I'd love to see this turn into an open ecosystem of harness tools that can be applied to a whole bunch of situations.
Hope you are well!
850 karma · joined November 13, 2009
I previously worked for Google in Seattle. I started Google Compute Engine and helped start Kubernetes/Google Container Engine. At Google I've worked on Google Talk, VoIP, and ad suggestion systems. Before that I worked at Microsoft on IE and WPF/Avalon.
I'm a dad of two and my wife is a doc.
Blog: https://www.eightypercent.net
[ my public key: https://keybase.io/jbeda; my proof: https://keybase.io/jbeda/sigs/kYkVr97UkWrL9KyWVJPi74ElW-neJn5vPgm8YW0qeGY ]
Hope you are well!
3 things come together:
1. A Schema for a domain model. The canonical definition is a YAML file.
2. Tooling to lint and render a domain model. It can be rendered to Markdown for human (or agent) consumption.
3. A set of skills for creating and updating a domain model. No one wants to author YAML directly so let your agent do it for you!
The pattern that you see often with PaaS is that they work great until they don’t. When you want to do something that doesn’t fit the patterns that the PaaS was built for you are out of luck and pretty much have to drop down to VMs.
We aimed to have k8s be something that was in the middle. Generic enough to work with a large number of workloads while providing significant advantages above raw VMs (and a level of consistency across infra providers).
The tradeoff is that it is more complicated than the typical PaaS. That is the tradeoff. The goal then was to build kubernetes to be built upon. We never claimed to be solving the whole problem in one system.
This is similar, in some ways, to the design of Mesos. But to do something new with Mesos requires an entirely different “framework” that exposes its own API. You see this with things like Marathon or Aurora or Chronos. While the underlying scheduler was shared across these there was no common API and so it was difficult for users to mix/match across these.
By focusing on a consistent IaaS-ish API for k8s (and eventually, with CRDs, building extension mechanisms) we aimed to create an ecosystem of things building on k8s.
Whereas PaaS are often a framework where you put your code in and a lot of decisions were made for you, I always viewed k8s as a toolkit where you get a lot of powertools that you can use in various ways. It is a very different philosophy.
12 factor and Docker obviously play a big part of this. And those built on experience and systems that came before them. We are all standing on the shoulders of those that came before us.
For me, this API design experience was informed by my time at Microsoft. Things like VB and IE/web were very much, at that time a framework (the web is turning into a toolkit over time now). I saw, multiple times, how those things were super useful but were limited. Things like the CLR and the C# ecosystem evolved the VB model to be much more of a toolkit.
We see this also in the Java world. I’m not an expert there, but the move from app servers to things like Spring Boot also show the shift from framework to toolkit.
AWS itself is a toolkit. And IMO, the difference between something like lambda/faas and a PaaS is that lambda/faas only works when it is in context of the larger toolkit that is IaaS cloud.
(I’m one of the founders of k8s and ended up with quite a bit of screen time in the documentary. My only regret is that more people that were so important in the k8s community couldn’t be featured!)
One thing not mentioned here is the history of the "Byte Order Mark" (BOM) in unicode.
(Not an expert here but my understanding having lived it.)
You see, given UCS-2, there are 2 ways to encode any codepoint -- either big endian or little endian. The idea was then to create a codepoint (U+FEFF) that you could put at the start of a text stream that would signify what order the file was encoded in.
Wikipedia page: https://en.m.wikipedia.org/wiki/Byte_order_mark
This then got overloaded. When loading a legacy text format often times there is the difficulty of figuring out the code page to use. When applied to HTML, there are a bunch of ways to do it and they don't always work. There are things like charset meta tags (but you have to parse enough HTML to find it and then re-start the decode/parse). But often times even that was wrong. Browsers used to (and still do?) have an "autodetect" mode where it would try divine the codepage based on content. This is all in the name of "be liberal in what you expect".
Enter UTF-8. How can you tell if a doc is US ASCII or UTF-8 if there are no other indications in the content? How does this apply to regular old text files? Well, the answer is to use the BOM. Encode it in UTF-8 and put it at the start of the text file.
But often times people want to treat simple UTF-8 as ASCII and you end up with a high value codepoint in what would otherwise be an ASCII document. And everyone curses it.
Having the BOM littered everywhere doesn't seem to be as much of a problem not as it used to be. I think a lot of programs stopped putting it in and a lot of other programs talk UTF-8 and deal with it silently. Still something to be aware of though.
The techniques were used in many contexts at the time.
A ReplicaSet (RCs are old and deprecated) manages a set of Pods. It has a template and a target replica count.
A Deployment manages ReplicaSets across versions/upgrades. When you change a deployment it creates a new ReplicaSet and does a rolling upgrade from new to old version by tweaking replica counts in new and old ReplicaSets.
Another goal here is to have something that is easy to embed/repurpose. ruby is pretty heavyweight from that point of view.
While jsonnet is a C program right now (https://github.com/google/jsonnet) we are looking at picking up work on a go port so it is easier to integrate with the kubernetes universe.
Early work was done in AMD and we had an a1 series pre launch.
IIRC, g is for Google or Generic where there are no promises on architecture. And f was for fractional.
I've been out of Google for 2+ years now so reality may have drifted from that original scheme.
Heptio is a new (funded, 8.5M from Accel and Madrona) Seattle based startup in the systems infrastructure space. Our mission is to make mainstream enterprise more efficient in how they build teams and use physical resources. Our approach is based on experiences at Google and other first tier software companies.
While at Google, the Heptio founders (Craig McLuckie and Joe Beda) started Kubernetes. Kubernetes enables IT organizations to offer API driven services to their organization and drive up resource utilization. It is also a fast-growing open source community and the "seed crystal" for a groundbreaking set of technologies.
We are looking for extraordinary engineering talent to continue the mission to bring Kubernetes to enterprise developers and beyond.
What you'll be doing...
* Working with and growing your stature in the Kubernetes open source community.
* Building distributed server side systems and developer tooling in Go.
* Listening to users in order to make complex technologies usable.
* Concentrating on user experience, especially for other engineers.
It would be nice if you've worked on or with technologies that make up the new "cloud native" stack. This would include Kubernetes, Docker, Mesosphere, Go.
Important: we are an equal opportunity employer and value diversity at our company. We’re strongly committed to providing equal employment opportunity for all employees and all applicants for employment. We want to embody the change we need in our industry.
We are open to hiring remote but want to make sure we do it right so are being careful. Having someone that has successful experience working remote and can help get us on the right path would be highly valued.
Email me at joe@heptio.com if you are interested.
What Brendan, Craig and I did was (a) meld those ideas with the external-to-Google world (including Docker), (b) motivate releasing it as Open Source (not the default model for Google TI) and (c) work to seed a really open community that we wanted to participate in.
Soon after we started some other folks from around Google and beyond joined in. From Google, those include Ville Aikas (not super active these days, doing other stuff at Google), Tim Hockin, Brian Grant, Dawn Chen, Daniel Smith. Most of these folks had worked on Borg and/or Omega. From outside Kelsey Hightower got involved very early (long before Kubernetes was a thing for CoreOS) and the folks from RedHat (led by Clayton Coleman). I'm sure I'm forgetting folks but hopefully that gives some context.
Some other pointers that give context here:
My blog post on my motivations: https://www.eightypercent.net/post/another-leap-heptio.html
The press release: https://blog.heptio.com/founding-members-of-kubernetes-team-...
Blog post: "Cloud Native Part 1: Definition" https://blog.heptio.com/cloud-native-part-1-definition-716ed...
Guest article (from Craig): "Cloud Native: Service-driven Operations that Save Money, Increase IT Flexibility " http://thenewstack.io/towards-cloud-native-operations/
When integrating something like Kubernetes with existing systems and processes there will always be opportunity for additional systems and support that aren't easily built in the open. Often times these aren't fun things and so are harder to get critical mass in the open.
The first part of our plan is to help fill usability gaps for Kubernetes in the open. This will grow the pie and expand our standing in the community.
Commercially we will filling gaps unique to large companies around integration with existing systems and managing ops and dev relationships. Specifics here will have to wait until product is ready.
In general we have a value around "Honest Technology". We want to build products that just work and do what it says on the tin.
Disclaimer: I co-founded Kubernetes and help to coordinate the k8s Scalability SIG, although I'm no longer at Google. I didn't see this before it was published, though.
Google Compute Engine itself was difficult to name. There were those that were pushing for Google Compute Cluster. But I veto'd as the TLA would have been GCC or GC2. Both would have been awful.
Naming is hard.