Flatcar Container Linux
flatcar.org
flatcar.org
CoreOS "just the containers, ma'am" plus ChromeOS "set in stone" OS (but updateable) approach was so obviously good once you started using it that it was just begging to be picked up again.
Microsoft has done quite a lot to rehabilitate their image, a decade ago few would be so quick to defend them.
I don't want to sound as a anti-MS fanboy, but that's the standard procedure when you get to the point "If you can't beat them...". Microsoft isn't to be trusted just because they're a business, but because like many others they're after profits above anything else. Just look at how they turned a potentially very decent OS (Windows 7) into worse privacy nightmare at every new version. I simply don't want them to mess with Linux.
[0] https://en.wikipedia.org/wiki/Embrace,_extend,_and_extinguis...
I can’t fathom how tech professionals can accept this kind of lack of control of their operating system.
Free software is an ideology and philosophy, not just a license.
We are so serious about this that I almost got a contract canceled because it looked like they were running on AWS, and we got them to switch to Azure.
Many of the sibling comments were coming from the maintaining freedom perspective, which I personally appreciate as well. I wanted to chime in that this information is also helpful from a corporate/competitive perspective.
Since then, the project has opened up and has become fully community driven - releases, features, stewardship, everything. To further underline independent stewardship, Flatcar was recently proposed to join the CNCF as an incubating project: https://github.com/cncf/toc/pull/991 .
These days we call them intermodal frames or an intermodal chassis. It can go from rail to cargo ship to truck to plane, and its all highly standardized and scalable :) CSX even uses them to test their robotic conductor/ automated locomotive technology
[0]https://www.flatcar.org/faq#what-is-the-significance-of-the-...
Cars that carry containers are now mostly referred to as well cars (https://en.wikipedia.org/wiki/Well_car)
An intermodal chassis or frame usually refers to the highway version used to pull a container behind a truck (https://en.wikipedia.org/wiki/Container_chassis)
We have contributing members from many FOSS projects gravitating around image-based Linux, including quite a few distros.
The kinfolk folks also run the update server to perform auto updates.
//edit
Continuation wouldn’t be a wrong word either as it all coincided with the Red Hat acquisition. But I guess “fork” is correct since it’s under a different name.
Like what?
Fedora CoreOS and RHEL CoreOS use OSTree [1], no shade on OSTree, it works well, but I slept better with A/B partitions, I just find them easier to reason about.
IIRC RHEL now has tooling for automatic rebooting into the old version when checks are failed. Not sure how hard that is to do with Fedora CoreOS.
Guess I’m not sure what functions are missing in these implementations that would cause you to lose sleep.
(Of course, you can lose the data partition.)
We were facing a lot of controversy behind Flatcar already - there's no clear Licensing policy for their "Fully OpenSource" release. Kinvolk should've prepped at least a License matrix of what's been hacked out of CoreOS - what is actually behind their "Pro Features" and how exactly it is "Fully Open Source".
"Flatcar Linux Pro Features", in terms of "Linux Kernel Optimization', "GPU instances support" with "Accelerated Networking", are Not So Pro, and are something that everyone use on a daily basis.
So, just from pure Support Monetization perspective, it's impossible to use Flatcar Linux reliably for Free (mic drop).
I'm the initiator of the Flatcar Container Linux project and former CEO of Kinvolk. Thus, I'm rather knowledgeable about the project and was involved in most decisions.
The controversy you speak of is very new to me. If you could point to any references, I'd love to be aware of them.
Firstly, there was nothing "hacked" out of CoreOS. Flatcar is literally the CoreOS Container Linux repos forked and carried on as is. Once the CoreOS EOL was reached we started updating the stale packages. That's it. Any further updates are what any distro would do in the course of maintenance to remain modern and relevant.
Secondly, anything that was previously termed the "Pro" version is now just available in the standard version. So there is no difference. To my knowledge, the project doesn't even produce any Pro versions any longer and I don't think there are even any references to it in our docs. But even when we did have a Pro version, all the work we did was done in the open and was in our source repositories. We just didn't release public builds of those.
Unlike CoreOS, we also developed* and open sourced the update server. It's called Nebraska and available here under an Apache license. https://github.com/kinvolk/nebraska
With regard to a license matrix, you can find all licenses for each release in the respective release directory. For example this one: https://stable.release.flatcar-linux.net/amd64-usr/current/f...
If you do find anything that is not 100% open source, let me know and I'll follow up to make sure that's corrected.
I'm happy your excited about your project. But I think you'll fine it's better in the open source space to compete on merit and form relationships rather than tear down other projects and the work of the people behind the projects.
* based on the Core Roller project: https://github.com/coreroller/coreroller
My point is that Flatcar positioning is too cryptic and ambiguous for a lot of folks. People don't really get why they should pay for it, and that's a customer reachability issue, - a marketing problem, not an engineering one.
I'm not saying that no one should use it, or it's a bad product, it's just takes too much time and effort to get into it, to understand how the pieces of the puzzle are tied together, and why. It's really great that you've picked up EOL'ed CoreOS services and developed your own. A visual component diagram, with a couple of license icons, and a simple graphical bare metal installer would've been really nice.
> you can find all licenses for each release...
Packages SBOM doesn't give a full answer regarding architecture, existing priorities (if there's any) and project structure: what had been adopted from CoreOS and why, and what had been developed from scratch, and why ?
My primary complaint regarding "Pro" feature set, and further monetization, is that literally every growing business, with a cloud hosting provider, would target those, which makes them a necessity, not an extra option to choose from.
FlatCar should've developed something innovative to differentiate the market a bit, and monetize complex enterprise deployments, instead of sabotaging onboarding for the new folks (customer reachability), with a flat fee and "unpublished builds", of a supposedly free Open Source product. At least that's the feedback I've been able to gather on my own, so far.
> We just didn't release public builds of those.
So, if I get it right, FlatCar monetization may crumble with a "Community build" that will package all the "Pro Features" and let people use 'em for free ?
Not trying to devalue anything, but "just a viable CoreOS fork" doesn't make things magically self sustainable. It's all about the extra services that you can put on top, and real engineering problems that you may solve, for those who are willing to pay.
People expect FlatCar Linux to be Free.
In last contact I had with Flatcar (before we moved to Bottlerocket) one could still use cloud-init yaml even with Flatcar so that's what I'd recommend instead of trying to read through the unbounded number of github issues trying to track down why simple things don't work with ignition
As for "what advantages Flatcar provides," the answer is always "in comparison to what?" I haven't run a general-purpose OS in production in over 8 years, and I don't intend to ever go back to having a mutable OS. There are just too many opportunities for randos to "just ssh in and do this one thing" and then I have to spend hours tracking down why some machines behave different from their peers
Could you elaborate on this? I helped [1] create Ignition with Alex Crawford [0], its original author, while @ CoreOS in ~2015. To call what we were making then a "tire fire" seems comically hyperbolic for what was such a rudimentary bootstrapping tool, and generally unexpected for something we'd ship.
Has its scope grown out of control since those early days, or has it been mismanaged? We're coming up on a decade since my involvement...
[0] https://github.com/coreos/ignition/commits?author=crawford
[1] https://github.com/coreos/ignition/commits?author=vcaputo
My biggest heartburn is that I have[1] to compile a boot configuration document. Some of that may be a concession to Ignition being inexplicably written in JSON, but it's possible the transpiler does some massaging and restructuring of keys, I dunno, I didn't want to compile a boot document
This is how one creates a file in Ignition: https://coreos.github.io/ignition/examples/#create-files-on-...
Contrast that with https://cloudinit.readthedocs.io/en/20.1/topics/examples.htm...
I grant you that https://coreos.github.io/butane/examples/#files may be the example of pre-transpilation using the `inline:` structure but for my use case of authoring that UserData dynamically, I don't have a `!TranspileUsingCuteNameOfTheDay` in CloudFormation
And since I haven't had my hands on the Flatcar ecosystem in a few years, I'm not going to have access to the very specific pain points of "well, this cutesy named thing is broken because it depends on this other cutesy named thing" but what it boils down to is cloud-init behaves sanely, has been around forever, and can be authored easily in a bunch of different circumstances
fn-1: yes, I can appreciate it's "just a JSON document" but no reasonable person is going to author JSON string literals containing data: URIs for contents
So, anyway, I actually did dig up a concrete example of my experience with it, and I cannot link to the "Additional information" section but that is both why I think the thing was a mess and also why the Miroservices YT joke resonated: https://github.com/flatcar/Flatcar/issues/220
I think the CoreOS boot strategy was decomposed into a bunch of different executables, each responsible for doing their own little slice of the world. Maybe it drew inspiration from systemd in that way. But, just like my real life experience with microservices, it requires keeping a bunch of different projects and their upgrade paths in ones head, knowing their disparate config formats, and when one of them inevitably has a bug, understanding how to troubleshoot what went wrong with the system as a whole
And, again in trying to be reasonable in this discussion[1] I do also understand why one would opt for the data URI, given how much of the rest of Ignition loads content from URLs. I don't believe cloud-init has that remote content paradigm baked into it nearly the same way, so I hear you about that.
And yes, my belief is that JSON is a data-exchange format from _computer to computer_ and making people write them is a poor DX choice, IN MY OPINION. And, to reiterate, I know that CoreOS's perspective is that it is a computer-to-computer transmission from the transpiler-project-o-the-day to the Ignition binary, but that is predicated on one having access to that transpiler binary in all cases, which is quite different from the problem that cloud-init is trying to solve
fn-1: I'm sorry you got hurt by my "tire fire" outburst, and that evidently derailed this whole interaction, but it was my experience
Maybe you wouldn’t be if you didn’t use phrases like “tire fire” at other people’s work?
It sounds like Ignition isn’t your cup of tea, which is perfectly fine and few people could take issue with that. Tire fire is unnecessarily caustic and doesn’t really improve the discussion.
I like it a lot more than cloud-init in this scenario.
- there is a limited amount of retries (5?), so a network hiccup can easily cause the download to fail
- no control over which responses are fatal, so if you boot a VM with a config that references a URL that's still 404 due to async nature of something, then only a single attempt is made.
The company I work for uses Flatcar exclusively, but we don't use config chaining due to these limitations, and we have moved all file downloads to the shell scripts embedded into the config.
The benefit is reproducible deployments: Any deployment of a given configuration will reproduce with the same end result. Configuration is kept outside of the instance and does not drift. This is guaranteed across releases / versions (something we very explicitly and thoroughly test, for each of our releases).
(Perceived) drawback can be that it's a bit of a paradigm shift for operators. While Ignition eliminates the need to meddle with instances after provisioning, it can be a bit of a learning curve to produce a node's full config beforehand.
I think Ubuntu is a popular base image in Docker because Ubuntu is popular for engineers to use on their desktops
Debian usually passes for Ubuntu, since the package names are mostly the same.
So either you do all your development in containers, or you have drift between the dev environment and the cloud.
Ubuntu passes for Debian, you mean. Debian has been around much, much longer than Ubuntu and set the standards Ubuntu got for free and commercialized.
What I mean is that people just hack on stuff on their desktop. They install Python NumPy or Rails or whatever. Like gigabytes of dependencies.
And then they want to deploy.
Is it easy to do that with Flatcar Container Linux? How do I do it?
Or is it easier to make a Docker file that does apt-get install X and pip install Y ?
You can do that with Ubuntu as a base image, and Debian usually "just works" if you're on Ubuntu. (I'm doing this with our Debian CI images)
Not saying you should always take the easier path, but I also think it's a good idea to minimize the differences between the dev environment and production
Flatcar releases are tied to git commits in our distro automation repo; so to develop for Flatcar, one would:
1. clone the distro automation repo
2. check out the release version one wants to develop for (tag or branch)
3. starts the SDK container via a wrapper script in the repo
This enables Flatcar development across all distros that run git, and docker or podman (it even works on WSL if one is inclined to use it that way).
More information here: https://www.flatcar.org/docs/latest/reference/developer-guid...
What happened was old school CoreOS was A/B partition based: https://github.com/coreos/docs/blob/master/os/sdk-disk-parti...
My memory is hazy but here's how I remember it: After Red Hat acquired CoreOS they announced their intent to rebase the entire thing around rpm-ostree, which is the CoreOS people know today: https://coreos.github.io/rpm-ostree/
At the time there was some anxiety in the community as to what would happen, as there was no direct upgrade path from old CoreOS to new CoreOS. Theoretically if we all believed the kool-aid we were drinking it's just a redeploy, no pets!
Kinvolk came along, forked it, and made Flatcar Linux, which kept the A/B partitioning system, and more crucially, let you just change a config file and all your old CoreOS nodes would just move to Flatcar and then you were good to go. So now if you wanted to stay on the system you were comfortable with you could just use Flatcar. If the composability of rpm-ostree attracted you then new CoreOS have you covered. Red Hat deserves a hat tip here because in their documentation/blog they explicitly mentioned Flatcar as an option for people who wanted to stick with what they know, which I thought was cool and how I discovered it!
Later on Microsoft acquired Kinvolk and and then people raised eyebrows. I have not checked in a while but the folks involved continued to do their thing and run it like a good OSS project, hold public meetings, all that stuff. Microsoft gets a good hat tip here.
I use both and they're both high quality.
> The Kinvolk Update Service allows for defining instance groups, assigning update channels and controlling the frequency, time of day and rate of updates.
What is this? From a quick glance I don't see it mentioned elsewhere. A hosted service? Something with a different license?
Our team hosts the public server, but any Flatcar user can run Nebraska themselves and point their nodes to that.
What's the upside of using flatcar or coreos compared to (say) ubuntu server?
Think of it as rather strong separation between operating system and user-defined workloads (containerised apps, Kubernetes, etc.).
For a (very) lightweight hands-on intro check out https://www.flatcar.org/docs/latest/installing/ .