If you, as an AWS customer, decide to fully embrace AWS lock-in, then why would you run this yourself on an EC2 instance instead of running ECS or EKS? If you're trying to avoid AWS lock-in, why would you choose an OS that's locking you into AWS Systems Manager and Amazon Linux 2 for debugging needs?
First Bottlerocket is not Amazon Linux 2, it is its own minimal operating system, with most components built from the ground up in Rust. This is totally different than the Amazon Linux 2 you may be familiar with (and most other operating systems for that matter). Bottlerocket is optimized for running containers with high security isolation. The host OS is extremely minimal, it does not come with bash, an interpreter, ssh, or anything beyond the system basics needed to run containers. In fact it uses an immutable root filesystem. You aren't intended to run or install things directly on the host at all.
Everything that is installed and runs on Bottlerocket runs as containers, which are kept isolated from each other and the host with best practice security features such as SELinux. For example you can connect to a container on the host via AWS Systems Manager, or you can optionally enable a container that lets you connect to it via SSH. Once again the thing you are connecting to is the container on the host though, not directly to the host.
For this initial release of Bottlerocket we are focusing on providing image variants that are prepackaged with the requirements to serve as underlying container hosts for an ECS or EKS cluster. However we also intend Bottlerocket to eventually be something that anyone can use, anywhere, even outside of AWS, if they wanted to benefit from the secure by default, container first design of Bottlerocket.
You can read more about the security features of Bottlerocket here: https://github.com/bottlerocket-os/bottlerocket/blob/develop...
And you can find a bit more of the charter / goals for the project here: https://github.com/bottlerocket-os/bottlerocket/blob/develop...
We do have firecracker-containerd (https://github.com/firecracker-microvm/firecracker-container...) which is designed to allow the containerd runtime to launch containers as microVMs via Firecracker, with that additional layer of isolation via the KVM hypervisor. This stack is not currently fully compatible with K8s or ECS though, so it is not implemented using that approach yet. Rather Bottlerocket is built as a progressive improvement on the current state of container hosts, which is that many people are running all their containers on their hosts without any strong security hardening at all.
I think from the similar naming scheme of Firecracker and Bottlerocket you can already see the pieces of the puzzle that are in progress and the future potential though.
So if I have a heterogeneous collection of servers - I could install Debian, and run docker on Debian. It sounds like bottlerocket would more comfortably run on top of a hypervisor abstracting away the actual hw a bit? Eg on top of xen, kvm or VMware?
Obviously the Linux kernel can be made to run on a toaster, but maybe bottlerocket isn't ideal for that purpose?
Bottlerocket has a GNU userland like many other distros. It is just one that is stripped down of many things including removal of interpreters, shells, and package managers.
If you want to explore more deeply, you can enable the admin container and jump into a shell on the host[1] to look at the filesystem and see what Bottlerocket's userspace looks like up close and personal. You can also see a bit more of this debugging/exploration tooling explained in an AWS Partner Blog[2].
[1] https://github.com/bottlerocket-os/bottlerocket#admin-contai... [2] https://aws.amazon.com/blogs/apn/getting-started-with-bottle...
I'd also recommend reading the ECS quickstart in the Bottlerocket repo: https://github.com/bottlerocket-os/bottlerocket/blob/develop...
Sounds exactly like Qubes OS [0], but the latter is probably much more secure (based on Xen and VT-d). So in comparison Bottlerocket seems like trading some of security for performance.
Qubes uses VM's. Subgraph OS is another option. It tries to achieve the same thing with containers, but their latest release is pretty old--not sure how alive that project is.
There are a variety of options currently, including one from Amazon, but Bottlerocket seems designed to be a next-generation OS for container instances. It's extremely minimal, and designed to be as secure as possible, with transactional automatic updates.
Ideally, I would just deploy Bottlerocket as the underlying OS on my ECS Cluster, and then never have to actually do any management of the Instance OSes whatsoever. I would deploy containers on top of my ECS Cluster, Bottlerocket would keep itself up to date and secure, and my containers would live happily ever after. It would hopefully feel more similar to using Fargate than not, except without paying the higher price for Fargate, and having access to the wider variety of hardware configurations available to regular ECS.
Amazon has specifically said this in the Bottlerocket repo:
> Bottlerocket is architected such that different cloud environments and container orchestrators can be supported in the future.
It's mainly useful in concert with ECS or EKS right now, but it is architected to be useful in other places as well.
I'm excited about Bottlerocket as a project, and I'm glad it's open source instead of just an opaque AMI that you can use on ECS or EKS.
Right, this is the big question. If a community forms to support Bottlerocket off of AWS, then that's one thing, but until then, it basically just seems like a better option for ECS or EKS.
>> <snip>
> If a community forms to support Bottlerocket off of AWS, then that's one thing, but until then, it basically just seems like a better option for ECS or EKS.
I'm failing to see why that's a problem or a source of confusion. There's a clearly defined market of "who they built it for", and there's a community option to expand Bottlerocket's target market.
The code is open source, so anyone could fork Bottlerocket and modify it to suit their needs immediately. Will Bottlerocket see success beyond ECS and EKS? Only time will tell... literally no one knows yet.
Forgetting Fargate there?
>> It would hopefully feel more similar to using Fargate than not, except without paying the higher price for Fargate, and having access to the wider variety of hardware configurations available to regular ECS.
If you're using an ECS Fargate Cluster or EKS Fargate Cluster, I don't consider those the same as an ECS Cluster or EKS Cluster. Unfortunately, there's no specific term commonly used for non-Fargate Clusters that I know of. AWS offers ECS Clusters and ECS Fargate Clusters.
If you were legitimately confused by my comment, I’m sorry. I could have said "non-Fargate" repeatedly, if it would have helped, but I thought the context made things clear, especially with the additional explicit mention of Fargate as a separate thing.
Anytime AWS finds many of its customers using something outside of AWS, they want to build it in-house. AWS customers were using CoreOS a lot, hence this.