Thank you. I understand it's hard to concisely define what Octelium is because it is designed as a unified/generic secure/zero trust access platform, a term that almost nobody would relate to. It's more of a generic Kubernetes-like architecture/infrastructure for zero trust secure access that can fit many different use cases (i.e. human to workload and workload to workload environments). Well, it can be used as a typical WireGuard/QUIC-based remote access/corporate VPN. It can be used as a ZTNA/BeyondCorp platform with identity-based, L7 aware, context-aware ABAC via policy-as-code with CEL and OPA where you can control access at layer-7 (e.g. HTTP request headers, serialized JSON body content, etc...). It can also be used as an ngrok alternative (both secure access via OIDC/SAML/GitHub IdP as well as anonymously which can fit for hosting, testing APIs, etc...). It can also deploy your containerized resources and automatically provide client-based/clientless secure access to them (kinda like a PaaS) and it does provide dynamic configuration and routing to upstreams via policy-as-code (e.g. route to different API versions, use different SSH credentials, different API keys, different postgres user/password based on identity/context, etc....). It can also fit as an API/AI gateway and a scalable infrastructure for MCP architectures/meshes. Therefore, it's not really a ZTNA/VPN in the rigid sense, it's a more generic platform where what it does to secure/remote access is similar to what Kuberentes does for containers.
I believe there is more to the problem than just laziness from the relying party side. The problem is what do you verify against? It's not as easy as in, for example, TLS handshake where browsers/HTTP clients have well-defined root CAs to verify the provided server cert against. It's easier for security keys (e.g. yubikeys) where you can just use the FIDO MDS service but there are many other uncovered use cases such as TPMs and browser/password manager generated passkeys. The current WebAuthn/passkey registration process is practically TOFU at this point which is funny since this by itself negates the entire rationale behind hardware-based phishing resistant authenticators. And since you mentioned Apple, as far as I know and I could be wrong for their newer products, it doesn't even provide real attestation statements to begin with like in Android or Windows Hello devices. The entire standard is currently practically based on TOFU "trust me bro, I am a hardware-based generated key lol"-tier registration process.
Great effort. I honestly doubt that any B2C or even the vast majority of B2B relying parties do verification of attestation statements during registration which means the relying party never really knows whether the authenticator's public key is actually generated by a real security key, TPM, etc... or just generated by software. I guess FIDO MDS can currently act as a solution to some degree but it might possibly break passkeys legitimately generated by software such as password managers, not to mention that when it comes to TPMs for example, the process is messy and unpredictable. Many TPMs don't even send their own entire x5c because of size and storage limitations.
Apart from 1 and 3, probably everything else can be added today if the people in charge have the will to do that, and that's assuming that I am right and these points are actually that important to be standardized. However the big enterprise-tier money in Kubernetes is made from dumbing down the official k8s interfaces especially those related to access control (e.g. k8s own NetworkPolicy compared to Istio's access control related resources).
I never said that CRDs are tied to Golang, I said that the experience of compiling CRDs, back then gen-controller or whatever is being used these days, to Golang types was simply ugly partly due to the flaws of the language itself. What I mean is that gRPC can standardize the process of compiling both k8s own resource definitions as well as CRDs to make the process of developing k8s controllers in any language simply much easier. However this will probably complicate the logic of the API server trying to understand and decode the binary-based protobuf resource serialized representations compared to the current text-based JSON representations.
I haven't used CRDs myself for a few years now (probably since 2021), but I still remember developing CRDs was an ugly and hairy experience to say the least, partly due to the flaws of Golang itself (e.g. no traits like in Rust, no macros, no enums, etc...). With protobuf you can easily compile your definitions to any language with clear enum, oneof implementations, you can use the standard protobuf libraries to do deepCopy, merge, etc... for you and you can also add basic validations in the protobuf definitions and so on. gRPC/protobuf will basically allow you to develop k8s controllers very easily in any language.
You can very easily build and serialize/deserialize HCL, JSON, YAML or whatever you can come up with outside Kubernetes from the client-side itself (e.g. kubectl). This has actually nothing to do with Kubernetes itself at all.
I would say k8s 2.0 needs: 1. gRPC/proto3-based APIs to make controlling k8s clusters easier using any programming language not just practically Golang as is the case currently and this can even make dealing with k8s controllers easier and more manageable, even though it admittedly might actually complicates things at the API server-side when it comes CRDs. 2. PostgreSQL or pluggable storage backend by default instead of etcd. 3. Clear identity-based, L7-aware ABAC-based access control interface that can be implemented by CNIs for example. 4. Applying userns by default 5. Easier pluggable per-pod CRI system where microVMs and container-based runtimes can easily co-exist based on the workload type.
Great work. I've always wondered if WASM could actually be used as a more generic alternative alternative to eBPF where you could actually do more complex logic than the average eBPF program at the kernel level and still have customized and restricted access to the kernel APIs and data structures like you're actually developing a typical kernel module.
Thank you. The harsh reality is that I've been undecided on the license for years while working the project (the initial private repo had it all AGPLv3 as opposed to only the Cluster side in this public repo). The reason for choosing it AGPL for Cluster-side/Apache for client-side is that I am basically working on it solo and I have neither the funding nor the press to prove that that my work is the original work of any derivative proprietary work. Also the license issue is very weird to say the least, If I release it fully Apache, it might mean to some that I am probably pulling the rug at some point like many others who did even if I have no intention to do so. If I release it fully AGPL, some might see this as not really true FOSS regardless of the fact that it actually is. It's truly a bizarre situation releasing open source these days with all the bait-and-switch VC-backed "open source" software and all the licensing changes that happened since 2020 for many of the most popular open source projects. Also, I never intended for Octelium to become a SaaS product and the architecture itself proves that. This project is more akin to Kubernetes and cilium (without the funding part!) that the idea of any company self-hosting it in anyway has no conflicts with me since that's what I intended to build in the first place (as opposed to releasing a "demo" FOSS project while forcing you to move to a separate, fully functional SaaS product).
Thank you. Actually one of the very hardest things for me working on Octelium is basically how to describe it concisely and clearly and I still can't say that I have an answer, that's why I prefer to describe it as a "unified secure/zero trust access" platform. It's a ZTNA platform but not in the typical sense, it's also a remote-access VPN but actually works via identity-aware proxies to control access at L-7 instead of at L-3 like in VPNs. It's BeyondCorp but actually supports client-less access for both humans via their browsers and SSO but it also supports client-less access for workloads via OAuth2 client credential flows and standard bearer authentication which makes it relevant for any workload written in any language to access all your HTTP-based Services via a single bearer access token without being aware of the Cluster's existence at all. And it's also a deployment platform that enables you to deploy and scale any containerized application, HTTP-based or not, and instantly provide secure client-based/client-less access to it via your policies or even completely expose it to anonymous access like it's a hosting platform if you wish.
I would argue that dev containers are more complicated than CI even though they share many of the challenges (e.g. devcontainers might need to load 10s or 100s of GBs to start and are write heavy). I would also argue that userns/rootless containers provide "enough" isolation when it comes to isolating CPU/memory/networking as well as access to the host's syscalls if you're careful enough; however when it comes to storage (e.g. max disk size that a container can use and write to, max opened files, completely hiding the host's fs from the container's, etc...), it's unfortunately still extremely limited,fs-dependent for some features, even though modern solutions (e.g. vDPA and ublk) can be used to fix that and virtualize the storage for containers.
Firecracker is more comparable to container runtimes than to orchestrators such as K8s. You still need an orchestrator to schedule, manage and garbage-collect all your uVMs on top of your infrastructure exactly like you would do with containers via k8s. In other words, you will probably have to either use k8s or build your own k8s to run "supervisor" containers/processes that launch uVMs which in turn launch the customer dev containers.
I've worked on something similar to gitpod in a slightly different context that's part of a much bigger personal project related to secure remote access that I've actually spent a few years building now and hope to open source in a few months from now. While I agree on many of the points in the article, I just don't understand how using micro VMs by itself replaces K8s unless they actually start building their own K8s that orchestrates their micro VMs (as opposed to containers in the case of k8s) ending up with the same thing basically when k8s itself can be used to orchestrate the outer containers that run the micro VMs used to run the dev containers. Yes, k8s has many challenges when it comes to nesting containers, cgroups, creating rootless containers inside the outer k8s containers and other stuff such as multi-region scaling, but actually the biggest challenge that I've faced so far isn't related to networkPolicies or cgroups but is actually by far related to storage, both when it comes to (lazily) pulling big OCI images which are extremely unready to be used for dev containers whose sizes are typically in the GBs or 10s of GBs as well as also when it comes to storage virtualization over the underlying k8s node storage. There are serious attempts to accelerate image pulling (e.g. Nydus) but such solutions would still probably be needed whether you use micro VMs or rootless/userns containers in order to load and run your dev containers.