Clusters become personal (like PCs did)
aranya.tech
aranya.tech
Maybe they’re assuming some massive amount of compute will be necessary for future tasks? Self hosted LLMs? I’m currently finding it difficult to come up with more uses for my vps beyond hosting trillium and some personal applications I’ve made
I have run Linux laptops before. After running it for five years, I came to the conclusion that it did not make a good laptop for my use-case. Poor suspend-resume support, poor wireless networking support means I can not just pick and go. (And no one has yet to replicate Apple’s trackpad experience). So yes, I run Apple laptop with MacOS and use my TUI tools, sometimes with Linux running in an VM, sometime remotely to a full headless VM with my full dev suite via mosh because I use cli and TUI for dev.
Your turn. You still have not defined “poor man’s Linux”.
Uptime, self healing, reproducibility, separating the system from app. There's probably a half dozen more.
K8s comes with resource consumption tax certainly but for anything beyond the trivial it's usually justified.
> Separate VM's for different apps works well for isolation
Sounds inefficient along with a lot more work doing the plumbing than simply writing a 100 lines of yaml.
https://commaok.xyz/ai/just-in-time-software/
I mean, I don't do that, but I'll type a prompt.
let me draw this out the way i've been playing with:
a classic vm exists, and supports kvm — this means you can run stuff like firecracker in there
an ssh server runs on this vm, and when you connect to it, you're dropped into a repl/tui where you can list existing microvms, create new ones, or destroy existing ones, and, of particular use, you can attach to one.
as an added nicety, if you connect with `ssh user+dev@example.com`, your connection skips the management interface and you are dropped into the `dev` machine — if it didn't yet exist, you wait 3s, and now it does
vms can talk to each other internally, can connect out, and persist if the server needs a restart
what i don't have yet is proper multi-tenancy, it treats each ssh key as an account, which is fine since it's just me; incoming connections is not figured out, internal supervisor to keep services running inside each microvm, isolation inside firecracker, snapshoting or backups, and the whole shopping list that would make it an actual mvp
The "cooperative task" they're engaged in is just, broadly, meeting your needs, whatever they are.
The isolation is a desirable property, and I agree this is much preferable to a highly inter-coupled bunch of machines, and also that thia stretches the typical sense in which we refer to a "compute cluster", but I don't think it's an entirely invalid framing of the term.
Not really. In my experience clustering implies multiple compute elements serving the same function with a coordination mechanism to provide redundancy and/or enhanced capacity.
JBOD vs. RAID.
as argued by OP, you can see this happening with exe.dev, and less explicitly with sprites.dev
I run a single-node K8s cluster on a dedicated server because it's way cleaner to manage than the previous mess and mix of docker compose + traefik routing + random stuff installed as package on the host.
I can create "vhosts" for practically anything in a declarative manner, and if the cluster blows up, I have 5 small scripts to bootstrap it and all I need is `kubectl apply -k .`.
Usually it's rather unlike a sane dev setup. Even if your prod setup uses hundreds microservices (you're Google or Uber or something like that), you don't want to run all of them in your personal dev environment, you reuse 90%-99% of stable microservices running in the QA / integration / whatnot environment, and only run a handful locally.
Or maybe you want to run stuffz on 1000 x Raspberry Pi just for fun.
It sits on top of Kubernetes and seems very hand wavy about how you create and manage those clusters.
The article itself reminds me of the enthusiasm I felt for plan9 when I first heard about it back in uni. I also thought everyone should have their own compute grids and that clustered computing was the future; of course now I realize there's a lot of reasons why that doesn't actually work. Considering this appears to be a start-up ad, I hope the author knows something I don't.
ClusterdOS appears to be a kubernetes-in-a-box multiple node setup that's goal is to work so well that the user doesn't know or care what it's doing. I wouldn't trust an LLM with managing one machine by itself, let alone a whole cluster of them running the incredibly complex mess that Kubernetes is (and that's not even counting the 8 other layers of software this is), so this feels like an order of magnitude worse.
[0] Using LLMs for sysadmin research or boilerplate writing is one thing, but after a certain amount of use you're really just paying $X a month for Anthropic to manage your systems for you. I'd rather just pay a real person to do it at that point. I'd also rather people get over their pathological fear of learning how to run a server but I've given up on that.
I'm also not a fan of all the complexity of Kubernetes, one directory with simple to read files makes it a lot more transparent what's where and how it's set up, and the commit history + changelogs make it relatively clear what's changing and why. No distributed database or fancy bootstrapping, just a ubiquitous config format and tool to apply. Changes at the granularity of "a new host is available at A.B.C.D, configure it as a dev server" or "add a new Debian system container named 'blah' to X, bridge it to the research network only, limit to 16 hyperthreads / 64 gb ram, set up for development on git://<whatever>". It works ok for now.
The next major change will be when models that run locally are capable enough to drive the config changes themselves.
> see CEO of Tailscale apenwarr's vibe-researched thread
“Vibe-research” is now a core part of my vocabulary.
A big advantage of clusters, and horizontal scaling in general, is the ability to easily dynamically scale to meet demand.
If you're running a system on a single machine that has N GB of memory and you need to scale to N+1, what do you do? Provision a new machine and migrate everything over?
No-one operates online real-time systems like this. Clusters make it much easier and less expensive to handle this.
On top of that, it's probably true that in some pure numerical problem-count sense, "most problems" don't need a cluster, but that's misleading. It's like saying "most businesses are mom-and-pop shops." Perhaps true, but it ignores hundreds of thousands of larger businesses, or even small business that have big data needs.
There are plenty of problems that involve large amounts of data, and that's increasingly true with ML applications.
I'm at a company of ~100 people which you've probably never heard of (classified as a "small" company in government stats, so not included in the hundreds of thousands figure I mentioned above.) We have 1.9 PB of data for our main environment. When we run processes that deal with it all, the clusters scale to thousands of vCPUs and tens of terabytes of RAM.
Several processes that run daily scale to 500+ vCPUs and many TB of RAM. For the latter, the data itself could probably fit in RAM on a humongous machine, but the CPUs wouldn't fit on a single machine. And we'd have to size the machines carefully every time we start them up. Clusters can scale up dynamically according to the demands of the jobs they're executing.
Besides, Mesos wasn't a good alternative for most companies, so saying "they could have chosen it instead" is a bit theoretical. Mesos was ambitious, but that made it less suitable for a plug & play system that fit easily into existing corporate systems, which had already adopted containers heavily.[] Another reason for Kubernetes' popularity is it didn't try to be a big leap forward the way Mesos did.
[]The Marathon container support for Mesos was released about a year after Kubernetes, but if you were going to set up a system for distributed orchestration of containers, it didn't make much sense to bring Mesos along for the ride. There's a reason Mesos is in Apache heaven now (the Attic.)
Mesos was designed from ideas with HPC and catered to that. Large hardware capex, which is not elastic. Containers did not make sense in that world when it was architected, and it was retrofited into Mesos's architectural foundation.
Kubernetes was designed for a wide variety of workloads, and designed to be composable, versatile, and extensible. It has a far more decentralized approach, an antithesis to Mesos's Data Center as an OS approach. It isn't that Kubernetes did not try to do too much. It's that they laid a much more flexible foundation.
It turns out, Kubernetes was a better fit for a many more use-cases, even beyond large and medium sized enterprises. Kubernetes works pretty well at the edge and locally as well, and it runs many ML workloads on the other end of scale.
This is not theoretical.
so I guess idk what you mean by 'elastic' here.
Wirh regard to AI, hopefully we can run it efficiently on an ASIC.
You can have more than one CPU and more than one storage connected to one mainboard and that works because the interconnect fabric is very fast.
We don't have have the possibility to connect different computers at the same kind of speed that would let them work together seamlessly.
10Gbps is now very cheap and 100Gbps is viable at hobby scale. That's Ethernet. I don't know anything about CXL and so on.
we built machines with RDMA that allowed fast one-sided transfers between memories at a decent fraction of the memory bandwidth. and operating systems that ran services to present a unified operating system interface on top of that.
there is a whole history of distributed operating systems if you're interested