psviderski

There is an open issue that confirms enabling ipv6 for containers works: https://github.com/psviderski/uncloud/issues/126 But this hasn’t been enabled by default.

What specifically do you mean by ipv6 support?

I took some inspiration from Kamal, e.g. the imperative model but kamal is more a deployment tool.

In addition to deployments, uncloud handles clustering - connects machines and containers together. Service containers can discover other services via internal DNS and communicate directly over the secure overlay network without opening any ports on the hosts.

As far as I know kamal doesn’t provide an easy way for services to communicate across machines.

Services can also be scaled to multiple replicas across machines.

Hey, creator here. Thanks for sharing this!

Uncloud[0] is a container orchestrator without a control plane. Think multi-machine Docker Compose with automatic WireGuard mesh, service discovery, and HTTPS via Caddy. Each machine just keeps a p2p-synced copy of cluster state (using Fly.io's Corrosion), so there's no quorum to maintain.

I’m building Uncloud after years of managing Kubernetes in small envs and at a unicorn. I keep seeing teams reach for K8s when they really just need to run a bunch of containers across a few machines with decent networking, rollouts, and HTTPS. The operational overhead of k8s is brutal for what they actually need.

A few things that make it unique:

- uses the familiar Docker Compose spec, no new DSL to learn

- builds and pushes your Docker images directly to your machines without an external registry (via my other project unregistry [1])

- imperative CLI (like Docker) rather than declarative reconciliation. Easier mental model and debugging

- works across cloud VMs, bare metal, even a Raspberry Pi at home behind NAT (all connected together)

- minimal resource footprint (<150MB ram)

[0]: https://github.com/psviderski/uncloud

[1]: https://github.com/psviderski/unregistry

Check out https://github.com/psviderski/uncloud I'm building. Multi-machine deployments and a private WireGuard network spanning locations (even behind a NAT) are its core capabilities.

Exactly, although my main motivation was to reduce the distinction between docker engine and docker registry. To make it possible for a user to push/pull to the docker daemon as if it was a registry, hence a registry wrapper.

This is a prerequisite for what I want to build for uncloud, a clustering solution I’m developing. I want to make it possible to push an image to a cluster (store it right in the docker on one or multiple machines) and then run it on any machine in the cluster (pull from a machine that has the image if missing locally) eliminating a registry middleman.

They're unbundled already. You can run unregistry as a standalone service and use your own way to push/pull from it: https://github.com/psviderski/unregistry?tab=readme-ov-file#...

The problem with running a registry locally is that Docker doesn't provide an API to get individual image layers to be able to build a registry API on top. You have to hook into the containerd Docker uses under the hood. You can't do this locally in many cases, for example, on macOS the VM running Docker Desktop doesn't expose the containerd socket. I guess the workaround you implemented in docker-pushmi-pullyu is an extra copy to the registry which is a bummer.

You're the OG! Hats off, mate.

It's a bummer docker still doesn't have an API to explore image layers. I guess their plans to eventually transition to containerd image store as the default. Once we have containerd image store both locally and remotely we will finally be able to do what you've done without the registry wrapper.

Hey Solomon, thank you for sharing your thoughts, love your work!

1. Yeah agreed, it's a bit of a mess that we have at least three different file system layouts for images and two image stores in the engine. I believe it's still not too late for Docker to achieve what you described without breaking the current model. Not sure if they care though, they're having hard times

2. Hm, push-to-cluster deployment sounds clever. I'm definitely thinking about a distributed image store, e.g. embedding unregistry in every node so that they can pull and share images between each other. But triggering a deployment on push is something I need to think through. Thanks for the idea!

It starts an unregistry container on the remote side. I wonder, what's the use case on your mind for doing it the other way around?

Not a torrent protocol but p2p, check out https://github.com/spegel-org/spegel it's super cool.

I took inspiration from spegel but built a more focused solution to make a registry out of a Docker/containerd daemon. A lot of other cool stuff and workflows can be built on top of it.

Wow ttl.sh is a really neat idea, thank you for sharing!

I guess you're asking about the registry part (not 'pussh' command). It exposes the containerd image store as standard registry API so you can use any tools that work with regular registry to pull/push images to it.

You should be able to run unregistry as a standalone service on one of the nodes. Kubernetes uses containerd for storing images on nodes. So unregistry will expose the node's images as a registry. Then you should be able to run k8s deployments using 'unregistry.NAMESPACE:5000/image-name:tag' image. kubelets on other nodes will be pulling the image from unregistry.

You may want to take a look at https://spegel.dev/ which works similarly but was created specifically for Kubernetes.

Ah neat I didn't know that podman has 'image scp'. Thank you for sharing. Do you think it was more straightforward to implement this in podman because you can easily access its images and metadata as files on the file system without having to coordinate with any daemon?

Docker and containerd also store their images using a specific file system layout and a boltdb for metadata but I was afraid to access them directly. The owners and coordinators are still Docker/containerd so proper locks should be handled through them. As a result we become limited by the API that docker/containerd daemons provide.

For example, Docker daemon API doesn't provide a way to get or upload a particular image layer. That's why unregistry uses the containerd image store, not the classic Docker image store.

I don't see a reason to not adopt this in Kamal. I'm also building Uncloud that took a lot of inspiration from Kamal, please check it out. I will integrate unregistry into uncloud soon to make the build/deploy process a breeze.

Yep, I'm familiar with Kamal and it actually inspired me to build Uncloud using similar principles but with more cluster-like capabilities.

I built Unregistry for Uncloud but I belive Kamal could also benefit from using it.

That's an interesting idea. I don't think you can create a subcommand/plugin for compose but creating a 'docker composepussh' command that parses the compose file and runs 'docker pussh' should be possible.

My plan is to integrate Unregistry in Uncloud as the next step to make the build/deploy flow super simple and smooth. Check out Uncloud (link in the original post), it uses Compose as well.

Met too, you're welcome! Please create an issue on github if you find any bugs

I'm glad the idea of uncloud resonated with you. Feel free to join our Discord if you have questions or need help

For sure, it's always a tradeoff and it's great to have options so you can choose the best tool for every job.

That's a valid concern. You can very easily give it whatever name you like. Docker looks for `docker-COMAND` executables in ~/.docker/cli-plugins directory making COMMAND a `docker` subcommand.

Rename the file to whatever you like, e.g. to get `docker pushoverssh`:

  mv ~/.docker/cli-plugins/docker-pussh ~/.docker/cli-plugins/docker-pushoverssh

Note that Docker doesn't allow dashes in plugin commands.

You need a containerd on the remote end (Docker and Kubernetes use containerd) and anything that speaks registry API (OCI Distribution spec: https://github.com/opencontainers/distribution-spec) on the client. Unregistry reuses the official Docker registry code for the API layer so it looks and feels like https://hub.docker.com/_/registry

You can use skopeo, crane, regclient, BuildKit, anything that speaks OCI-registry on the client. Although you will need to manually run unregistry on the remote host to use them. 'docker pussh' command just automates the workflow using the local Docker.

Just check it out, it's a bash script: https://github.com/psviderski/unregistry/blob/main/docker-pu...

You can hack your own way pretty easily.

Yeah exactly, which is crucial for large images if you change only a few last layers.

The unregistry container provides a standard registry API you can pull images from as well. This could be useful in a cluster environment where you upload an image over ssh to one node and then pull it from there to other nodes.

This is what I’m planning to implement for Uncloud. Unregistry is so lightweight so we can embed it in every machine daemon. This will allow machines in the cluster to pull images from each other.

Thank you! Can you please clarify what kind of support you mean for docker compose?

Thanks! For zero-downtime deploys, it does simple rolling updates one container at a time in a similar way k8s or swarm does it. It starts the new container alongside the old one, waits for it to become healthy, Caddy picks it up and updates its config, then removes the old one. The difference is that this process is driven by your CLI command (not a reconciliation loop in the cluster) so you get an instant feedback if something goes wrong.

I'm actually building something in between Docker and Kubernetes: https://github.com/psviderski/uncloud. Like you I wanted that middle ground without the operational overhead. It's basically Docker-like CLI and Docker Compose with multi-machine and production capabilities but no control plane to maintain.

Still in active development but the goal is to keep it simple enough that you can easily understand what's happening at each layer and can troubleshoot.

Can you still justify K8s even if you need to jump from one cloud/dedicated instance to say 2-3 for reliability?

After a decade of professionally working with container orchestrators like Kubernetes and ECS I quit my job to build Uncloud — an open source lightweight version of those [1]. I want the tool to bring me joy when I use it, in a similar way Docker did when I first tried it in 2013.

Imagine Docker Compose and Tailscale had a baby. Uncloud is the baby that allows you to deploy and manage containers across a network of Docker hosts. I feel like we forgot how to build simple tools that do the job and not a thousand other unnecessary things.

I've started migrating some of my self-hosted web apps from my k8s cluster and I really enjoy it. Using the tool early on helps me better understand where the biggest pain points are and what I need to prioritise.

[1]: https://github.com/psviderski/uncloud

What's your approach with getting users for your OSS product?

Thanks! I'm more than happy to catch up to discuss the challenges. Feel free to reach out to me on twitter @psviderski or email 'me at psviderski.name'