Hacker News Reader

Comment by phillebaba | Hacker News Reader

Shameless plug but this might be a good time to install Spegel in your Kubernetes clusters if you have critical dependencies on Docker Hub.

https://spegel.dev/

View on HN

If it really is fully open-source please make that more visible on your landing page.

It is a huge deal if I can start investigating and deploying such a solution as a techie right away, compared to having to go through all the internal hoops for a software purchase.

How hard is it to go to the GitHub repository and open the LICENSE file that is in almost every repository? Would have taken you less time than writing that comment, and showed you it's under MIT.

It's not entirely uncommon to only have parts of the solution open. So a license on one repo might not be the whole story and looking further would take more time than giving a good suggestion to the author.

Agreed. For all the people arguing "just click the link and the license is there!!", I have been burned several times before where a technical solution has a prominent open permissive license github repo (MIT or similar etc) based component as its primary home, only to discover later on that essential parts of the system are in other less permissive or private repos behind subscriptions or fees.

The rest of us get around that particular issue by going through the source code and all the tradeoffs before we download, include and adopt a dependency, not after.

Good for you! This of course doesn't help in the situation where a dependency author retroactively changes the licensing state of a component, or reconfigures the project to rely on a new external dependency with differing license states (experienced both of these too!).

Having the landing page explain the motivations of the authors vis-a-vis open source goes a long way to providing the context for whatever licensing is appearing in the source repos, and helps understand what the future steer for the project is likely to be.

There are loads of ostensibly open source projects out there whose real goal is to drive sales of associated software and services, often without which the value of the opensource components is reduced, especially in the developer tooling space.

> Good for you! This of course doesn't help in the situation where a dependency author retroactively changes the licensing state of a component, or reconfigures the project to rely on a new external dependency with differing license states (experienced both of these too!).

No, but I also don't see why that matters a lot. Once you adopted a third party project as a dependency, you also implicitly sign up to whatever changes they do, or you get prepared for staying on a static version with only security fixes you apply yourself. This isn't exactly new problems nor rocket science, we've been dealing with these sort of things for decades already.

> There are loads of ostensibly open source projects out there whose real goal is to drive sales of associated software and services, often without which the value of the opensource components is reduced, especially in the developer tooling space.

Yeah, which is kind of terrible, but also kind of great. But in the end, ends up being fairly easy to detect one way or another, with the biggest and reddest signal being VC funded with no public pricing.

If I have to dig through your website/documentation to find basic information we’re not getting off to a great start. It’s pretty common for open source projects to proudly proclaim they are open source from the get-go. “____ is an open source tool for ______.” Simple as that

Today's kids are way too lazy.

Seriously all the nitpicking I see of any project people post here but “tell us you’re open source at the top when you’re open source” means we’re lazy? Being open source is an important decision and you should tell people! It’s a good thing!

Isn’t a big part of getting a project out there actually letting people know what it is? Especially if you’re trying to give a tool to the open source-valuing community. That’s a high priority for them. That’s like having a vegan menu and not saying you’re a vegan restaurant anywhere public facing.

I agree it's a good thing, but I'd also agree it's not something you need/have to shove in people's faces, especially when it's literally one click away to find out (The GitHub icon in the top right takes you to the repository, and you don't even have to scroll or click anything, the sidebar shows "MIT License" for you).

> I'd also agree it's not something you need/have to shove in people's faces

Agree to disagree. It should be front and center the moment I find your tool IMO.

There is a GitHub icon fairly prominent on the top right. Choosing to spend precious text for a fleeting would be user on it is a choice and not everyone wants to market that fact very prominently. Should everyone who writes their project in rust include that prominently as well? It seemingly markets very well and a lot of people seem to care about that as well.

Also it's good feedback for the developer of this solution

After some digging - https://github.com/spegel-org/spegel/blob/main/LICENSE says MIT

https://spegel.dev/project/community/

What's the difference with kuik? Spegel seems too complicated for my homelab, but could be a nice upgrade for my company

Kuik: https://github.com/enix/kube-image-keeper?tab=readme-ov-file...

It's been a while since I looked at kuik, but I would say the main difference is that Spegel doesn't do any of the pulling or storage of images. Instead it relies on Containerd to do it for you. This also means that Spegel does not have to manage garbage collection. The nice thing with this is that it doesn't change how images are initially pulled from upstream and is able to serve images that exist on the node before Spegel runs.

Also it looks kuik uses CRDs to store information about where images are cached, while Spegel uses its own p2p solution to do the routing of traffic between nodes.

If you are running k3s in your homelab you can enable Spegel with a flag as it is an embedded feature.

There is a couple of alternatives that mirrors more than just Docker Hub too, most of them pretty bloated and enterprisey, but they do what they say on the tin and saved me more than once. Artifactory, Nexus Repository, Cloudsmith and ProGet are some of them.

Spegel does not only mirror Docker Hub, and works a lot differently than the alternatives you suggested. Instead of being yet another failure point closer to your production environment, it runs a distributed stateless registry inside of your Kubernetes cluster. By piggy backing off of Containerds image store it will distribute already pulled images inside of the cluster.

I'll be honest and say I hadn't heard of Spegel before, and just read the landing page which says "Speed up container pulls and minimize downtime with a stateless peer-to-peer OCI registry mirror for efficient image distribution", so it isn't exactly clear you can use it for more things than container images.

What exactly does "stateless" mean in this context?

Spegel itself does not manage state as a normal registry would. Maybe ephemeral would be a better word to describe it. A normal registry would require some stateful storage solution along with a database to store image that clients push to it. Spegel exploits the fact that images used by containers will be stored on disk by Containerd for its benefit. Any image currently being used by a pod in a cluster will be available for all other nodes in the cluster to pull.

Gotcha. That's definitely an important point and seems difficult to communicate in a single word or quick blurb. I can see why you went with stateless. It's just a little confusing in this context (for me at least).

This looks good, but we're using GKE and it looks like it only works there with some hacks. Is there a timeline to make it work with GKE properly?

I am having some discussions about getting things working on GKE but I can't give an ETA as it really depends on how things align with deployment schedules. I am positive however that this will soon be resolved.

Thanks. I will keep an eye on your project as it looks great and something we would definitely benefit from.

P.S. Your blog could do with an rss feed ;). I will track https://github.com/spegel-org/spegel/releases.atom for now

Google Cloud has its own cache of Docker Hub that you can use for free, AWS does as well

Our images are in a private docker registry on quay.io

Reply on news.ycombinator.com