alexellisuk

Self-hosted runners are brilliant, but have a poor security model for running containers or building them within a job. Whilst we're focusing on GitHub Actions at the moment, the same problems exist for GitLab CI, Drone, Bitbucket and Azure DevOps. We explain why in the FAQ (link in the post).

Thanks for commenting.

It seems like buildjet is competing directly with GitHub on price (GitHub has bigger runners available now, pay per minute), and GitHub will always win because they own Azure, so I'm not sure what their USP is and worry they will get commoditised and then lose their market share.

Actuated is hybrid, not self-hosted. We run actuated as a managed service and scheduler, you provide your own compute and run our agent, then it's a very hands-off experience. This comes with support from our team, and extensive documentation.

Agents can even be cheap-ish VMs using nested virtualisation, you can learn a bit more here: https://docs.actuated.dev/add-agent/

It is, but is also a very low-level tool, and there is very little support around it. We've been building this platform since the summer and there are many nuances and edge cases to cater for.

But if you just want to try out Firecracker, I've got a free lab listed in the blog post.

I hear Podman desktop is also getting some traction, if you have particular issues with Docker Desktop.

Thanks for commenting, and your product looks cool btw.

Yeah a lot of people have talked about Firecracker in the past, that's why I focus on the pain and the problem being solved. The tech is cool, but it's not the only thing that matters.

People need to know that there are better alternatives to sharing a docker socket or using DIND with K8s runners.

When you say it's limited in what it can do, what are you comparing it to? And what do you wish it could do?

Fly has a lot of ideas here, and we've also been able to optimize how things work in terms of downloads and as for boot-up speed, it's less than 1-2s before a runner is connected.

You can checkpoint and restore, but only once for security reasons, so it doesn't help much.

https://github.com/firecracker-microvm/firecracker/blob/main...

The VMs launch super quick in < 1s they are actually running a job.

Great questions, we answer those here in the FAQ: https://docs.actuated.dev/faq/

Thanks for the feedback.

That video covers this is great detail. Click on the the video under 1) and have a watch, it should answer all your questions.

I didn't want to repeat the content there

Hey thanks for the feedback. We may do some more around this. What kinds of things do you want to know?

To get hands on, you can run my Firecracker lab that I shared in the blog post, then add a runner can be done with "arkade system install actions-runner"

We also explain how it works here: https://docs.actuated.dev/faq/

Hey thanks for the interest, this is probably the best resource I have on Firecracker, hope you enjoy it:

https://www.youtube.com/watch?v=CYCsa5e2vqg

For info on actuated, check out the FAQ or the docs: https://docs.actuated.dev

We're running a pilot and looking for customers who want to make CI faster for public or self-hosted runners, want to avoid side-effects and security compromise of DIND / sharing a Docker socket or need to build on ARM64 for speed.

Feel free to reach out

We've done a lot of work on "self-hosted AND automated"

inletsctl - automates a VM with the inlets tunnel server preinstalled, including with HTTPS termination with Let's Encrypt, or pure TCP pass-through

inlets-operator - the same for Kubernetes Load Balancer services

Feel free to take a look at https://inlets.dev/

We talk a lot about the community adoption here in the first half -https://inlets.dev/blog/2022/11/16/service-provider-uplinks....

(I'm the original founder of inlets)

Mainline support is available, we are missing the device tree blobs for this specific unit. OpenBSD AFAIK boots in APCI mode so doesn't require them.

It me: https://blog.alexellis.io/linux-on-microsoft-dev-kit-2023/

I've written up my findings with trying to getting Linux working on the new Dev Kit - https://blog.alexellis.io/linux-on-microsoft-dev-kit-2023/

I've tried it. It seems like even with a 6.0 Kernel, additional DTBs are required to boot.

https://twitter.com/alexellisuk/status/1585562359254421504?s...

Does Linux work on it? No, not yet. It may need some custom firmware like you see in the instructions for the Surface.

Support for the Qualcomm chip did land in v6.0 of the Kernel - https://github.com/torvalds/linux/commit/b6a6535b339776d32fa...

I just had mine arrive today, which I wanted to enroll as an ARM64 build agent (with Firecracker) for self-actuated ( https://docs.actuated.dev ) - this does actually work with Asahi Linux and the Mac Mini M1, but that was double the price for half the RAM.

So far, it's not looking good, with Ubuntu Server 22.04 and 22.10 both crashing after the Grub menu. I'm sharing progress here:

https://twitter.com/alexellisuk/status/1585562359254421504?s...

Hi, it's Alex, the OpenFaaS founder here. This post is by Han from our team, but I have written an in-depth intro that I hope you'll find interesting.

Here's a quick overview:

> We’ll show you how to access any part of the Kubernetes API or a Custom Resource using a ServiceAccount and granular permissions

> You’ll learn how you’re already using the Kubernetes API, what an Operator is, and why a Function may be a better fit some of the time. We’ve also included a code-example so you can get going with that today.

I've had a few calls with users recently who are moving from AWS Lambda to bare Kubernetes, and struggling to get this kind of working scaling right for them. So we sat down and put together an example to show how it could be done.

Disclaimer: we both work at OpenFaaS Ltd and maintain the OSS project full time.

I was on a call with an inlets customer, and they were struggling to push multiple different servers over an inlets TCP server. It's designed to only have one upstream server, but multiple ports.

So they'd been writing pretty verbose / obtuse HAProxy configs. They also needed a Linux host for HAProxy, which they couldn't guarantee.

I knew that a tiny Go binary could do this task and give us more control over configuration for the future. I asked them to give me an hour, then wrote up this tool and sent it over for them.

Hi I'm the founder of inlets (a self-hosted network tunnel that has no inherent connection / domain or bandwidth limits) and was built to run in containers in K8s:

> If you’ve ever run a reverse proxy, tunnel or load balancer (LB) and wondered why you saw internal IP addresses in your logs, then you’re not alone. The plaintext HTTP protocol has a number of “X-Forwarded-*” headers that covers this for proxies which decrypt traffic. But what if you’re using TLS, or TCP which works at a lower level in the Open Systems Interconnection model (OSI model)?

> That’s where Proxy Protocol comes in.

If you missed the news yesterday, https://istio.io/latest/blog/2022/introducing-ambient-mesh/ - Istio is moving to a "proxy per node" / "per namespace" model.

Chris has a video demo here (also featuring K9s) - https://www.youtube.com/watch?v=nupRBh9Iypo

This is quite a revolution - similar to when the micro services got melted down into "istiod".

Even with this automation, I've been bitten a few times, where a vendor shows stock, I order and pay, then 2-3 weeks later there's no news or contact. Usually it means they had out of date info on their site.

Hope this is useful and entertaining for some of you that read it.

The idea is that some of you will go on to adapt the example for your own use at work / home.

I wanted to share a new OSS tool I've written because I felt like the user experience for running a job and getting the output from the Pod was too cryptic.

If you can see yourself using it, let me know. Try it out.

This is how it's being used in OpenFaaS to help support customers: https://github.com/openfaas/config-checker

A similar and more generic tool from Replicated is: https://github.com/replicatedhq/troubleshoot

And if you wanted to see how easy it is to play around with the K8s API with Go, hopefully this example will give you a bit more confidence to try some hacking.

This is really cool. I've also been working with Firecracker, but for isolated CI runners with Docker and KinD/K3s support. Starting with GitHub Actions [1] I've also had interest in making OpenFaaS use pause/resume from Gatsby.js who wanted to reduce their hosting costs. The main challenges were around the networking - if you use CNI and the Go SDK [2] then restores simply don't work. Not sure if you're working with netlink and IMAP directly to get around it?

My question is how are you guaranteeing uniqueness, or do you only clone snapshots for a single tenant? [3]

[1] https://github.com/self-actuated/actuated [2] https://github.com/firecracker-microvm/firecracker-go-sdk [3] https://github.com/firecracker-microvm/firecracker/blob/main...

On one hand, this is about inlets, a product I wrote to help developers, self-hosters and enterprises to connect containers to the cloud. On the other hand, I hope you'll learn as much as I did about Proxy Protocol and how hard it is to configure for Traefik behind K3s.

This is written by our newest hire (Han) and shows how to use the Argo CD Image Updater with OpenFaaS functions in Helm.

What if you're not using OpenFaaS yet? Glad you asked. The examples work with any chart or CRD, so if you're using K8s, you can probably use what we've written here.

Check out other articles on CD on the blog.

This article was shared recently on Twitter, so I wanted to circulate it and see if it could be of help to more folks. I wrote it from my own perspective, from using K8s for ~ 6 years and helping OpenFaaS / K3s users.

It's been interesting watching you build this on Twitter.

What prompted you to invest your time in this project vs the others that you were probably considering?

Alex here, the founder of OpenFaaS. We took some time to explain why we think JetStream is the next generation of queueing for OpenFaaS functions, what the use-cases are, and why we invested 4 mo of R&D into it so far.

There's a thread on Twitter where I'm happy to take questions, suggestions, etc. https://twitter.com/openfaas/status/1550063046663569409

Hi folks, I'm the author of inlets and openfaas. I wrote this article to explore the three options I see for exposing a K8s cluster from a private network for remote access. There's also a disclaimer in there, about checking if your distribution can be exposed publicly. For something like K3s, it's made to be run on the Internet, but they do still have a few recommendations: https://rancher.com/docs/k3s/latest/en/installation/

Feel free to try out the steps in the post, or hit me up on Twitter @alexellisuk to chat.

Alex