bleevht

If you are looking for learning projects in your home lab this weekend, check out these 6 projects from my recent home lab testing. These cover Linux troubleshooting, VLAN isolation, Proxmox self-service, TrueNAS monitoring and storage, and Docker image scanning. Each of these I think are a great starting point for learning new things over a weekend.

I put together seven Linux commands that I think are some of the most often misunderstood or unknown tools that I find useful for troubleshooting. We take a look at ss, lsof, findmnt, journalctl, watch, ip route get, and timeout. If you are looking for a way to find what is listening on a port, identifying processes using files, understanding mounts, tracing routing decisions, and troubleshooting services. These are the tools that can help you do that quickly.

Check out this walkthrough for checking which connections can cross home lab VLANs. It is important to identify the firewall rules that allow traffic, and make sure you test your changes with port tests and firewall logs. We also take a look at Proxmox VM firewall rules for blocking traffic between workloads on the same VLAN.

If you have ever been in need of an easy self-service portal for Proxmox to provide access to developers, or for selling resources, I found a solution that just might fit the bill. PVE Panel is a self-hosted solution that provides a self-service portal for Proxmox where you can define customers and assign specific resources to those customers using the workflows the solution provides. I found it easy to setup and easy to start providing resources. Check it out.

I connected my TrueNAS box to Graphite and Grafana. I had both of these running in a Talos Kubernetes cluster. The metrics were sent over correctly, but the imported dashboard I was using from the Grafana community showed no data because it expected older TrueNAS metric paths. So, I got this fixed after remapping the panels. Then, I found another issue. It interpreted an ARC value in bytes as megabytes and displayed 12.7 PB instead of roughly 12 GB!

Check out my full post detailing what I ran into, why I saw what I did and then how I fixed them. At the end, I was able to have a really good TrueNAS monitoring dashboard in Grafana.

I’ve been testing Dockhand’s vulnerability scanning in my home lab. It can scan a newer image with Trivy or Grype before it does an automatic update to replace the running container. You can also block the update based on the rule you choose which I think is pretty cool. I wrote up a post on configuring this setup and the rule you can choose. Curious how others decide which container updates to automate in your home labs or production?

ma2kx,

I think these are good points. I also run a Talos Linux K8s cluster in the home lab and have full GitOps running with ArgoCD. It is a slick way to run services. However, I think many don't want that kind of complexity in the home lab, even though I think it has a lot of advantages to running things just on Docker. But the great thing about self-hosting is we have tons of options.

In case you haven't heard, Portainer 3.0 is moving to a Kubernetes-first codebase. It will keep native Docker, Swarm, and Podman environments available but this is no longer the focus. I looked at what this shift means for home lab users, including KubeSolo, the D2K solution Portainer has developed, continued support for Portainer 2.45 LTS, and we look at whether home lab Docker users need to change anything yet.

I tested the new TrueNAS storage plugin with a three-node Proxmox cluster and a TrueNAS VM. The really cool part was watching a disk created in the Proxmox VM wizard appear as a new zvol in TrueNAS. I also ran into a few setup details worth noting. These include creating unique serial numbers for virtual disks, the iSCSI target configuration, and installing the plugin on each node. The post walks through all of my setup and what I verified in the lab.

I wrote about Pangolin last year, when its clustering capability was a bit cumbersome. It still required a custom deployment engagement with Pangolin. Version 1.23 brings DNS and certificate management into Pangolin and publishes a two-node HA deployment guide. They have now introduced the HA clustering component as a native part of the deployment without jumping through hoops.

You still have to supply your own HA load balancer as part of the solution but the process to get HA for Pangolin now feels within reach for the home lab.

A two-node Proxmox cluster can lose quorum if either of your hosts fail. This is true even if the server that is left is healthy and its workloads are still running. If Proxmox HA is involved, the HA watchdog service will do what is called self-fencing that can also reset an isolated node to keep it from having split brain in the cluster.

I explored how Corosync quorum works, what self-fencing technically does, and how an external QDevice is used. It provides a third vote without requiring another full Proxmox server.

The new official TrueNAS storage plugin I think is going to lead to a broader change for Proxmox. When you combine this with Proxmox introducing 24/7 enterprise support, storage vendors now have a really good reason to build native integrations instead of relying only on generic NFS and iSCSI connections that are found in Proxmox itself.

I look at five trends that could shape Proxmox storage in 2027. These including things like vendor-supported plugins, API-driven provisioning, per-VM storage objects, broader NVMe/TCP adoption, and the role storage support will play in VMware migrations.

Do you think other major storage vendors will follow TrueNAS with official Proxmox integrations?

I put this together as a weekend project list based on tools I have recently tested in my own home lab. It includes a few newcomers like Changerawr, which I have found useful once I worked around a migration bug. Also, along with Proxmox and TrueNAS-related projects. These are genuinely projects that I think don't take that much time, but they provide some really great features for our home lab environments.

I put together a few steps I go through and checks I make before I start moving real workloads over to a new Proxmox host in the home lab. In the post we take a look at what I think are several important steps and areas, including hostname and DNS configuration, update repositories, and storage health. But we also look at the networking side with speed tests and MTU checks, VM hardware defaults, backups, etc.

Also, outside of Proxmox settings themselves, there are checks you need to make in the BIOS to make sure your mini PC or other home lab server is ready to accept new virtualization workloads.

TrueNAS has released an early-adopter Proxmox VE storage plugin! Pretty slick. It automates zvol provisioning, snapshots, resizing, migration, and deletion over iSCSI or NVMe/TCP. I looked at how it compares with NFS and manually managed iSCSI, what it changes for Proxmox clusters, and why I would still test it outside production first.

This is so cool! I’ve been testing Komodo as a Docker management platform, and found an MCP server that opens up letting an AI agent interact with my Docker environment in the home lab through Komodo.

In this post I walk through how the setup works, what the AI can actually see and do, and where I think MCP fits into the workflow. We also look at the security considerations around giving an AI agent access to your real container infrastructure.

What I found most interesting is the shift from AI as an advisor to AI as an operator. This is powerful change, but it also changes how I think about permissions, guardrails, and what I am comfortable with giving AI access to in the home lab.

Curious how you guys are allowing AI to be used and worked with in your home lab environments or production environments?

I’ve been try to get more of my home lab behind Authentik and SSO lately, but that raised a question in my mind. What happens when the identity system is the thing that breaks? How do I get into the rest of my lab?

This post looks at the difference between getting Authentik back itself, fixing access to an application tied to it, and getting into infrastructure when Authentik is completely down. I also walk through where I keep local break-glass access and why I would avoid having one shared emergency credential. Also, we look at why it is so important to test these accounts just like we test our backups.

Curious how others handle emergency access in their self-hosted environments without having just another single point of failure.

I’ve started out treating reliability in my home lab as something I need to test. This, like backups, is not something you just want to leave to assumptions. There are 7 checks I recommend making before I really depend on a service. These are pulling Internet access, testing host failure, killing the app itself, watching predictive health metrics, verifying configuration history, tracing application dependencies, and making sure I can rebuild from documentation and Git instead of memory.

The article ties together lessons I have learned from things like Proxmox HA, Keepalived, Docker, DNS, monitoring, Git version control, and application mapping.

I have quite a lot of Docker Compose stacks spread across many different home lab hosts. I realized the difficult part was not rolling out containers. It was keeping them updated over time.

I kept SSHing into individual servers, finding the right Compose directory, pulling new images, and then redeploying my stacks. Then, once they restart, I check whether everything came back healthy.

In this post I walk through how I started using Komodo to automate part of that workflow. With it, I can monitor container image updates, and automate redeployments. It also allows you to import your Compose stacks, and you can use deployments that are stored in your Git repos without having to change your Docker Compose.

I also look at the difference between Poll for Updates, Auto Update, the global update schedule. Also, see how Komodo can pull from Git when you make changes.

Curious how others are managing this once Docker Compose environments go beyond a single host.

I put together a practical two-hour home lab maintenance task list focused on the things I tend to neglect when everything appears to be working: checking updates, verifying backups, cleaning storage, patching, rebooting long-running server. And, also, validating critical services, and documenting what changed.

The main idea is that it doesn't take a lot of time to get rid of a lot of uncertainty in the lab without it being an all-day exercise. Curious what other people put at the top of their home lab maintenance checklist?

I stumbled onto a project called ANAS and spent some time looking at what it adds to Proxmox VE. The interesting part is that it runs directly on the Proxmox node. It also adds storage management inside the existing PVE web UI instead of making you have a separate appliance or VM running on top of Proxmox.

It can manage ZFS, SMB/NFS shares, iSCSI targets, snapshots, replication, PBS backup/restore, and its own mixed-size Hybrid RAID implementation. Very cool.

I wrote this up from the angle of whether this could replace the common "Proxmox + TrueNAS VM" setup. I don’t think it replaces TrueNAS across the board, but I think the architecture adds a lot of value and is an interesting take on storage in Proxmox moving forward.

I found that a pair of VMs I thought I had that was redundant had quietly ended up on the same Proxmox host. The services were redundant at the application layer, but not at the physical failure-domain layer on my Proxmox hosts.

I wrote up what I found, how VM placement can drift during migrations and maintenance, and how Proxmox HA resource affinity rules can help keep redundant workloads on separate nodes.

The bigger lesson for me was that two copies of a service do not necessarily mean two separate failure domains.

I had always thought of Keepalived mainly as host-level failover with VRRP. While working on containerized DNS HA, I realized Keepalived track scripts can make failover decisions based on the application itself. This is pretty cool functionality. For instance, with DNS you can check to see whether port 53 or a DNS query is actually responding. Check out the full config here, including rise, fall, interval, and the difference between automatic failback and nopreempt.

Proxmox has just announced 24/7 vendor-direct enterprise support. Also, they are opening a new North American subsidiary. I wrote about why I think this matters beyond just longer support hours, especially for organizations that have been evaluating Proxmox as a serious VMware alternative. The bigger shift is that Proxmox is starting to build more of the enterprise support, procurement, and operational structure around the platform. So they are concentrating on adding more than just hypervisor features. Check it out.

Very cool. Will check this one out as well. Do you use this in your homelab?

I’ve been running Proxmox for quite some time in the home lab, and one thing I’ve learned is that some choices are easy to make on day one. However, they can get super painful to change once your home lab grows. I wrote up seven of the ones I think are worth planning early, including storage, networking, CPU compatibility, clustering, backups, and infrastructure dependencies.

I’ve run GitLab and Gitea in the home lab. I then decided I wanted to see where Forgejo fits in. What stands out to me is how much it includes in terms of features while still feeling lightweight. It has Git hosting, Actions-based CI/CD, runners, package registries, and OCI/container images. I also looked at where it looks different from Gitea, since from the features they are still pretty close.