0xC45

For an open source, local (or cloud) vector DB, I would also recommend checking out Chroma (https://trychroma.com). It also supports full text search. Disclaimer: I work on Chroma cloud.

Yeah, I guess it's somewhat "half-and-half" currently. The k8s cluster is definitely an experimental zone. The spare cycles on the main ESXi host are also available for testing out ideas, etc. However, I guess a benefit of having regular backups and an easy way to re-provision the services with ansible means that the risk (and cost) of breaking something is fairly low.

Haha. I'm a little bit lacking in my "k8s ops" skills and hope to learn by practicing! Currently there isn't any serious "production" workload running on the k8s cluster. Perhaps if I become more comfortable with managing the k8s cluster, I will migrate my other services onto k8s. We'll see!

It felt like overkill to dedicate an entire NUC to running the K8s control plane. So, I split each NUC in half. Only one node is running the control plane (and, I have prevented any other workloads from running on the control node as a best practice). The other 5 nodes are available for running whatever workloads. There's probably a bit of a performance hit from running two separate OS's and kubelets on each NUC, but I'm hoping it won't matter too much in practice.

Hey, I like this idea! Actually, I only have a single node running the control plane. And, to answer the original question, it felt like overkill to dedicate an entire NUC to running just the control plane. So, I decided to run two k8s nodes on each NUC. I guess there's potentially a bit of a performance hit due to the extra overhead of running multiple kubelets, but it hopefully won't matter too much in practice.

Yeah, unfortunately the free ESXi makes it difficult to "programmatically" create VMs. However, I decided that it was an acceptable trade-off for now. It would be super cool to run a script with "nothing" running on your network and have it set up everything from the ground up. Perhaps I'll manage to get there someday as well.

Thanks for the advice about the NAS. Perhaps it's best to leave critical data and backups to a professional product.

As for the firewall, I had a Ubiquiti Security Gateway for a while and it worked great. Not sure how it compares to the UniFi Dream Machine, however. The only reason I made a custom firewall is because I wanted to try and learn a bit about firewalls and routing in general. And, since my own device has worked well for me so far, I haven't bothered to replace it.

Hey, thanks for all the good ideas.

About the custom "VM-only" machine: currently I'm a little lacking in my "k8s ops" skills. So, until I am more comfortable managing the k8s environment, I have decided to continue using VMs for my "production" workloads. Until then, the K8s cluster will be treated like more of an experimental zone and I can worry less about breaking things.

As for the K8s storage options -- I don't really know! Haven't quite figured that one out. Like I mentioned in the blog post, I haven't really placed any serious workload on the K8s cluster yet. However, I have heard good things about OpenEBS (https://openebs.io/) and am considering that as a potential option in order to make use of the local SSDs on each NUC.

For single-node K8s, there are lots of good options. I haven't used it personally, but I know that MicroK8s (https://microk8s.io/) works well even in a single-node environment. It might be worth checking out.