Comment by devjab | Hacker News Reader

Comment by devjab | Hacker News Reader

Parent Full thread

Yo dawg, I heard you liked containers so I put a container inside your container!

In all seriousness though, I wonder if this could solve one of our issues of running mutli-container apps in Azure, which is currently causing us some trouble in regards to VNETs and Private Endpoints. I’m sure it’s likely because we should be using Kubernetes or similar, or at the very least have someone who knows how to actually set the Ops part of DevOps up better than we currently do. Long story short, those things won’t realistically happen, and if we could solve them by running multiple podmans inside a docker docker-container deployed as an azure app then we could likely fix our security issues of not being able to setup VNETs and PEs on multi-container apps because it’s been a “up-coming” feature for at least a year.

I’d be interested to know your thoughts on it. The efficiency doesn’t matter as much, our workloads are around 1-5% of our Azure budget which is basically a blip on the bigger budgets.

I’ve used Consul for instances where I’ve needed some of the benefits of k8s but without all the overhead of running k8s.

I’m fact currently using Consul for one such problem as it happens: managing Docker from inside Docker.

Are you sure you mean Consul and not Nomad (or are you talking about some other Consul?)? In either case, could you elaborate how you do it?

Maybe they just need service discovery and load balancing (as in replacement for kubernetes service). I may be wrong and would like to here more too.

Are you running the multi-container apps under App Service?

Yes, with the docker compose option.

App Service won't allow the level of privilege needed to start the "head" container that enables container-in-container to work. Options like `--privileged` and `--cap SYS_ADMIN` or mounting special volumes from the host are the first step on "how to escape from a container" so shared container services don't allow them. The options might be more allowable in your own App Service Environment but I don't think the API is even there to set the options in the first place.

Container-in-container also pushes a number of responsibilities from the the App Service down into the head container, like ingress, service management, health, logging etc. You might as well run compose/swarm on your own VM(s) than reinvent those wheels.

AKS + kompose might be a not too bad option. AKS can deploy on the private vnet and can do private endpoints. The cluster will probably fall over once a year for <reasons> but it will mostly manage itself. If you leave k8s to auto upgrade, run a microk8s instance as a test env somewhere and it will hit the upgrade issues before AKS releases a k8s version.

- https://github.com/kubernetes/kompose

- https://learn.microsoft.com/en-us/azure/aks/private-clusters...

are you using Azure Container Apps?

Do you mean their vendor-lock-in but much easier to use kubernetes? We can’t because of EU laws prohibiting us from that kind of vendor lock-in.

Reply on news.ycombinator.com