Hacker News Reader

> Not trying to start a Docker vs. Ansible flame war

Don't worry, they serve very different purposes. You already probably know but Docker is for running applications in isolation, while Ansible is for provisioning and configuring hosts. For instance, you won't use Docker to harden sshd on your hosts but Ansible.

Are there any good resources (besides OP) for using Ansible in a home-server environment? Especially in a mixed docker + Ansible environment.

After figuring out Ansible work I started using it to manage several personal VPS machines and the various boxes at home. I just used the Ansible docs, which are pretty good.

The official doc, StackOverflow and /r/ansible are good resources imo.

https://github.com/davestephens/ansible-nas

Uses ansible + docker to setup and run a home server. Its feature-set is close to that of a typical FreeNAS setup in a home environment.

Just start a simple project with Ansible and have a look at the repositories using Ansible to install Docker and deploy containers on your host afterwards.

You may want to put some effort (not that much) into managing your credentials with Ansible Vault[0] and you can try your playbooks e.g. on a Vagrant Machine[1] before applying them to a real host.

Any tutorial will do for the beginning but you should always notice what version of Ansible you are using (vs. the one used in the tutorial) as features change and also there have been some changes to the syntax to improve readability of your playbooks.

[0]: https://docs.ansible.com/ansible/latest/user_guide/vault.htm... [1]: https://www.vagrantup.com/docs/provisioning/ansible.html

They definitely have overlap. People use Ansible for much more than hardening sshd.

Yet they are both security disasters.

How is Ansible a security disaster?

A system designed to SSH into production host with high privileges and make changes automatically instead of enforcing staging...

That’s not really a security disaster unless you consider all of SSH a security disaster.

If you are concerned about Ansible SSH’ing in (which means you are concerned about any person SSH’ing in), you just do the standard SSH hardening.

...and yet SSHing on a production system to make configuration changes is prohibited in many large companies including some FAANGs.

Because you (or some intern hired a week ago) are 1 typo away from making a disaster.

For the record, security includes availability and this is an availability risk.

How is docker container, running a process as unprivileged user with reduced capabilities security disaster? With Kubernetes you can even have network policy or service mesh with mTLS. Compare it to bunch of bare processes running on the same machine, lots of services listening on localhost, difficult updates due to different dependency version requirements by different software, requiring you do the manual, painful update only once a year. It may look bad at first, but container can be far more secure that common non-container environments. It's always matter of configuration and maintenance.

It has a huge attack surface, also a lot of its code runs with high privileges. There's plenty of documentation on this.

Sovereign can be used as is to create linux container with LXD without any changes. You can create container image and than use it across your immutable infrastructure.

Don’t waste time in Docker, for this purpose use lxd containers [1]. LXD containers are more secure than docker in general and provide support for unprivileged containers for over 5 years. You can use your ansible scripts to create and manage your container images the way you manage bare metal or a VM. Indeed the modern LXD can help you seamlessly manage VM or Linux container [2], because when you need to directly access underlying host hardware VM is still more secure than container. Also with mount syscall interception in version 3.19 of LXD [3], it is possible to mount NFS inside container in a more secure way with unprivileged container running in user namespace.

Initially docker itself was built using lxc [4], but then decided to re-invent the wheels to build its own libcontainer without any significant advantage over lxc, just NIH. Obviously given all the money which went into docker and than kubernetes using it. Docker is more famous in spite of being inferior because of the marketing money spend on it like Java (a language which is famous because, sun spend over 500 million in early years of its inception on marketing). Kubernetes is famous and a valid tool for google level of problems for 90% of startups LXD is a better fit. This is the same fight like old times when inferior technology wins due to sheer marketing like Blu-ray won but rendered not as useful, same is docker.

[1] https://linuxcontainers.org/

[2] https://discuss.linuxcontainers.org/t/lxd-3-18-has-been-rele...

[3] https://discuss.linuxcontainers.org/t/lxd-3-19-has-been-rele...

[4] https://vyomtech.com/2014/03/04/docker_and_linux_containers_...

Come on. I like LXD, and I used it before docker. Docker is a standard and community. You can find prebuilt containers online and layer your own additions with a single configuration line.

Don’t get me wrong, Docker has made some stupid technology decisions, but the network effect of so many adopters means that it should be the default choice for any container situation.

> ...the network effect of so many adopters means that it should be the default choice for any container situation.

People assume that just because Docker and lxd do "containers", they are somehow equivalent and it's simply a matter of choosing between the two. But they are far from equivalent. It's not a choice between two competing technologies that achieve the same goal at all.

That’s what I wanted to say docker is only designed with specific goal to have one process running per container. So to run a system like Sovereign will need many many containers, one each for each daemon and then environment variables for each config settings. It’s a nightmare to manage. LXD is designed for multi-process with proper init and do not suffer zombie process problem of docker and unlike Docker supports unprivileged containers.

Personally I will prefer model like Guix System instead of container to run systems and services.I find that solution to be much more elegant than Linux containers or distributions. Linux containers initially came as chroot, jails and zones and than addition of cgroups and namespaces by google made it popular as lxc and later adopted and forked by Docker to make something complicated. These are bolt on solution to have immutable infrastructure. Guix is designed from ground up to be new OS for 21st century server and application infrastructure. May be it will or one of its derivative will become mainstream in time to come.

What makes it a nightmare to manage? My guess is it's all about how you do the declarative config, that fight is age old: http://mikehadlow.blogspot.com/2012/05/configuration-complex...

> You can find prebuilt containers online and layer your own additions with a single configuration line.

That's the kicker that got them over the line. Look at usage graphs and docker is running circles around Linux containers. I use docker professionally but privately will use lxd barring some complicated setups that are a docker pull away.

Truly can't stand some of their design choices, eg the utter distain for iptables and even non-technical ones like requiring signup to get the daemon on Windows/Mac, it's frustrating software overall whose only saving grace is the ecosystem around it.

"LXD containers are more secure than docker in general"

Please explain. This statement doesn't look correct to me. Both use the same technology - namespaces and cgroups. LXC is just meant to host the full OS installs, so you have to manually do things like "apt upgrade", resolve all breaking changes manually etc. So you end up with bunch of VM-like full OS installs, taking lots of time to manage.

Docker is basically the same, except there are layers of filesystem data and that those base OSs are minimal. Minimal also means less attack vectors, btw. Now, in the image there are all the required dependencies and you can prepare new version in your laptop, resolve breaking changes, test it properly and then easily deploy.

So why are LXC containers more secure?

LXD/LXC by default launches unprivileged containers in user space and map the userid’s to hosts user id’s, now they use new kernel feature shiftfs which makes it much easier to provide consistent user id mapping inside a container across hosts, with consistent security model, it doesn’t exist in Docker as it was designed from beginning not to have init process which results in zombie processes inside docker containers and recommends only one process per container. Docker doesn’t by default launches containers running in user space, and there are so many design difference as I posted a link in another post which really makes it a security nightmare unless company like google provides you managed container services taking the burden away from you, but indeed the security vulnerabilities which impacted all k8s didn’t impact LXD running unprivileged containers.

Every image in LXD can be locally hosted including the base one privately, so do not need to rely on inspecting a hotch-potch of Dockerfile, scripts and pull from other docker images to know what’s inside.

Now with the release of LXD 3.19 they introduced interceptions of syscall so even unprivileged containers Running in user space can securely access hardware. So NFS can be mounted inside unprivileged container I haven’t tried the latest docker container but in the old one cannot mount NFS from inside a container without running it in privileged mode with kernel access.

[1] https://linuxcontainers.org/lxc/security/

The two are not incompatible. You can deploy containers with Ansible.

Indeed:) Ansible will happily interface with pretty much anything; consider that it officially supports Swarm (https://docs.ansible.com/ansible/latest/modules/docker_swarm...), k8s (https://docs.ansible.com/ansible/latest/modules/k8s_module.h...), and puppet (https://docs.ansible.com/ansible/latest/modules/puppet_modul...) for example:) It makes excellent glue if used with other things, or a good tool all on its own. Which is especially nice because it lets you try new things without changing your overall setup too much; you can deploy all your services with RPMs, and then move one thing into docker, and still use ansible for everything:)

I run everything in containers and I spin up all the images / containers from ansible scripts. Am I doing it wrong? I didn't think these two tools were in conflict.

They do complement one another but docker has tooling that solves similar problems. Does docker-compose (https://docs.docker.com/compose/) fit your needs?

I tend to run ansible to setup a VM, e.g. a few packages, lock it down to my taste, and then have ansible run docker-compose to deploy the application(s) of interest.

You wouldn't use docker to manage the host networking (which is where ansible comes in), but packaging whatever is listening on ports as a container works really well for me

I use ansible to build a machine from zero software installed on it to running multiple docker containers that also make use of various filesystem stuff on the host, as well as set up other non-docker things on the host like networking, selinux stuff, and crons. I like that ansible is not hardwired towards everything being a container.

No need for an Ansible vs. Docker argument when you can combine them both!

I run a similar project called Ansible-NAS - https://github.com/davestephens/ansible-nas - which originally came about because I fell out of love with FreeNAS, and felt I could do a better job with Ubuntu, Ansible, and a bunch of Docker images.

Sovereign is awesome, I've been watching it for a while, but I'm not keen on everything being installed directly onto the system which is what I tried to solve with Ansible-NAS.

</shameless_plug>

Docker use Dockerfile (ini) to create container images, so you need to create a soup of Dockerfile, bash scripts combine it with ansible scripts. So it is kind of incompatible, it's better to use LXD containers as I suggested in my previous post for this kind of purpose. Also if you know Docker always maintained one process one docker container so a multi-daemon system with SSH and various mail and groupware servers running is not right choice for Docker.

Using docker for such things is like putting a square peg in a small round hole.

Dockerfile format is not ini. https://docs.docker.com/engine/reference/builder/

It also doesn't usually make sense to put SSH in a docker image or container, as you can enter the container using docker exec

> I would say my biggest piece of advice is to learn Docker. (Not trying to shill for Docker here

The first step to learning Docker by downloading it, irritatingly, is that getting Docker Desktop on Mac or Windows requires creating a Docker Hub account and signing in! There’s even a long issue thread about this on GitHub and the response was totally irrelevant! Luckily for those who don’t want to jump through these hoops (through disposable addresses or reusable shared logins) or provide an email address, there are many people who have posted direct links to the different downloads available.

I think you can use chocolatey to choco install it without going to the docker site.

I remember when they changed that and the GitHub issue was one of the most hilarious examples of corporate doublespeak.

"Why are you doing this?"

"To improve user experience"

"But now I have to sign up for an account whereas before I could just download"

"Yes this will improve the experience for everyone over time"

> In my personal experience, Docker images (and docker-compose orchestrations) are more pervasive than Ansible playbooks and when it comes to self-hosting for personal use, hosting all my services in Docker has made it much easier to deploy/maintain them

Are you building every container your self? I hope so: https://blog.banyansecurity.io/blog/over-30-of-official-imag...

:) Yeah, that was one of the big reasons I emphasized that my setup was focused on personal use. I have decided that the potential vulnerabilities present in the official images that I use present an acceptable level of risk that I am willing to assume. Enterprise deployments and other people with different personal levels of acceptable risk will require a better (but much more time-intensive) setup.

Thanks for sharing these list of FLOSS alternatives. Have you had much experience with MailU and anti spam / deliverability issues? My needs are only a few emai's a week, but previous tests with running my own mail server resulted in a high % of emails ending up in client spam folders.

Are there any resources you recommend for learning to use Docker specifically for self-hosted services in a home lab setting?

IMHO the official Docker documentation is great! Their "getting started" page is a good overview of the Docker basics. https://docs.docker.com/get-started/ The CLI --help is pretty useful too. I have also found Docker's in-depth documentation to be very helpful when trying to figure out the more complex features.

I got started by just grabbing a $5 DigitalOcean droplet (can get them with Docker pre-installed) and then played around trying to setup a simple app. (I think it was RocketChat.) https://rocket.chat/docs/installation/docker-containers/

https://www.linuxserver.io/

not a resource but docker containers you might want to run in your homelab. checkout awesome docker on github or katacoda