Hacker News Reader

Comment by prima-facie | Hacker News Reader

Did they sort out rootless networking? I've tried podman twice but simply cannot get networking to work properly when using a docker-compose file.

View on HN

I haven’t had issues with rootless networking but I run pretty simple apps. I gave up on podman-compose though and run individual commands to create podman objects. Once rootless containers are in the same pod you can access other containers in the same pod over localhost

Podman has a Docker-compatible service API, so you can use it via docker-compose.

I embraced pods and now I don’t think I would want to go back to docker without them.

Once I get everything built the way I want I dump the config to a yaml with “podman generate kube” for future reference.

It’s also nice being able to use systemd service files to start/stop everything, and podman has a command to generate those too.

Since docker-compose is designed to exclusively work with rootful containers (basically the only thing docker supports), it's not a surprise you're having problems trying to run it rootless.

Podman added an optional rootful daemon a few releases ago that you can run in place of/next to the docker daemon that (theoretically) supports all the same docker network functionality specifcally so docker and docker compose commands can be run unchanged on podman.

I too have issues with this, but mainly trying to run protected ports (port 53 for DNS with PiHole) as rootless Podman -- it just did not work. Went back to docker and have not had an issue since.

Just don't use rootless. You can still do user namespace containers (the real benefit to rootless, IMO) as a root user. I predict we will find all sorts of fun, niche security bugs that were made to get rootless containers working, especially networking--linux really doesn't like unprivileged changes to the networking stack.

If you already have sudo/root access for your user, rootless is just going to cause more problems.

I think that’s by design. Non-root users aren’t supposed to be able to listen on protected ports. If you need to run PiHole as a non-root user, you could start the container rootless to listen on 5353/udp (arbitrary). Then (as root), add a firewall rule to redirect traffic from 5353/udp -> 53/udp. That probably the only way to make this work while still running the container as a non-root user.

There are two ways around this:

Frist, you can run the container with --cap-add=net_bind_service which allows processes to bind to privileged ports:

  $ podman run  --cap-add=net_bind_service --rm -it --user=1 localhost/socat 
  bash-5.1$ id
  uid=1(bin) gid=1(bin) groups=1(bin)

  bash-5.1$ capsh --print
  Current: cap_net_bind_service=eip
  Bounding set =cap_chown,cap_dac_override,cap_fowner,cap_fsetid,cap_kill,cap_setgid,cap_setuid,cap_setpcap,cap_net_bind_service,cap_sys_chroot,cap_setfcap
  Ambient set =cap_net_bind_service
  Current IAB: !cap_dac_read_search,!cap_linux_immutable,^cap_net_bind_service,!cap_net_broadcast,!cap_net_admin,!cap_net_raw,!cap_ipc_lock,!cap_ipc_owner,!cap_sys_module,!cap_sys_rawio,!cap_sys_ptrace,!cap_sys_pacct,!cap_sys_admin,!cap_sys_boot,!cap_sys_nice,!cap_sys_resource,!cap_sys_time,!cap_sys_tty_config,!cap_mknod,!cap_lease,!cap_audit_write,!cap_audit_control,!cap_mac_override,!cap_mac_admin,!cap_syslog,!cap_wake_alarm,!cap_block_suspend,!cap_audit_read,!cap_perfmon,!cap_bpf,!cap_checkpoint_restore
  Securebits: 00/0x0/1'b0 (no-new-privs=0)
   secure-noroot: no (unlocked)
   secure-no-suid-fixup: no (unlocked)
   secure-keep-caps: no (unlocked)
   secure-no-ambient-raise: no (unlocked)
  uid=1(bin) euid=1(bin)
  gid=1(bin)
  groups=1(bin)
  Guessed mode: UNCERTAIN (0)

  bash-5.1$ socat -dd - TCP-LISTEN:22
  2023/02/21 09:19:34 socat[82] N reading from and writing to stdio
  2023/02/21 09:19:34 socat[82] W ioctl(5, IOCTL_VM_SOCKETS_GET_LOCAL_CID, ...): Inappropriate ioctl for device
  2023/02/21 09:19:34 socat[82] N listening on AF=2 0.0.0.0:22

Second, you can run the container with --sysctl=net.ipv4.ip_unprivileged_port_start=0 which tells the kernel to allow any process to bind to ports above 0 instead of the default of 1024:

  $ podman run --sysctl=net.ipv4.ip_unprivileged_port_start=0   --rm -it --user=1 registry.access.redhat.com/ubi9/ubi-minimal 
  bash-5.1$ id
  uid=1(bin) gid=1(bin) groups=1(bin)

  bash-5.1$ socat -dd - TCP-LISTEN:22
  2023/02/21 09:26:55 socat[59] N reading from and writing to stdio
  2023/02/21 09:26:55 socat[59] W ioctl(5, IOCTL_VM_SOCKETS_GET_LOCAL_CID, ...): Inappropriate ioctl for device
  2023/02/21 09:26:55 socat[59] N listening on AF=2 0.0.0.0:22

That seems like a security issue if podman can do that without root access, yes?

As in -- on a multi-user machine, this could be a problem.

This is only within the context of the pod's network namespace.

Start the container via systemd and ensure it has CAP_NET_BIND?

Podman 4.0 brought a new network stack for rootless called Netavark. Works nicely for me.

What was wrong with it?

It is very difficult getting containers that aren’t in a pod together to be able to talk to each other, with rootless pods, and without just host-binding everything. None of the normal hostname conveniences are in place, and if it’s possible to do it isn’t default and probably needed much more underlying knowledge than I had when I tried.

I think the new network implementation in Podman 4 ("netavark") helps with that sort of thing.

So the following works:

    [sam@isis ~]$ podman network create foo
    foo

    [sam@isis ~]$ podman run -d --name=host1 --network=foo --init registry.access.redhat.com/ubi9/ubi-minimal sleep inf
    1f7ffa0e474f7116d782ffa4d8dd9c0454893e9dac54b2cd9e61481a7f86a2ff

    [sam@isis ~]$ podman run -d --name=host2 --network=foo --init registry.access.redhat.com/ubi9/ubi-minimal sleep inf
    d8ffa58cb9de77305545b9f85043c016d03193adb6543b1768a68623b044b0e5

    [sam@isis ~]$ podman exec host1 getent ahosts host2
    10.89.0.3       STREAM host2.dns.podman
    10.89.0.3       DGRAM  
    10.89.0.3       RAW    
    
    [sam@isis ~]$ podman exec host2 getent ahosts host1
    10.89.0.2       STREAM host1.dns.podman
    10.89.0.2       DGRAM  
    10.89.0.2       RAW

i.e., we have a network called 'foo' to which containers 'host1' and 'host2' are attached; they can resolve each others hostnames.

It's worth noting if you are trying this on a system where you previously ran podman <= 3, I think you have to 'podman system reset' to get a new configuration with the new Podman 4 network implementation.

(Personally I just run all containers with --host=net, this is just for local development, who cares)

I think this will work with docker-compose as long as you 'systemctl --user start podman.socket' and 'export DOCKER_HOST=unix:/$XDG_RUNTIME_DIR/podman/podman.sock'.

Had the same issue, the containers did not resolve each other by hostname. This fixed it (run on the host):

apt install containernetworking-plugins podman-plugins

That's not needed/used since Podman 4, where the networking stack was rewritten; one of the features of the new "netavark" stack is that it has DNS resolution for containers within a network by default, no plugin needed.

Reply on news.ycombinator.com