Docker particularly has always required extra work to run in rootless mode because it was released soon after in March 2013, and for whatever reason it hasn't been a priority to rework the codebase to make that the default. I switched to podman for exactly this reason as my go-to oci implementation and haven't looked back.
The linux kernel features that enable various forms of isolation all require root privileges (CAP_SYS_ADMIN). Once user namespaces were a thing, that allowed you to use user namespaces to get around the root requirements for all the other isolation namespaces.
All of the below still require CAP_SYS_ADMIN:
CLONE_NEWCGROUP: cgroup namespace, for resource control (mem/cpu/block io/devices/network bandwith)
CLONE_NEWIPC: ipc namespace for sysv ipc objects and message queues
CLONE_NEWNET: network namespace, for isolated virtual networking
CLONE_NEWNS: mount namespace, for isolated mounting (filesystems, etc.)
CLONE_NEWPID: pid namespace, for isolated view of running processes
CLONE_NEWUTS: unix timesharing system namespace, for isolation of hostname and domain name