I just ran into this issue after recently disabling IPv6 on my machines, a fix is to add "AddressFamily inet" to the /etc/ssh/sshd_config file and reload the ssh service.
And some people still don't understand why distros are so eager to dump X11 for wayland.