Can you elaborate on what this means?
With an SFU you end up having thousands of remote peers each with their own port on your server. However you could easily listen on a single port and then handle the inbound packet depending on what the remote 3-tuple is (clients ip/port/protocol). Effectively you would just be running all your ICE Agents on one port, but doing one additional step of processing.
I need to fill out [1] more to fully explain the idea, but I think it could make a huge difference when making it easier to deploy WebRTC SFUs.