It's one more protocol and one more executable (sftp-server) where things can go wrong and be exploited. Because it's less popular than the pure shell, it's likely less tested. So the right security posture, if you don't need it, is to disable it.