My read (not an expert) is that you are safe if your rsync is only via secure connections, to & from systems where untrusted parties can neither run rsync, nor play clever games with the files which rsync is accessing.

Which (in my paranoid opinion) is pretty much the only secure use case anyway, for code like rsync.