Which is to say, you don't need the borg executable server-side if you are happy to run it over SFTP transport which limits some functionality.
Yes, of course you could talk over an sshfs mount point but I can't think of why you would do that over the more basic SFTP transport.
The accepted, and full-featured, way to run borg is with the executable on the server side and I will point out that the borg project distributes a "frozen" version of the tool that allows you to run it without having python in your environment. I believe they use py2exe to pack it up.
This is important for us[1] as we have no interpreters of any kind in our environment (no shell, no python, no perl) so we can only run binary executables ...
[1] rsync.net