Basically... yeah, if you do not take any additional security measures, anybody can just "dump" files on your server.

You could opt for the S3 storage provider, which will dump the files to Amazon AWS instead.

The YouTransfer project does not implement access control or SSL, so it is highly recommended that you look at the hosting options on the Wiki (https://github.com/remie/YouTransfer/wiki/hosting).

I'm afraid there is not much the project can do concerning upload speeds of individual connections at home :)