Show HN: YouTransfer – Self-hosted file sharing | Hacker News Reader

Show HN: YouTransfer – Self-hosted file sharing

youtransfer.io

133 pointsrbolte76 comments

YouTransfer is a simple but elegant self-hosted file transfer & sharing solution. It is an alternative to paid services like Dropbox and WeTransfer by offering similar features but without limitations, price plans and a lengthy privacy policy. You remain in control of your files.

Created to be installed behind the firewall on private servers, YouTransfer aims to empower organisations and individuals that wish to combine ease-to-use file transfer tooling with security and control.

You can watch a live demo at http://demo.youtransfer.io

If you want to see it in action on your own environment, you can use the Docker image (https://hub.docker.com/r/remie/youtransfer/) or NPM package (https://www.npmjs.com/package/youtransfer)

There really needs to be a link to a demo on the project homepage or README file. The description says it's "simple but elegant", however I couldn't find a link to see it in action until I came to the HN comments to say so.

Edit: Sorry, just saw the Demo wiki page now. But a link on the homepage wouldn't hurt as it's what most people would want to see.

You're right! I was actually still working on the demo and am a bit overwhelmed by the attention generated by the HN show case. I've update the README and website, thanks!

Sounds fairly similar to bradfitz's Camlistore (https://camlistore.org/).

If you're familiar with that project, could you comment on the main differences in YouTransfer?

I'm not familiar with camlistore, but a first glance at the project website tells me that it has way more features and serves a different goal.

YouTransfer is basically a very simple hit & run file sharing application. The goal is to make uploading & sharing files a matter of 2-3 clicks. The files will be stored with a configurable retention time and will be deleted from the server after they expire.

YouTransfer it's not meant to help you organise your digital life, or have access to all your files remotely. It will only help you share files in a more convenient way compared to SFTP or email.

It's not at all similar. Camlistore is your private datastore for life with sharing-capabilities, YouTransfer is an on-premise WeTransfer.

If I read it correctly, this is how a file token is generated, which is supposed to be secure:

     file.id = md5(file.name + (Math.random() * 1000));

First of all please do not use MD5 for anything anymore, it has known collisions. But you shouldn't also use any hash functions here at all: just generate a long enough random token. Math.random is not a secure PRNG, use crypto.randomBytes in Node or window.crypto.getRandomValues in browsers.

PS And multiplication by 1000... is it just for fun?

Thanks for scrutinising the codebase! You are absolutely right that there is no need for creating a hash. This was just plain laziness on my part. I've created an issue (https://github.com/remie/YouTransfer/issues/101) to change the token generation.

Just to reiterate what dchest said, you should never use MD5 anymore, even if you do intent to hash something. MD5 is is broken and should not be used for anything anymore.

> MD5 is is broken and should not be used for anything anymore.

With the possible exception of demonstrating brokenness in hashing algorithms ;) (Sorry, couldn't resist!)

MD5 is is broken and should not be used for anything anymore.

Actually, HMAC-MD5 is secure.

It's a stretch to say it's "secure".

Yes, there aren't any known attacks right now, but since MD5 itself already has practical collision attacks against it, there isn't any good reason to use HMAC-MD5 in a new cryptosystem when there are better alternatives.

---

Supporting evidence: new versions of OpenSSHD do not use HMAC-MD5 by default anymore: it has to be enabled manually.

  The default is:
  
  umac-64-etm@openssh.com,umac-128-etm@openssh.com, 
  hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com, 
  umac-64@openssh.com,umac-128@openssh.com, 
  hmac-sha2-256,hmac-sha2-512

http://www.openbsd.org/cgi-bin/man.cgi/OpenBSD-current/man5/...

What about Gravatar?

Of course, you'll have to use whatever the third-party API uses, which includes Gravatar, but don't use it for anything under your control.

Replace with:

- BLAKE2 if you need a fast cryptographic hash function (e.g. for hashing file contents)

- SHA512 (or SHA256, or SHA-3) if you want a standard cryptographic hash function that is available in your programming language libraries

(Speaking of Gravatar, their use of md5(email) or even more secure hash function won't help protect email addresses against dedicated attackers, as it's easy to iterate over billions of them in seconds, just like in the stories about password cracking you hear, but it works as a simple anti-spam measure.)

Please just stop using it? It's the feature no one asked for.

I am not a very important person but I certainly ask for it, and I'd like you to stop discarding me and my "kind" as "no one".

Do you ask for that with the awareness that gravatar then gets to track your presence around the internet for the cost of a tiny picture?

I would rather have less ad tracking pixels on someone elses websites if possible, but I am genuinely interested in the value that gravatar provides to people who like the service.

There's a lot of things that can track presence around the web and gravatar's not one of them. Iff you decide to implement gravatar without mirroring their images, avatars are indeed loaded on their first query (and not subsequent ones) and, god knows what they're doing with that information ohgod. They certainly can't "track my presence around the web", though - no js means no fingerprinting, no tracking cookie, nothing. Juuuust a blind IP address.

But the recommended way is to prefetch the avatars directly from your server and offer them on your own cdn.

As for the value it provides, well for one thing I pretty much never have to upload my avatar to websites anymore - it's an avatar attached to my email addresses instead and that's very nice. Of course I'd prefer a proper identity protocol but nobody's working on one. If you want to, be my guest...

I think we disagree on one basic topic, I dont want an easily distinguishable identity to track across the internet.

I would rather external actors (say gravatar does nothing wrong) not be able to identify which email address I use on a site they do not own, and not be able to track my user signups by something that might be public information, which generally a site does not advertise.

It just feels wrong.

Unlike every actual tracker on the internet, Gravatar is opt-in. Things like google analytics, facebook like button/tweet button etc, they are all opt-out.

Good on you for taking the criticism well.

This doesn't seem like a "create an issue on github" problem though. Surely it is a push a patch today problem?

Working on it as we speak! I'm currently running the tests and hope to have a 1.0.2 hotfix ready by lunch.

EDIT: The 1.0.2 hotfix is now available with the token generation fix as well as 2 other enhancements.

Don't worry, it's not critical or anything. JupiterMoon just seems to be a bit of an entitled *ss.

I thought that HN policy was that posts should keep a civil tone.

My post was more civil than yours, so what's the problem?

Basically forcing him to put up a patch for an unimportant issue on the same day with your entitled comment was a really crappy thing to do.

I've opted for the 16 bits version right now as it fits better in the UI. There will be an additional issue to deal with improving the bitrate as well as making a suitable UI for it.

EDIT: I'm using `crypto.randomBytes(16).toString('hex')` to be precise

I hope you meant 16 bytes :-) If so, it's fine.

Edit: yep, 16 bytes.

And it's done. The 1.0.2 hotfix is now available with the token generation fix as well as 2 other enhancements.

Great! Make sure it's long enough: at least 16 bytes (32 hex characters) if you want to keep compatibility with current tokens, 32 bytes (64 hex characters) ideally.

Good stuff. Sorry if I came across as overly critical.

I'd use a uuid

RFC 4122¹:

6. Security Considerations

Do not assume that UUIDs are hard to guess; they should not be used as security capabilities (identifiers whose mere possession grants access), for example. A predictable random number source will exacerbate the situation.

Do not assume that it is easy to determine if a UUID has been slightly transposed in order to redirect a reference to another object. Humans do not have the ability to easily check the integrity of a UUID by simply glancing at it.

Distributed applications generating UUIDs at a variety of hosts must be willing to rely on the random number source at all hosts. If this is not feasible, the namespace variant should be used.

① https://tools.ietf.org/html/rfc4122#section-6

UUIDv4 is the only version of UUID anyone should ever use, and it should be generated with a secure PRNG, so using UUIDv4 is fine for this purpose (it has 122 bits of entropy).

You’ll excuse me if i believe the text of the RFC over some unsubstantiated claim from some person on the Internet. Expecially regarding a security issue, where the RFC explicitly says it’s not secure, and you go “naah it’s fine”. The RFC could easily have said “only use UUID version 4 as secure identifiers”, but it didn’t; I choose to believe that it does this for a reason.

UUIDs still need a sound source of randomness.

It's a different use case, but I love magic wormhole for directly and securely sending files to people:

https://github.com/warner/magic-wormhole

It's file sending from 1995.

Might I suggest that you split the folders up in your uploads.

0b692a00635682fabc78b6a50655242c.binary gets stored, for example in "./uploads/0b/69/2a/0b692a00635682fabc78b6a50655242c.binary" etc. Too many files in one dir can have problems or be slow.

Good suggestion! I've added an issue on GitHub (https://github.com/remie/YouTransfer/issues/106).

BTW: normally the files will expire within a specific timeframe and will be removed by a scheduled cleanup process. This should limit the impact, but if the system is heavily used it might become a problem.

You might want to fix the XSS [1] on the page and prevent the path traversal (try typing ../config in download input)

[1] XSS Example: http://demo.youtransfer.io/download/%3Cscript%3Ealert(%27xss...

Good catch! The XSS error was introduced with the implementation of error handling, but is a really unwanted side effect :)

I've created two issues on GitHub (https://github.com/remie/YouTransfer/issues/107, https://github.com/remie/YouTransfer/issues/108) which will be fixed in a new hotfix release asap.

I made this once pomf.se went down:

https://github.com/SirCmpwn/sr.ht

pomf.se was my replacement after my own hosting service, MediaCrush, went down.

I've been searching for a pomf.se replacement for some time now, so this is great! Can I request an account?

I would prefer if you ran it on your own infrastructure.

Oh, thank goodness. I was looking for a solution like this less than a month ago and couldn't find ANYTHING usable.

Much appreciated, added to my list o' useful things.

Thanks! I'm glad I could help :)

Released version 1.0.3 which fixes the XSS and path traversal security issues (as well as 3 other issues, see https://github.com/remie/YouTransfer/issues?q=milestone%3A1.... for more info)

These kind of projects are a lot of fun. I wrote a simple one, in Ruby, using Sinatra here:

https://github.com/skx/web-file-publisher

But then as an experiment I wrote another which uses TOTP to authenticate uploads, so you can upload a file directly via CURL with a suitable TOTP device. This is written in golang:

https://github.com/skx/go-experiments/tree/master/publishr

Using TOTP limits compromise if your upload is sniffed, although I run it behind SSL so I'm protected against that regardless.

Interesting project though; and I like that you can deploy it via Docker.

Thanks!

I've just seen the demo. What about security? Can anybody just "dump" files on your server?

Also, when people run this at home on their home computers there is limited upload speed (typically 1/10th of the download speed).

Basically... yeah, if you do not take any additional security measures, anybody can just "dump" files on your server.

You could opt for the S3 storage provider, which will dump the files to Amazon AWS instead.

The YouTransfer project does not implement access control or SSL, so it is highly recommended that you look at the hosting options on the Wiki (https://github.com/remie/YouTransfer/wiki/hosting).

I'm afraid there is not much the project can do concerning upload speeds of individual connections at home :)

This is very nice! Personally I'd really like some kind of login for the uploader so I can offer this service to friends and family (and myself ;)) without the risk of someone discovering the url and using it as a way to distribute illegal things... Perhaps it is easy to do with Apache/Nginx (when the upload site is on another subdomain for example), I don't know actually.

The speeds issue can be solved by running on a cheap DO droplet or scaleway arm server btw (my city luckily has fiber everywhere :)).

The problem with ACL is that I'm worried it will make the project more complex. I've added an issue on GitHub for future reference (https://github.com/remie/YouTransfer/issues/105)

Facebook integration might be nice here so that you could simply allow your FB friends, or perhaps those in a certain group, to share files.

The software looks interesting, however the web page is hard to read due to low contrast and very thin fonts.

Join http://contrastrebellion.com/

Are you referring to the generated GitHub pages on http://youtransfer.io or to the demo instance (http://demo.youtransfer.io) which is the actual application?

Probably http://www.youtransfer.io/

I'm having a little trouble, too. Even making the type a little bit darker would help.

I'm a bit hesitant to change this as I'm using the GitHub site generator for convenience. There is a limited set of templates available, most of which are either ugly or have readability issues. As the website basically only consists of the README file, you can also look at the GitHub project for more information (https://github.com/remie/YouTransfer)

haha, fair enough. I just checked my github, and as you can probably guess, I've just used their template which is pretty low-contrast.

Hoes does it compare to Seafile or Owncloud?

It has less features :)

The success of services like WeTransfer or Dropbox is that it is dead simple to use. It does one thing (sharing files) and makes this as easy as possible.

Seafile seems to be easy enough, yet still has a multitude of features compared to YouTransfer. OwnCloud simply has a whole different goal. It's not about sharing files, it is about organising your entire cloud presence (with e-mail, calendar, foto's, etc).

With YouTransfer, you can have the same ease-of-use but on your own terms. It runs on your own servers, with your own (secure) storage. You are in full control.

Given that it is also published as an NPM package, YouTransfer can be modified to suit your specific needs. This makes it interesting for companies to rebrand it and use it as their file-sharing system.

Congrats on the project. Without loading and looking myself, how are the file accessible on the server? I asked because I'd like to have clients use this to send ME files only. Is that an option?

Currently, the files are stored as a flat file list on the file system using their randomly generated token to create a [token].json and [token].binary file. The JSON file contains meta information, the .binary file is the actual file.

Using the default settings, you would get something like "./uploads/0b692a00635682fabc78b6a50655242c.binary" in the application directory.

I've already has plans on making it possible to change the interface, for instance not allowing direct download from the homepage. I could ament this with the feature to send email notifications to the system administrator upon successful file transfer. The combination of both would allow you to use YouTransfer.io as a public drop box for files. Does this sound about right to you?

Yes, it sounds as expected. Thank you again for your work.

The amount of of infrastructure you have to deploy to use this seems excessive.[1] You need NGrok (a commercial service) to get through NAT. You need Docker to install. You need Heroku, Microsoft Azure, Amazon AWS or Google Cloud to host. Then you need a reverse proxy, HAProxy, Nginx or Apache Httpd, for some reason.

How many sysadmins does it take to screw this in?

[1] https://github.com/remie/YouTransfer/wiki/hosting

Actually... you don't need any of those. It's only a suggestion. You can also install NodeJS, download YouTransfer, set the port to 80 and run it.

However, it is highly recommended to either use Docker, a reverse proxy, any of the PaaS providers or a combination of the above.

EDIT: I've updated the wiki with additional information on running it locally.

I think those are supposed to be alternate options, depending on where and how you want to run it.

How is this related to Dropzone? http://www.dropzonejs.com/

It's not really related. YouTransfer.io uses DropzoneJS as the file transfer UI for javascript enabled browsers.

Any chance for a more readable font?

BTW: if this concerns the youtranfer.io website (instead of the YouTransfer application), I'm actually a bit hesitant to change this. I'm currently using the GitHub site generator for convenience. There is a limited set of templates available, most of which are either ugly or have readability issues. As the website basically only consists of the README file, you can also look at the GitHub project for more information (https://github.com/remie/YouTransfer)

Any suggestions?

I created a simple Stylish

  @namespace url(http://www.w3.org/1999/xhtml);

  @-moz-document domain("www.youtransfer.io") {
  body{
    color:#000000;
    font-size: 18px;
  }

  }

Not San Francisco not San Francisco not San Francisco ;)

How about Proxima Nova? (http://www.marksimonson.com/fonts/view/proxima-nova)

Simply "Sans" will work just fine.