Show HN: Gfile, a WebRTC based file transfer utility in Go | Hacker News Reader

Show HN: Gfile, a WebRTC based file transfer utility in Go

github.com

77 pointsAntonito42 comments

Despite how awesome WebRTC is, every time I see it my mind instantly goes to whether I understand STUN/TURN enough. They're part of the overall concept of ICE (Interactive Connection Establishment)[0], and STUN is a protocol for negotiated direct peer-to-peer connections[1] whereas TURN[2] is a relay mode that is used when a direct connection cannot be achieved for either side.

When I saw the claim "without the need of a third party", it didn't fit with my knowledge of WebRTC so I went spelunking through the code and found that they do indeed use a STUN server[3], in particular google's "stun.l.google.com:19302".

Just want to point out to those who might be relatively new to WebRTC, p2p mechanisms, etc -- it is surprisingly hard for two computers to talk to each other over the wide internet, even in this day and age. Even if you're behind the same router things aren't necessarily straight forward.

[EDIT] - Also note that in the case of TURN, it is up to the application layer to encrypt communications[4].

[0]: https://en.wikipedia.org/wiki/Interactive_Connectivity_Estab...

[1]: https://en.wikipedia.org/wiki/STUN

[2]: https://en.wikipedia.org/wiki/Traversal_Using_Relays_around_...

[3]: https://github.com/Antonito/gfile/blob/v0.1.0/internal/sessi...

[4]: https://stackoverflow.com/questions/23085335/is-webrtc-traff...

> it is surprisingly hard for two computers to talk to each other over the wide internet, even in this day and age

Only if both are behind NAT without forwarded ports and even then you can just use services like this[1] if you want to send files/data and it's super easy.

> Even if you're behind the same router things aren't necessarily straight forward.

I don't think this is true for most cases today.

1. https://justbeamit.com/

> It allows to share a file directly between two computers, without the need of a third party.

I'm guessing it still depends on STUN and TURN servers?

Yup -- https://github.com/Antonito/gfile/blob/v0.1.0/internal/sessi...

There's no TURN server specified, and I'm not 100% sure whether the one that's specified there (Google's) does both (so if STUN didn't work, would it fail or attempt TURN?).

Google only provides STUN server. Providing free, performant TURN server would require HUGE amount of resources.

Yup, it still relies on a STUN server to get the connection's metadata. As hardwaresofton pointed out, I'm using Google's STUN server.

Since it's only STUN, and not TURN, the actual transferred data does not go transit through it though.

Super late but thanks for clarifying this

Is there no way to run this, without relying on a third party server/service?

Honest question: what is the advantage over nc + compression utility of your choice?

If you are just learning Golang, then congrats on shipping!

Hey!

It can be useful if your transmitting data between two different clients on two different networks. In this case, you don't need to open ports on your firewall nor change your network configuration :)

Thank you!

Thanks for this! I've wanted to learn more about WebRTC and Go, and digging through the source seems a good opportunity.

To solve the same problem I've been using another [0] simple silly solution, but security is a concern.

[0] https://github.com/tpaschalis/goof

Glad you like it!

If you think of any feature/improvements, feel free to open a issue/pull request! :)

> gfile is a WebRTC based file exchange software.

> It allows to share a file directly between two computers, without the need of a third party.

Can it be done with a single HTML file instead of Go?

You know, which can be hosted anywhere even on Github pages.

There is Go WASM runtime, isn't there?

Yeah, you can compile to WASM since the last few versions of Go (1.10 or 1.11)!

I actually did such kind of experiment recently [0]. As a bonus, the users do not have to copy/paste the SDP data manually - instead it is transmitted through sound!

[0] https://ggerganov.github.io/wave-share

That project is really cool!

I'm planning to work on the SDP stuff at some point. It works well, but sharing two base64 encoded strings isn't very practical :/

The use case that excites me is transfer between servers/no X11. Just a single binary, no deps to worry about.

They could also be air gapped, both behind NATs and other non-standard topologies. ICE will take care of all the hard problems :)

> transfer between servers

A simple NC command would do.

Or something like python -m http.server and curl

Heh can't tell you how many times I've used that python http server. Sure enough no matter the OS if Python is installed, it gives me the edge if I can access machines behind the same network.

That wouldn’t work if they don’t have a direct connection (my main use case)

I have a bunch of servers with no public IPs split between different regions.

On a server it makes sense if there is no UI involved.

Potentially the sender could output a URL that contains the SDP. That way the receiver can either be a CLI or a web browser.

The page needs to run the appropriate javascript. That is all. Of course a STUN server at the least is required.

Yup, it can be done! Actually, it's one of my next objectives for it. I started to write a front-end client, with WebAssembly. It's not fully working yet, but you can take a look at it here: https://github.com/Antonito/gfile/tree/master/_client/web

Another similar service I've found useful is https://instant.io/ which does bit-torrent over WebRTC.

Didn't know about it, that's pretty cool!

I wonder if I can use this to send ZFS snapshots over WebRTC

Interesting, what's the use case here?

Possibly a more robust protocol to send/recieve datasets, perhaps involving a Project-FiFo installation.

How about a tool like croc instead? https://github.com/schollz/croc

croc currently doesn't do peer-to-peer with WebRTC, but I've actually been working on a branch that does. [1] Currently, croc uses a rendezvous server unless you happen to be on the same local network.

Thanks to Antonito (and their usage of MIT license) I'll fork their code into croc so it can do WebRTC with a secure PAKE channel.

[1]: https://github.com/schollz/croc/tree/v5

I'm not a fan of Go. If anyone is interested in developing a Rust equivalent of this (all Rust with no C/C++ deps) I'm willing to pay for it, contact me at andrew.forsure@gmail.com (end result can be opensource)

As someone who is personally a great fan of Rust, I have to say that this is a very defective attitude.

If you're not writing it yourself (WebRTC is super simple, and this tool is basically the absolute bare-bone example of it—make a pipe and transfer something over it), why in the world would you care about what language it is written in beyond whether the language is safe, performant enough, and not a nightmare to execute on the target PC?

If you're not making it, you don't get to pick the tools. Paying just to get decide tools for other people also seems like a really weird thing to do.

Tools affect the result. If you ordered a tool in C, you will get more features than you wanted.

Tools, at a high level, do not affect what the end result is. Instead, it affects how end result is achieved.

In real life, languages are more than tools, but as long as features/performance/behavior is acceptable, it still reduces to an implementation detail.

(also, unless segfaults can be considered features, I don't see why C would get you anything extra.)

I guess black hats would consider memory corruption welcomed features.

re: performance, as someone who isn't particularly enamored with rust or go (or any language), how is this thing only doing 16MBps local to local?!?

Transfer is done via DTLS, a protocol on top of UDP, so we have to deal with packet loss and retransmission, which itself is controlled by SCTP. gfile uses pion/sctp[0] (via pion/webrtc[1]), and there has been some work going on to improve performances. :)

The performances used to be far worse, but with the recent improvements on pion/sctp, I felt gfile had become fast enough to be useful to some people.

[0]: https://github.com/pion/sctp

[1]: https://github.com/pion/webrtc

You may want to look at QUIC. It'll be part of http/3 and it's on top of UDP. There is a go library called quic-go [1] that works quite well.

I was recently playing with a similar thing and used the lib [2]

[1] https://github.com/lucas-clemente/quic-go

[2] https://github.com/binwiederhier/natter

> If you're not making it, you don't get to pick the tools. Paying just to get decide tools for other people also seems like a really weird thing to do.

I'm making it, I'm a dev (but focused on other projects). I want a specific solution that I can use from my projects. That's why I have technical requirements, not functional ones.

Paying to have something rewritten from Go to Rust out of spite.

This is a new low as far as hatred that I've witnessed.

It's not so much "hey mom I want that too", but that I've been looking for a similar solution, but written in a decent language that I can consume from my app.

Reply on news.ycombinator.com