How we use BitTorrent to improve our design sharing workflow | Hacker News Reader

How we use BitTorrent to improve our design sharing workflow

medium.com

63 pointssgottit24 comments

Note that the author seems to misunderstand a few things about BitTorrent sync:

1. Firstly, Btsync is not open-source (unfortunately), as he claims. [1]

2. 24-hour secrets don't give someone read access for 24 hours. They must be redeemed within 24 hours after creation, but if they are redeemed on time they become permanent read-only secrets.

[1] - Answered in these threads: http://forum.bittorrent.com/topic/8816-will-syncapp-be-open-... http://forum.bittorrent.com/topic/17782-bittorrent-sync-faq-...

I'm curious why this is being downvoted. Is the information incorrect?

Ah my bad – I've corrected the open source claim and I'll reword about the 24 hr secrets. Thanks.

There's git-annex for git with big files: http://git-annex.branchable.com/

"git-annex allows managing files with git, without checking the file contents into git. While that may seem paradoxical, it is useful when dealing with files larger than git can currently easily handle, whether due to limitations in memory, time, or disk space."

and

"The git-annex assistant creates a folder on each of your computers, Android devices, removable drives, and cloud services, which it keeps synchronised, so its contents are the same everywhere. It's very easy to use, and has all the power of git and git-annex."

I was hoping this was going to be something where the files themselves were distributed so that they didn't take up space on everyone's machines, a la MogileFS[1], but alas.

This is always a point of pain for us: in our studios we work from a central file server, so our shiny iMacs with their 1TB drives all have about 90% capacity on them — but we don't have any easy way to pool all that extra capacity and use it even as overflow for our central fileserver.

The search continues...

[1]: https://code.google.com/p/mogilefs/

That's what git-annex is for. Everyone has a list of all the files, and then if you want to access one, you can just do "git annex get [file]" and it'll get it from any of the machines that happen to have that file. If you want to delete it, "git annex drop [file]" will ensure at least N machines keep a copy of the file before deleting it.

I use it to manage files between my VPS, desktop, laptop and NAS, with each machine having a different subset of the whole pool, and it works great.

Is there any way to get around the fact that adding a new machine involves adding N*2 remotes? (That is, one remote on each of the current machines to represent the new one; and then on the new machine, one remote for each existing machine). This seems like it would be the sticking point with Git annex.

That's what the new Assistant (now part of git-annex) is for!

Tons of pairing work, which culminated today in pairing fully working for the very first time. And it works great! Type something like "my hovercraft is full of eels" into two git annex webapps on the same LAN and the two will find each other, automatically set up ssh keys, and sync up, like magic. Magic based on math.

http://www.kickstarter.com/projects/joeyh/git-annex-assistan...

http://git-annex.branchable.com/assistant/local_pairing_walk...

Congrats on using bitorrent for syncing files. You need at least RAID1 setup to don't lose information. Never use only one hdd.

How does this handle a situation where multiple designers edit large files at roughly the same time?

I also always wondered if there was some version control system for pictures.

Github supports some basic features: https://github.com/blog/817-behold-image-view-modes

There are a few, one by adobe that comes free with the Creative Suite. Some are integrated in Photoshop with a clever UI: http://sixrevisions.com/project-management/the-ultimate-guid...

If you want to transfer a big file easily without the need to upload it to a server http://www.sharefest.me does the trick. bit-torrent like service but instant and without any client.

That looks pretty slick. There was a static HTML version I could host on my site, do you know if this does that? I don't want my files going through any third parties.

Not sure what you mean there, but if you want to share files you can use this platform. If you want to hose your own version of sharefest the open source is in github.com/peer5/sharefest

Thanks, I meant I didn't want to have to run the node.js server, but I see it's only used for STUN/negotiation. Thanks for your reply.

" not to mention it means everyone who wants access needs to commit to losing 1TB of space on their hard drive."

Dropbox has selective sync for years already, doesn't it?

Yup, but for some reason most people don't seem to know about it. They could set up a shared top level project directory with sub directories for each aspect of the project and then people only need to sync the stuff relevant to them.

OP mentions this app "Arq" for AWS S3/Glacier backups. Does anyone know a similar app for Linux?

dschep is dead for some reason, so copying & pasting his answer so all can see:

Duplicity[0] supports it through a boto[1] backend. [0] http://duplicity.nongnu.org [1] https://github.com/boto/boto

Depending on what functionality you need, one alternative could be Tarsnap (http://www.tarsnap.com/) by HN's own cperciva https://news.ycombinator.com/user?id=cperciva.

IMHO btSync is a great ideia but the use of random ports makes hard to use in some setups.

You can set the port in the configuration:

http://labs.bittorrent.com/experiments/sync/get-started.html...

The closed source nature of Bitorrent Sync is biggest drawback IMHO. But I guess FOSS clones are on their way. The bit torrent protocol in itself is very well known after all.

Reply on news.ycombinator.com