GitTorrent: A Decentralized GitHub (2015) | Hacker News Reader

GitTorrent: A Decentralized GitHub (2015)

blog.printf.net

314 pointsmbroncano56 comments

Hi! I'm the GitTorrent author. It feels strange to have it show up on the front page whenever GitHub does something bad, more than five years later.

Bitcoin and BitTorrent v1 were plausible substrates in 2015, but no longer. I think Radicle (https://radicle.xyz) is the project most deserving of attention at the moment.

The hardest thing about such a project is economic: how do you handle issue/comment spammers? Or someone creating as many accounts as they can? How do you incentivize someone to mirror your repo? Or to build you a delta to satisfy your `git pull`, without making the network fall over when someone realizes they can ask you to spend infinite CPU building packfiles for them? And what happens when someone forgets their password or loses their private key? This space was a humbling experience in the necessity of incentive alignment for me.

Usenet solved this with a Bitcoin precursor, Adam Back's hashcash.

In effect you can do the same thing better by using a cryptocurrency with cheap transactions, e.g. Bitcoin Cash's Memo.cash. This way git hosts are compensated while at the same time making it too costly to spam.

Longer packfile creation processes will require a greater transaction fee or otherwise be ignored based on heuristics created by the client. This will prevent infinite CPU building packfiles issues in addition to fair compensation for increased computation cost. All while the miner is incentivized to minimize the time to be able to reap that block reward in the context of market competition.

Wrt crypto, this is the LBRY [1] approach I believe, and seems to work quite well.

[1] https://lbry.tv/

Absolutely, because it is truly innovative - and you're mostly in uncharted waters. We're at peak cloud right now, and what projects such as yours offer is a glimpse into the post-cloud future. We're going to look back and say - what were we thinking when we decided to move compute and personal data storage into datacenters while carrying powerful supercomputers in our pockets.

Add:

> And what happens when someone forgets their password or loses their private key?

Here's an approach I found interesting: https://darkcrystal.pw/

bookmarking dark crystal to read later.

With the cheapness of CPUs networking and storage I’m still amazed that there isn’t a revolution in hardware offerings for single purpose boxes. If you need photo storage you should be able to walk into any store and buy something the size of an external hard drive that you can just plug into your router and it just works. Same thing for email, private messaging, social networking, calendars, whatever. Most people don’t use enough data on these things to make it a problem. The biggest challenge would be security but you should be able to buy an off the shelf firewall and have some form of automatic updates. We’d need to all agree to use existing open protocols and there’s a decent amount of code to write to enable federation but it’s not an impossible task.

Actually, alas, the challenge here is the durability of storage. Changing a disc out and synchronising its contents is out of reach for average joe, and nothing is foolproof enough. The cloud is about giving up responsibility, and sadly most people give it up gladly

These devices could be sold with access to an encrypted back up service so you can have you local data and search it and it's encrypted locally before being sent to an off-site back up. But then you get into those businesses going under, people not maintaining thier accounts, companies who do not actually encrypt because users really want cloud access at the end of the day, etc.

When I think of something that we should be able to do but can't ("I just want a button to do X" type requirements) it is usually apparent why that doesn't exist if I give it some thought.

I don’t think site security is the main concern for the average person when it comes to this use case. They run the same risk with all of their physical possessions and there are existing solutions for this eg firesafes. Making sure that drive failure doesn’t wipe out your photos or files would be more important. Multiple redundant drives are the solution. If each is a small SSD and plug and play it’s something anyone could handle by just buying a new drive and swapping it out. Physical data appliances need to be as simple as a lamp. Plug it in, turn it on and occasionally replace a light bulb.

Or use encrypted mutual backups on others like systems (eg you friend buys the same model and you each clone each other’s, transparently, securely). Storage is cheap so you just over provision the space so that it can be both super durable, and have space to backup friends’.

I think doing some peer to peer striping should solve this. Selling a 3TB that only uses 2TB with an extra $5 to pay for a matchmaking service for 20 years.

This.

Aren't you describing a NAS?

Yes it turns out I am. I did some googling and found some decent options. I do wish that there were more network appliances in this vein and that they were more widely developed for and used rather than the current cloud infrastructure.

I think that such problems requires humans to make a decision. There should be a vote mechanism / leader election / democratic process to give some people the power / admin role to solve these issues by manual intervention.

The radicle website seems to do a bad job in advertising itself. I'm missing links at the top for a simple explanation of the project and quick tutorials to get radicle running for existing GitHub/GitLab/Gitea users. If you want people to use your project it has to be easy.

I'm lost browsing the radicle website and all its links. Ultimately, I found https://github.com/radicle-dev/radicle-link , following the link "browse the protocol", which mentions: "WORK IN PROGRESS - In fact, there is nothing substantial to see here yet". Is it a yet unreleased software? (that would explain why I can't find a way to try it out).

If it's the case and if anyone from the project is reading this, it would be helpful to mention it on top of landing page.

Also, the landing page is mentioning ssb, is this related to git-ssb?

> how do you handle issue/comment spammers?

I love how ssb is handling that : it's not a problem because you create a "web of trust" and only content from people you trust and people they trust reaches you - and trust can be revoked in case of problem. Obviously, it creates an other problem, though : how a benevolent total stranger can reach you?

By the way, yet an other alternative is hypergit ( https://github.com/noffle/hypergit ). It uses hyperdb/dat. I used it a few times to share code with friends, I loved it because it was the most straightforward for them to install and use.

But web of trust hasn't been very successful in practice. It's the usual reason quoted [citation needed] for the failure of PGP/GPG to become a mainstream success.

Keybase's approach is more scalable, but it's still centralized.

PGP’s web of trust hasn’t been a mainstream success because (and I say this as a dedicated GPG user) the UX is awful. I don’t think it has anything to do with the web of trust though; in fact GPG doesn’t really expose the web of trust ideas much to the user.

I can easily imagine a web of trust system with UX more similar to Facebook’s (and in fact this is basically what SSB’s clients are).

https://radicle.xyz/radicle-link.html

It frustrates me that they use the relation "on top" backwards, but otherwise it's a good read.

Radicle provides a network overlay and gossip protocol, on top of which they run the git smart protocol (they describe it the other way around, which makes no sense).

How does this compare with having a Github/Sourcehut like "website" hosted via IPFS? It appears that Git's versioning model might be nicely compatible with IPFS.

You use federation. Federation means ppl can get banned, but there's no central authority for who gets banned - different instances have different ppl banned.

Then just pick an instance you like.

When I saw this Radicle I immediately thought of Radicale[0]. Completely unrelated to the current topic, but I thought someone else might find it interesting that two projects share a very similar name, but are completely different.

[0]: https://radicale.org

After thinking about this for a couple of years while helping the Safe Network (a decentralisation project worth getting to know) I began my own effort to decentralise github based on Safe, just last Sunday!

Even before Friday's censorship, I was pleasantly surprised by the level of support and offers to help I received in mastodon. People there are aware, skilled and ready.

BTW I've got ideas on how to handle the issue/comment spam and other problems rightly highlighted by @cjbprime in his reply to the OP. But first I have to get git-bug (really worth of support too) compiled to WASM and running in the browser.

It's early days, but you can see what I'm up to here: https://safenetforum.org/t/safe-git-ui-discussion/32793?u=ha...

Or follow: https://mastodon.technology/@happybeing https://twitter.com/safepress

This is a solution to a non-problem. Hosting taken down git repos is easy an due to gits design all developers already have the source code.

The real problem is hosting issues and PRs in such a way. Github has an API and it's possible to script the backup but source code gets backup automatically so when the takedown strikes it's not a big problem.

While others could have a backup of a Git repository, I need to find them first — so discoverability is a problem. Next thing is integrity - how do I know that the git history I receive has not been tampered with?

Do signed commits solve the integrity problem?

Yes. Git commits include the hash of the previous commit, creating a cryptographic chain. If you can verify the signature at the tip of a branch you effectively verify the complete history of the branch.

Sadly not every maintainer signs their commits or tags.

Shameless plug, but that's exactly the aim of https://github.com/MichaelMure/git-bug.

It's a distributed bug-tracker: it stores issues (and one day, PRs) within git. You can work offline and you always have a full copy of everything. It also has bridges for Github, Gitlab and Jira.

Yes folks, take a look at git-bug, Michael has done a brilliant job of adding issues and comments into your git repos, and for a pre v1 project it works really well already, and you can import from / to github, gitlab and JIRA.

git-bug is a pleasure to use so I'm attempting to get it running in the browser using wasm, to create a decentralised github on p2p storage. I'm targeting Safe Network but the same approach could be used on anything with a storage backend, from NextCloud to IPFS, even [cough] AWS.

Fossil is an approach to this.

https://fossil-scm.org/home/doc/trunk/www/index.wiki

Not git obvs, but Fossil keeps bugs (wiki, forum) as part of the repo.

https://www.fossil-scm.org/home/doc/trunk/www/bugtheory.wiki

And another angle, if you want to store a snapshot of a repo on IPFS https://github.com/whyrusleeping/git-ipfs-rehost

This proviedes just read only publishing currently.

Can one simply place a git repo in an IPFS directory (e. g. with some IPFS fuse implementation) and share an IPNS / DNSLink so that future changes can be found under the same address?

In this aporoach github issues and pull requests can be replaced by discussions and patches in a mailing list.

1. That's basically just a bash script though, not something very fancy.

2. How popular is IPFS? How easily usable by lay users?

How popular it is I don't know. It's easy to access through the ipfs.io public gateway, and you can use your own gateway if you host a node.

I don’t understand the need for crypto and/or a global user name. I’m “withinboredom” in a lot of places and it doesn’t bother me when someone gets there first. I also have other names that are better well known, but I digress.

Why not just stick a txt record on a domain? You can clone gittorrent://awesome.withinboredom.info.

You can easily find a valid gpg key for me in the usual places.

Honestly some hash or public key would probably work fine. "withinboredom" means nothing to me (no offense) and it would be easy to add your ID as a contact or git remote and never deal with it again.

This project really needs more attention. Would have resolved the issue we had earlier today. Some others ideas:

* Write a client in C, JS sucks

* Make a web-frontend for this so average users can jump on quickly.

There’s also git-ssb

https://github.com/noffle/git-ssb-intro

Unfortunately, SSB uses far more JS than they really should.

Regardless, just learned about https://radicle.xyz/, which looks more active and mature than what I linked

I understand the sentiment “JS sucks,” but not in relation to C for most things. Modern js is rather nice in syntax, and quite performant in V8. Heck, you can transpile it to C with Bellard’s QuickJS (or just compile directly to binary) if there’s some C library interop that your comment is based on (although nodejs has a perfectly good C++ plugin system for this).

Looks like a good fit for youtube-dl

Yeah it’s an apt time. Of course I’m not sure the legal request will stick.

The maintainers should probably just run their own git server as a hidden service, change their git name/email/signing keys and be done with it.

Going after youtube-dl so publicly has a solid Streisand effect. Feel as though the lawyers aren't going to give up easily on this one and will hound them one way or another.

Regardless, there is some github alternatives available through tor:

Darktea: http://it7otdanqu7ktntxzm427cba6i53w6wlanlh23v5i3siqmos47pzh...

Rootgit: http://rootgit4rghbuenb.onion/explore/repos

And on I2P:

http://git.volatile.i2p/

http://git.psi.i2p

Switching to hidden services while we're building with decentralization does seem to be the pragmatic thing to do.

Thank you.

I have created the repo at Darktea, feel free to create your anonymous accounts and join the project, until we find a better/decentralized place.

For now this should be sufficient:

http://it7otdanqu7ktntxzm427cba6i53w6wlanlh23v5i3siqmos47pzh...

I will give the main contributors the neccessary access for reviewing/merging/etc.. or will entirely hand over the project if you can prove you are the maintainer.

Bittorrent V2 may offer some advantages that improve this idea.

another promising project: https://pijul.org/

I believe the main developer of pijul has temporarily made his work closed-source for the past few years while he rewrites it to avoid receiving issues while it's unfinished. Looks like it's supposed to release soon, though [1].

[1] https://twitter.com/pijul_org/status/1319159283938983936

Why, yes, as long as each cloned repo is a copy of some set of revisions, it makes perfect sense to use peer to peer downloads.

Internal hash consistently could be used for verification.

Wouldn't an activitypub[1] type of github be the perfect solution? Like Mastadon or Peertube but for git. Anyone can setup their own hosting and interconnect to others. There can be global index like peertube is building and spam can be dealt with on each instance.

[1] https://www.w3.org/TR/activitypub/

This is being worked on: "ForgeFed" https://github.com/forgefed/forgefed

Reply on news.ycombinator.com