ZeroNet and IPFS: uncensorable auto-scaling BitTorrent powered websites | Hacker News Reader

ZeroNet and IPFS: uncensorable auto-scaling BitTorrent powered websites

turnkeylinux.org

97 pointsnpongratz24 comments

Drawback of ZeroNet is that it can host only static pages, page can't be generated dynamically. Max size for a page is 100MB. I tried to wget and convert my 5yo wordpress blog to static files, but it currently takes 1600MB, so... zeronet is no go for me, I2P and Tor work fine. Honestly I can' think of good example what zeronet should host. Maybe plaintext pages with leaks from WikiLeaks?

They have some examples of dynamic apps, like reddit style.

See: https://bit.no.com:43110/Talk.ZeroNetwork.bit/

However, how they are doing this is by having the owner's bot accept writes and perform a git merge with one big static JSON list. At least that is what I remember reading.

Which doesn't seem scalable or fast. We're trying to solve that problem with https://github.com/amark/gun , and hopefully will be able to team up with the ZeroNet guys at some point. I love their work.

The owner of a site is unneeded for the site to continue to work in a dynamic fashion (outside of the single 'ZeroBoard' demo that was made before the new system was in place).

Interactive sites allow users permission to sign/post in a particular directory of the site (data/userpublickey/stuff.json). They sign/publish this like the owner of the site would. The site can either access the .json files directly, or compile them into an SQLite database locally and read from that.

No bots are needed at all. AFAIK, git itself isn't used in ZeroNet at all, besides open-sourcing it's development.

Essentially, the entire site's functionality is entirely local. You can fully use any website (except those that do weird requests to outside of zeronet) without any internet connection.

Try visiting the zerotalk site, disabling your internet connection, making a post/comment, and then refreshing the page. You'll see that the comment is still there. It's just inaccessible until other users obtain the .json file you've modified.

Likewise, many sites are freely cloneable, so you can try running your own local copy and acknowledge that there isn't a bot on the backend.

Thanks for a better explanation! Still have some questions:

I don't run my own client, but use the browser proxies. So what is happening then, service workers? Or you have to have the client in order to actually post to the website? Such that there is file access to mutate the local files or do SQLite stuff?

How does sync/merging those .json files work then? This is what I thought git was for. The peer just sends the edits to the local files they have made out to the other peers with git.

>Try visiting the zerotalk site, disabling your internet connection, making a post/comment, and then refreshing the page. You'll see that the comment is still there. It's just inaccessible until other users obtain the .json file you've modified.

How does it deal with conflicts? Is that a chronological list that gets merges after sync?

> Drawback of ZeroNet is that it can host only static pages, page can't be generated dynamically. Max size for a page is 100MB.

http://127.0.0.1:43110/1KmvqK1WUidEA5vH3KVb6xExAfW7maj9jK

about 5 GB site

Quote:

"In terms of censorship resistance, IPFS includes blacklists of forbidden content in the default distribution, though it's configurable so you can turn it off 'at your own risk'.

Personally I think all of this willful censorship stuff is just a phase to protect the protocol in its infancy. Blacklisting all forbidden content on a global filesystem is ridiculous. The only popular use case I see for that is blocking ads.

So censorship-free distributions of IPFS implementations that integrate with Tor will probably end up as the default if this ever catches on."

Unfortunately, IPFS' built-in censorship tools are not meant to be optional, nor are they temporary. https://github.com/ipfs/faq/issues/47 is an FAQ entry written by an IPFS member. Quote:

"IPFS has as a strict requirement that content be able to move as fast as the underlying network permits. this rules out designs like freenet's and other oblivious storage platforms, as the base case."

"Oblivious storage platforms" are systems like Tahoe-LAFS or OFF, where storage nodes are unaware of the actual content that they are handling, usually due to some sort of encryption. Quote:

"IPFS has as a design requirement that nodes be able to only store and/or distribute content they explicitly want to store and/or distribute. This means that computers that run IPFS nodes do not have to host 'other people's stuff', which is a very important thing when you consider that lots of content in the internet is -- in some for or other -- illegal under certain jurisdictions."

So, IPFS is explicitly censorable; if enough IPFS nodes refuse to carry some content, then that content is effectively unavailable. Quote:

"users and groups can express what content should or should not be stored and/or distributed. This is required by users to (a) comply with legal constraints in their respective countries, (b) required by users with stricter codes of conduct (i.e. content that is legal but undesired by a group -- e.g. a childrens website)."

So it's possible to create walled gardens in IPFS.

What does this all mean? IPFS is awesome for making content highly redundant and highly available, but it fails at being uncensorable, it fails at being oblivious, and it fails at advancing the state of security for those using it. The analogy to BitTorrent is not just facile, but also very accurate.

A more compelling use case than ad blocking is child porn. One of the problems with networks like Freenet is that people do not want to donate resources to replicating and hosting unknown (but known to be substantial) amounts of CP. It's a personal moral thing as well as a legal thing-- it's not something I or most other people want to in any way support.

Security is another use case. It would in theory be possible for malware to be blacklisted by members of the network.

What you call a "problem" I call the fundamental strength of a network like Freenet over IPFS. If you can impose your moral beliefs on what content ought to be available on other users of the network, then the network is useless. Controversial and unpopular speech is the exact kind of content that these networks must be designed to support. If they can't do that, then they fail their primary purpose.

For a content host (ie the one who bought the hardware, who pays the bills and maintains the equipment) any system that puts all the cost on me and gives all the benefit to someone else is useless. See the Tragedy of the Commons and Free Rider problems.

I'm sure for some people the externalization of the costs and risks are the benefit. But since most of those people won't be contributing anyway, who cares what they think.

Unpopular content falls off of freenet as well (or used to, I'll admit I haven't studied it in years). If no one is requesting it it won't end up stored on the various nodes and will eventually disappear.

The difference between Freenet and IPFS is that IPFS is opt-in for duplicating data, where as Freenet was not (you couldn't even opt-out, only on the total amount you would store).

Not to mention that content filtering can imply liability for anything that does manage to slip through.

That doesn't seem like a reasonable liability. If I've got a spam filter that's pretty effective, but some false negatives still happen I don't see how I'd be liable for that. Filters like these are always heuristics, they're not omniscient so can't be 100% correct (false positives and false negatives will happen).

Doesn't this "fundamental strength" also doom it to be abysmally slow?

Quick proof: If there were a network like Freenet (meaning, nodes can't control what content they host) that wasn't abysmally slow, companies that currently pay to host stuff (e.g. mobile app banner ads) would move their files on to it until it was abysmally slow again.

P2P networks like BitTorrent and Freenet are designed such that the most popular content is the most replicated and therefore, quickest to access. Popularity makes these networks stronger and faster, not slower.

> If they can't do that, then they fail their primary purpose.

That's not the primary purpose of IPFS, and you've been given quotes/links saying as much.

It might be your primary purpose, in which case IPFS (on its own) isn't suitable, so use something else (e.g. FreeNet). Why criticise IPFS for not having the same goals as you, when you could say the same things about the World Wide Web, Gopher, IRC, email, Gnutella, DropBox, etc.?

It "fails"? IPFS has never been about obliviousness, it's a distributed CDN.

You can make oblivious storage (and even ORAM) on top of IPFS, without too much work actually. A critical point here is that making something oblivious by default is a nonstarter for 99% of users on the internet, because major consumer and corporate applications would never use something that (a) adds that much latency to requests, or (b) may have pushed "bad bits" to their computers. This is a design constraint BECAUSE of adoption.

The point is to establish IPFS as a base layer, and build the oblivious storage platform on top.

Which is exactly what we're doing with Peergos[1]. IPFS definitely made the right choice here.

[1] https://github.com/Peergos/Peergos

> So, IPFS is explicitly censorable; if enough IPFS nodes refuse to carry some content, then that content is effectively unavailable.

This is not a definition of censorship I'm familiar with. If no one wants to host a file, there's nothing for the censor to do. There are currently no IPFS nodes willing to store a copy of the essay I wrote on plate tectonics in eight grade, but that's not because it's been censored.

Let's remember that when we are talking about censorship, there are different cases. There is blocking something 99+% of the population thinks should be blocked, like child porn. And then there is blocking things much of the population wants to see but the government doesn't like, like a political dissident's website or wikileaks docs. I would be happy with a solution that would make it possible to block the former, but not the latter.

IPFS does not, and does not try to, solve the sybil attack, which has been a hard problem in securing decentralised systems for the past decade or so.

So using "uncensorable" here is talking it up way too much; it will not survive attacks by nation-states. More research is needed in these security topics, and the IPFS guys don't take those seriously enough. At the very best, this is highly-resilient to random errors; malicious attackers, not so much.

However, in order to censor data, you'd have to have all the nodes connected to the target be Sybils. This would be hard. If not, I think IPFS uses gossip-style protocols to query the rest of the network if your local node links don't have the data. Correct me if I'm wrong.

That's not actually true. IPFS is like BitTorrent, maybe, but it's not BitTorrent.

Reply on news.ycombinator.com