Freenet: A Distributed Anonymous Information Storage and Retrieval System (2000) [pdf] | Hacker News Reader

Freenet: A Distributed Anonymous Information Storage and Retrieval System (2000) [pdf]

snap.stanford.edu

152 pointsrayvy71 comments

I've always loved the ideals and design of Freenet for what it does, but the main issues I always ran into when I'd check it out were that discoverability was horrible and then once you did manage to dig into things and find content it was usually stuff I wouldn't want to find to begin with. That's left a lot of bad impressions on people I think, and caused it to have a reputation for only illegal things on it.

I had a similar experience with various other peer-to-peer networks: I2P (except the I2P torrent scene is pretty good), Zeronet, GNUnet. Tor stands out as the only one to have caught on.

Similar impressions here. Freenet was an impressive vehicle for exploring some very important topics and ideas. Unfortunately, besides the problems you mention, the community around it was ... unpleasant. On a good day they could seem a bit overzealous. On a bad day they were downright crazy, and they were always hostile to any advice or criticism from others even in their own field. For example, Freenet isn't a permanent data store. It's more like a cache; if something isn't requested long enough it just ages out. That's actually fine considering the goals of the system, but any time anyone mentioned it Ian and others would go a bit ballistic. Not sad that Freenet and its promoters have faded into obscurity.

> That's actually fine considering the goals of the system, but any time anyone mentioned it Ian and others would go a bit ballistic.

The system did not and could not store information permanently, if it did it would fill up, which is a non-starter when it depends on people volunteering their hard disk space.

I don't know about going ballistic, but since this issue is addressed directly in their papers (section 3.4 of the linked paper) I can see us getting irritated by people re-asking questions that have already been answered.

I would think you should have long ago learned the lesson that if you don't have a FAQ page somewhere with huge titles and colorful answers, people will never read a paper.

Hell, most people don't read FAQs either.

In future we'll use <blink> tags ;)

Despite the fact that it was mentioned in the paper, developers continued to deny that it was a serious problem or claim that it had been solved since. Periodic re-insertion and date-based redirects were both touted as answers.

As for "can't" that's not accurate. You can certainly prevent old data from being pushed out, by returning an error on insertion of new data if there's no free space. That's how many other storage systems (e.g. every filesystem ever) work. While it's true that you can't fully protect against a reduction of capacity when nodes go offline, that's a very different issue. Freenet being cache-like rather than storage-like was a decision of convenience, not a technical necessity.

Filesystems don't rely on space voluntarily donated by others, and that's the critical difference here.

Freenet is designed for sharing information, not archiving it. You can't have both.

Other systems have also relied on space voluntarily donated by others, and still made the choice to behave as storage systems instead of messaging systems with some history. Or caches that just happen to turn over slowly because the system's too slow and unusable to create cache pressure. If Freenet ever had really taken off, you would have had to deal with these issues. Sharing information doesn't work when the information falls out before the recipient can get it.

"it’s like a cache" is what we’ve been saying as the default description for years. When did you last interact with the freenet community?

> When did you last interact with the freenet community?

When it was still relevant, i.e. a long time ago.

then you missed quite many changes

That's rather inherent: if you create a network whose primary feature is anonymity and lack of rules, your early adopters will be everyone barred by the rules of other services, which means you become a cesspool.

Tor did a good job with marketing and optics early on, to make its intended use cases highly visible.

> That's rather inherent: if you create a network whose primary feature is anonymity and lack of rules, your early adopters will be everyone barred by the rules of other services, which means you become a cesspool.

> Tor did a good job with marketing and optics early on, to make its intended use cases highly visible.

I used to work on Freenet. The mechanic you're pointing out doesn't have to be the dominant effect if the system performs as well as the alternatives. If everything else is equal (speed and convenience), users would of course prefer to have more security than less. If the performance and usability is only slightly worse, users may still be willing to choose security over that (e.g. Tor) - people today are becoming more aware of just how much their data can get abused.

Unfortunately for Freenet the technology wasn't capable enough - the security wasn't worth the usability and performance tradeoffs for most users. That doesn't mean strong security inherently attracts unpleasant types.

VOAT turning into a cesspool of everything that was being banned from Reddit is another example of that, too.

“The moral of the story is: if you’re against witch-hunts, and you promise to found your own little utopian community where witch-hunts will never happen, your new society will end up consisting of approximately three principled civil libertarians and seven zillion witches.”

The solution to that may be to separate the hosting from the discovery.

If the hosting is anonymous and censorship-resistant then you'll get every type of content, but for anybody to use it they would first have to find it.

Then you have a slew of independent discoverability portals. Some are created for child pornography, so the FBI hacks into them and then uses them to arrest everybody there. Others are created to replace Tumblr and YouTube and refuse to link to child pornography, so child pornographers have no reason to go there and the people disgusted by child pornography can safely use them.

But setting up a discoverability portal is a lot less expensive than setting up a hosting service (because a link uses much less bandwidth than high quality video), so there should be more competition, and in particular it should still be difficult to censor anything that has more than a modest amount of popular support.

This is what Freenet already provides with three different messaging systems and anonymously run indexes.

Are you aware of any YouTube-equivalent? Something that allows you to search user-generated/submitted content and content feeds and has a builtin player to display them.

I am aware that we are just one release away from serving video in a video-tag. That’s not as convenient as youtube (because it takes time to load), but much closer than before.

For convenience, we need m3u lists of video-chunks and a simple player shipped, since browsers cannot play m3u out of the box.

Some of my best friends are witches ;)

It is amazing that we're still not used to the idea that freedom has drawbacks, and that even with those drawbacks it's still preferable to the alternative.

> Tor did a good job with marketing and optics early on, to make its intended use cases highly visible.

Even with their site saying one thing, the result in the media is clearly different. It's got a public reputation for only two things to those who've heard of it: darknet markets and ransomware payments.

It doesn't have bad reputation beyond some smear campaigns from law enforcement in some countries.

I'm glad they did. The "Who uses Tor?" page is still what I give to people thinking privacy/anonymity tech is only for crooks. I always point out it helps cops, folks fighting terrorism, journalists, and victims of domestic abuse. And that the only way it works right is if lots of people are using it so they can hide in the crowd. I add that crooks can use it, too, but they also use cars, cameras, cheap phones, and so on. Doesn't mean we keep those things public or under government control. These pieces together in the argument gets agreement or at least sympathy out of lots of people who media freaks out about Tor, etc.

https://www.torproject.org/about/torusers.html.en

It's no wonder really. If you build a system that allows anonymity and untraceability you will end up hosting everything that can't normally happen without anonymity and untraceability.

Hackers marvel at the idea of complete anonymity and untraceability because it's technically exciting and fits so well with the hacker thinking. I mean, why should you have to reveal who you are because of mere accountability? Everyone knows hackers don't cause harm and when they do it's only a good deed because the system they "broke" was obviously broken already. But the same technology is a much more prominent tool for criminals, molesters, and such. So that's what you will get on the anonymous networks and not so much hackers with good intentions.

‘Information wants to be free’

Interestingly, many of that principle's strongest proponents are selective in its application. While they're adamant that information in the sense of content should be free, they're equally adamant that information in the sense of identity should be strictly limited and controlled by its owners. Unfortunately, as soon as you remove identity you also remove accountability, and while there are edge cases (e.g. political dissent) where that's a good thing, most of the time it's not.

Personally, I think anonymity sucks. I'm all for pseudonyms, which have some continuity and reputation and thus some accountability. Modern cryptography provides ways to prevent impersonation, and even to claim a pseudonym if the owner desires, but when you consider methods such as traffic analysis it's not trivial. These are problems Freenet tried to address, but then they took a bit of a wrong turn toward complete anonymity. That, and the bad taste left in people's mouths by positioning a message system with some accidental persistence as a true data store, is why other systems eclipsed it.

Pseudonymity is what you get when you allow people to tightly control their own information. This is what Freenet uses: You have a private key to which you can upload and which you can use as pseudonym in messaging systems.

Yes, the public Freenet opennet contains lots of horrible content. But then, Freenet's focus has always been censorship resistance. Content gets distributed and replicated according to its popularity. It's a lot like BitTorrent, but with serious transport encryption and randomized routing.

And arguably it's worked pretty well for that, given that there's been consistently lots of horrible content for 18 years. Content providers (and users) have been busted, but that hasn't eliminated much of their content.

However, being a pure P2P system, you only have "anonymity" through plausible deniability. And that's a dangerous game to play. Criminal investigators have used modified Freenet nodes to track chunks of illegal content, and have prosecuted users whose nodes processed those chunks.

The Freenet Project argues that it's impossible to reliably determine whether nodes are requesting those chunks, or merely relaying requests from other nodes. But if you end up in court, you'll need an expert witness to convince the jury of that. So you'll likely end up with a plea bargain.

So anyway, it's safest to run Freenet on anonymously leased VPS, and access its webGUI as a Tor .onion service. Just as you'd prudently run a BitTorrent seedbox.

Is it even possible to anonymously lease a VPS?

I guess you would need to use cash, at a minimum.

Sure. Do everything via Tor, using Whonix to avoid leaks. Hit Tor through nested VPN chains. Buy some Bitcoin, and then mix two or three times. Use a different Whonix instance for each mixing step, and a different mixing service.

Then lease your VPS. And if it really matters, you can daisy chain VPS via Tor .onion SSH.[0]

0) https://www.ivpn.net/privacy-guides/onion-ssh-hosts-for-logi...

[...] Then lease your VPS

... With a credit card (which of course has your name on it), since AFAIK the providers tend to only accept bitcoin for established accounts.

No, of course not. I've found lots of VPS providers who accept Bitcoin. Just search a while, and you'll find them. Some of them have rather "unsavory" reputations, I admit. And you want to avoid the ones that charge a huge premium for being "bulletproof".

Vultr is good and cheap. You have to use a disposable visa gift card for your first $20 payment, but affer that bitcoin payment is enabled. the best visa cards are the $500 ones from simon malls-- 2.99$ activation fee and no ongoing or expiration fees. pay cash.

Also good:

Cockbox (Bitcoin and Monero) at https://cockbox.org/

HostSailor (cards, transfers, Bitcoin, etc) at https://hostsailor.com/

Pulse Servers https://www.pulseservers.com/

Trilight Zone (cards, transfers, Bitcoin, etc) at https://www.trilightzone.org/

VPS.BG (PayPal, Bitcoin, etc) at https://www.vps.bg/en

> Buy some Bitcoin, and then mix two or three times.

Simply exchange them to Monero. If you want, mix them, but only a single pass is needed. A mixer is a trusted 3rd-party, which introduces serious counterparty risk, and vulnerable to multiple forms to input-output correlation attacks, in the worst case, provides no security.

If you are paranoid, create multiple Monero wallets, and send the funds back and forth, each time with a randint(1, 100) hours delay to frustrate timing analysis. Please note that you should only run full node Monero, and must run it inside Whonix, otherwise, the unencrypted Monero traffic is exposed. The situation of P2P-level encryption of the entire cryptocurrency world is a disaster, almost all traffic is clear.

Ring-CT is possibly not the best cryptographic implementation of transaction anonymity, but currently the best practical implementation available. CoinJoin is a P2P version of mixer, but has no wide deployment, zn-SNARKs involves cutting-edge cryptography and provides 80-bit security (read, not confident), but 90% of Zcash transactions are clear transactions, makes private transactions vulnerable.

Or, once GNU/Taler is available, making an anonymous payment would be as easy as using PayPal.

Thanks. Gotta start using Monero, I guess. It's still no where near as widely accepted as Bitcoin. But you can just convert, as the final step.

I think there needs to be a way to have personal filters on P2P systems. I don't want censorship but I don't want to support the distribution of child pornography, not because it's illegal but because it doesn't align to my personal moral compass.

We already have that — it’s normal communication.

There’s no way to make the change you’re proposing without seriously compromising the set up of such digital, secure communications, though. You’re asking to MITM the secure net, as a feature.

You and the feds, both.

I get your concerns, but I believe that would require a major rewrite of the codebase. And it would likely introduce serious security vulnerabilities. It's been a recurring request, over the years.

That was my impression when I ran Freenet years ago. A few cool materials, a horrifying amount of pedophilia and other really skeezy stuff.

And this is why censorship is the future of the internet. If you run an online service and don't censor, you become a wretched hive of scum and villainy. Exhibit A: USENET. Exhibit B: Freenet.

There's a place for both highly moderated and free for all spaces. I prefer moderated discussion forums, because discussions work better without jerks. But I also value free for all spaces, because they guarantee that messages can't be silenced, no matter how unpopular and illegal they are.

When did you last look at Freenet? When did you last check the moderated discussion boards? (FMS)

There’s a lot of original non-scum user content on Freenet nowadays.

I'd like to see what can be done with anonymity but with more filters. When this software was first designed, AI-driven content classification was virtually nonexistent.

If bots can be designed to be moderators, I'm thinking if you were to fuse that functionality to an anonymous data storage, you would be able to ensure only productive content enters storage, and content that is irrelevant can be excluded. Of course that bot would need to know precisely what it should exclude from storage which seems hard to define.

It makes me wonder about use cases for tools like freenet, tor, etc. Espionage of some sort comes to mind and the need to deliver a message from sender to receiver without identifying any participants . Otherwise there is some other implicit recognition that anonymity can be productive.

Anonymity clearly changes how individuals communicate, but the research I know of tends to focus more on how people like to behave badly and mischievously when there is no known reputation or name associated with the consequences of a action.

Anyone who has spent any time on the internet knows the ability to obscure identity, however thin or unsophisticated, elicits changes in behavior. I highly doubt the producers of these tools construct them with the goal of inviting mayhem in mind. Regardless, the more anonymous a data transfer is, the less social pressure there is to communicate within certain permitted boundaries.

Outside of a authority point of view, there's also the potential for creativity and free association related to anonymity. If one feels you won't be judged because of saying something, you might open up.

https://en.wikipedia.org/wiki/McIntyre_v._Ohio_Elections_Com...

> It makes me wonder about use cases for tools like freenet, tor, etc.

Considering the content accessible through such services, their best use by far is as honeypots to catch dangerous criminals.

You might be interested in the method Freenet uses to avoid disruption of communication: https://www.draketo.de/english/freenet/friendly-communicatio...

" In the past decade there hasn’t been a year without a politician calling for real names on the internet. Some even want to force people to use real photos as profile pictures. All in the name of stopping online hate, though enforcing real names has long been shown to actually make the problem worse.

This article presents another solution, one that has actually proven that it keeps communication friendly, even in the most anonymous environment of the fully decentralized Freenet project.

And that solution does works without enabling censorship."

When I was young I thinkn internet is freedom. These day it is more of a control device by the state.

In the older days, you can hide from the state. These day you can’t. Your day to day activities are recorded.

China is just one example. And they are not the only one.

>A recent court case in the Peel Region of Ontario, Canada R. v. Owen, 2017 ONCJ 729 (CanLII), illustrated that Law Enforcement do in fact have a presence, after Peel Regional Police, located who had been downloading illegal material on the Freenet network.

I haven't looked into the architecture of Freenet, but would someone be able to give a quick rundown of how this is possible? I was always surprised there weren't more well established, long-form, human rights blogs on Freenet but maybe there are security concerns I'm not aware of.

Investigators run modified Freenet nodes that serve illegal content. They log connections with peers, and track chunks of illegal files, based on hashes. So if your node peers with one or more of their nodes, they can see if it requests any of those illegal chunks. If it does, you are "downloading illegal material".

Freenet is a pure P2P system. Your peers know your IP address, and vice versa. So your only defense is that your node was just relaying those illegal chunks to some other node. But that requires expert testimony, and a jury that's capable of understanding that expert testimony.

I enjoyed this nugget of info. But a few questions (if you can answer them)

> They log connections with peers Of course (I'm assuming via just IPs)

> and track chunks of illegal files meaning what exactly? How could they track a 5KB chunk being sharded across say 10 different nodes? (further, flexible nets like freenet adjust the location of data over time based on the use of said data)

> they can see if it requests any of those illegal chunks My understanding is that yea you can see who's talking to who (via IPs), but I guess traffic analysis is the way to see the chunks (i.e., Peer A sent Peer B 100 bytes at 5:65 PM PST)? Even then, the payloads are encrypted, so I'm not seeing how you could infer that the content is "illegal"

Again. I enjoyed that little nugget of insight you shared. Hoping you can share more :)

Missouri Law Enforcement's Freenet Attack Now Public Record[0]

Levine et al. (2017) Statistical Detection of Downloaders in Freenet.[1]

0) https://www.reddit.com/r/Freenet/comments/66f0n3/missouri_la...

1) http://ceur-ws.org/Vol-1873/IWPE17_paper_12.pdf

Awesome, thanks so much

I'm going to assume Ontario is using the same method that the Missouri police are using above. Here's a response from Freenet outlining an 83% false-positive rate.

https://freenetproject.org/police-departments-tracking-effor...

Yes, that's the Freenet Project response that I was thinking of. Thanks for citing it.

But the problem now is that there's technical backup for both sides. So defense counsel will likely need a credentialed expert to submit a report, be deposed, and testify. Although I haven't followed any of these cases, I'm guessing that many defendants have accepted plea bargains. Because battles of experts can get expensive fast.

And then you've got the challenge of explaining this stuff to a jury. And countering the emotional "evil child molester" rhetoric. I wonder if the Freenet Project would provide such an expert?

There is one. If you’ve been falsely accused and you need one, contact press@freenetproject.org

If that is the case, I would imagine it's like prosecuting a lobbyist for the bribe they're relaying.

Does anyone still use Freenet? What happened with it?

You can find usage stats on Freenet. But I found a couple reports on clearnet:

Uploaded: 2018-11-28 (UTC) https://d6.gnutella2.info/freenet/USK@sUm3oJISSEU4pl2Is9qa1e...

Generated August 25, 2017. https://www.asksteved.com/stats/

My brother worked on freenet it was cool until tor came up and supplanted it

No, not tor. I guess ipfs would be somewhat similar today, but it has its own problems and I don't see it succeeding either.

Care to elaborate problems of IPFS? I just recently set up a node (which I run while I heat my apartment with GPU mining) and am wondering if there's some other project I should give my spare cycles to.

Well, for one thing, they don't care about anonymity at all. And already implemented a mechanism to blacklist content hashes on the demand of law-enforcement agencies. Both these things essentially defeat the purpose of many enthusiasts who would use it -- and its only function remains that of a volunteer-driven p2p network for making sure certain data doesn't die. And that's it, nothing else. Even that is somewhat ruined by paid pinning services.

For many IPFS is enough for what it does. But in a world where WikiLeaks and Edward Snowden are facts of life, it falls vastly behind on what's needed out there.

I'm not an expert on IPFS, but the things I personally don't like about it is the lack of anonymity (although I think it is possible to run it over Tor), and also it seems to only be useful as a store of static files.

I don't think it's possible to do a dynamic site, eg something that needs to read from/modify a database, using IPFS.

Not sure what project is better however. Dat looks pretty cool but also doesn't have anonymity.

Freenet still does a lot of things better than Tor. For example, no server is involved so there's nothing to attack or track down, making anonymity guarantees seem to be much better. Yet they've still managed to pull off some forms of interactive communications.

IPFS is closer really, but doesn't move in an anonymity friendly direction at all.

> For example, no server is involved so there's nothing to attack or track down, making anonymity guarantees seem to be much better.

Not really "no server". I mean, Freenet users collectively are the servers. As in BitTorrent, I2P and IPFS.

What Freenet does better than Tor is keep content available, even if the provider goes away. As long as it's popular content, that is. Although Tor .onion services are perhaps harder to find, once they're gone, they're usually just gone. There are exceptions, of course, such as the The Hidden Wiki. But any mirroring that happens is entirely ad hoc. In Freenet, it's automatic.

The point is, Tor relies on establishing a connection with a server which will provide content, Freenet simply distributes data and that data can come from any client/server on the network, Tor's hidden services are modeled off the traditional client server model, while Freenet operates more like a P2P network.

That difference in model is what provides a potential anonymity benefit as there's no longer a path to the origin of the content

IPFS is indeed more similar. Tor is more of a compliment to a DFS such as Freenet (though Tor can very much works as a DFS itself)

Reply on news.ycombinator.com