Gnutella: A Protocol Outliving the World That Created It

I did a client called Gnucleus back in the day, the original website is still up at https://gnucleus.org/ I also designed GWebCache with a friend in college (also the IRC caching before that was me, I'm sorry). I think in the end the cost/benefit of the Gnutella architecture wasn't there. Especially in privacy - too much of a liability exposing your machine to serve files and route traffic. Also the efficiency of search over the Gnutella architecture - your query never hit all the peers.

BitTorrent was designed with the good parts like decentralized file transfer, and ditched the decentralized search - simplified with a centralized website/tracker model that can also be members only. That helped people stay more under the radar, as well as allow people to jump on/off to get what they want.

Ultimately the better balance which is why BitTorrent is still going strong today, but there is some nostalgia for the craziness of a single global network. Where people can freely share all their stuff, and downloading/opening files was like rolling the dice.

As the eldest of millennials, I regularly find myself going through “ah, they’ve reinvented a situation best served by Gnutellla” events. The most most recent was an ad hoc Resilio Sync library of epub files, and I did the “yeah that works… but have you ever heard of Gnutella?” routine I have become so accustomed to: “gather around the camp fire, young zoomers, for grandpa has a story to tell you”.

>Imagine we all had HTTP servers running on our laptops and could give our friends an IP address whenever we needed to transfer a file. In theory, an HTTP server on everyone's machine would be enough for file sharing, right?

Opera browser at one point included a personal webserver built in called Opera Unite. This webserver was accessed by the public through the free web proxy service that Opera also integrated. That way your personal home computer hosted website was behind an Opera IP address. It was all automagic. I really wish it had caught on. Simple static only hosting is a very small attack surface.

Firefox recently started offering VPN services built in to the browser. I think this is the perfect time to try to bring back static personal webservers in the web browser.

I'd say the biggest reason that Gnutella (and other services like it) is no longer in much use is because for a long, long time now the easiest way to download music for free, without ads, and with virtually zero chance of getting caught, has been via YouTube and downloader clients. For most practical uses, it's good enough.

You wouldn't want to share the resulting downloads (not only is the audio quality slightly degraded, but I imagine it's highly likely there would be audio watermarks), but when everybody can download straight from YouTube anyway with a minimum of hassle, why would you need to share anything other than a video URL?

Of course, a big part of why this is so simple is because of the massive amount of work that the downloader client devs put into working around YouTube's attempts to stop this. I imagine it can be a difficult job.

If YouTube ever win the battle against the downloader clients, I imagine the landscape will change again. Maybe Gnutella will make a comeback.

The problem with Gnutella was that it was always slower than the direct competitor, EDonkey (aka EMule, aka ed2k, aka Kademlie later, etc.)

Searching took ages (if you got any results back at all), and when you tried to download something it took ages to even start, dripping through your dial-up internet like molasses. EDonkey on the other hand was quick. The first search results usually arrived within seconds (granted, it took equally long to get all search results), and usually downloads started (slowly) after a few seconds to 1 or 2 minutes.

I don't know if this was because of popularity (more peers in ed2k so faster download speeds) or if it was a particular problem with dial-up internet (Gnutella worked better for people with fast internet, like at universities etc., at least from what I heard back then)

The main issue with Gnutella was - IIRC - that it didn't scale. At least the initial version, not sure if there was a revised one.

Basically, if you open the log window and look at the peer messages, then beyond certain network size all you'd see was a flood of relayed search queries with duplicates that ground all other activity to the halt. And the whole thing just became unusable.

PS. Also, Gnutella was released almost to the day when some clause of the AOL's purchase contract of Nullsoft (stock option vesting?) has expired so the devs were ultimately free to do whatever the f they wanted. So released the file sharing app. That was a nice touch.

Great article. It's been almost 20 years since I've used Gnutella, but I seem to recall that, as flexible and resilient as it was, it had some pretty major issues with speed of queries and scaling to millions of users. I think Bittorrent become popular in the mid-to-late '00s in large part because its model of redundant copies scaled better than Gnutella.

> The Gnutella project began as an internal demo that leaked to the public after its corporate overlord, AOL, cancelled the project.

I don't think it was actually a leak in the usual sense of the word. There was no unauthorized release; rather, AOL didn't really understand what their new subsidiary was releasing.

As I understand it, Gnutella was written as a new project by Justin Frankel, the author of Winamp (which probably did more to popularize using computers to listen to music than anything else!), during or shortly after the sale of Winamp and Nullsoft to AOL. It was probably a chaotic time, and a massive culture clash between this big behemoth of a late-'90s tech and communications company, and this small startup of Early Internet Nerds.

Nullsoft's new corporate overlords probably didn't understand what they were creating: a new file-sharing/music-piracy program that would be like Napster, but more decentralized and resilient.

Frankel/Nullsoft released Gnutella and it was downloaded by thousands of people immediately. Perhaps friends in the recording industry called AOL execs, or in some other way they finally understood what it was, and AOL shut down the downloads less than a day later and cancelled the planned open-source code release, but due to its decentralization the network kept running, and it was soon reverse engineered.

As far as I know, the source code of the original Windows implementation of Gnutella never leaked.

I would read a follow-up about LimeWire's dynamic query routing. I enjoyed the writing style very much and now I'm reading Rick's other articles on topics that normally wouldn't interest me. Thanks Rick!

Nice writeup. It goes deep into Gnutella, but it's also worth mentioning the slew of sharing programs back then, which was truly like the wild west. Napster, Emule, DC++, Kazaa, to name a few. On many of these networks it was possible to literally browse other people's sharing folders, find really cool stuff, and maybe make some guesses on the what this user was like.

> Gnutella has gone mostly forgotten. Some of that is because it was a component technology hidden beneath more visible projects like LimeWire. The other half of this is that the walled garden model of modern platforms means most internet users don't even remember what a filesystem is anymore.

Really? Not going to mention bit torrent?

Gnutella died because bit torrent took over in popularity. Bit torrent only did the parts that scaled pretty well, so it wasn't a true replacement, but it turned out that its difficult to go after index sites in foreign countries, so nobody minded putting index sites back on the http web.

Thank you for reminding me about this. Next to soulseek I'm going to use it to not obey!

I always find this kind of thing interesting. A protocol can outlive the companies, devices, and culture around it, mostly because nobody has to keep owning it for it to keep making sense.

On a tangent OiNK's Pink Palace never died either, if you know you know…

The high seas have never been better.

Hot take: the simple reason Gnutella declined is that it was replaced by Bittorrent.

A client for this was one of the first complete programs I ever wrote.

That’s what Limewire used? It definitely came pre-bootstrapped then.

just reading gnutella triggered a really old memory of times when Ares, Limewire and eMule where places to try your luck getting mp3s and software

Is Bittorrent connected to this. Isn't that still going strong?

Gnutellas real legacy is not the file sharing, it is the lessons it taught about protocol design. Flood-based query routing is awful at scale, but it is dead simple to implement, with no central server, no SPOF, and it works behind NAT. The structured overlay networks that came after, like Chord, Pastry, and Kademlia, fixed the routing efficiency problem, but they also added enough complexity to make them harder to deploy in practice. Most modern P2P systems ended up somewhere in the middle: DHT for discovery, direct connections for data transfer. It is the same tradeoff people keep running into with mesh networking and federated protocols today.