Demo: Fully P2P and open source Reddit alternative
plebbit-test.netlify.app
plebbit-test.netlify.app
In any case, it sounds like we'll be getting many new Reddit clones so I may as well plug what I've built recently: a Reddit alternative API that is free. Check it out if you're a dev that is being caught out by the Reddit pricing changes[1].
It's easy enough to centralize a caching layer, perhaps through a popular CDN.
I've had an idea in the back of my head for what would make a decentralized video service work, and it plays with the idea of peer driven networks.
So, there a few kinds of people on the network: advertisers (hang in there, I'll explain why), leechers (or consumer), content producers, and seeders.
The prime relationship is between a consumer and a content producer. Every other part of the network is designed to support and benefit their interaction.
First, a content producer creates a piece of content (or steals it, let's be realistic) and publishes it on the network. Attached to the content is a pay-per-view requirement; the producer has elected to require payment to view this content. This money can come from one of two places: first, an advertiser can bid on slots the producer has chosen, second, the consumer can pay the producer directly. The producer doesn't have a choice whether the consumer pays or not, only that payment is required.
Requiring payment for the content is optional, but advisable. You see, the producer has a second order relationship with the seeders. The producer can't possibly feed a million viewers in the first hour. That would be a very difficult task. But what a producer can do is elect to provide a percentage of their required income to seeders. Now there's a financial benefit to seeding a popular video/song. The producer can set their payments to whatever they want. The market may equalize itself.
Finally, we have advertisers, which are the monetary fuel for the fire. Most viewers will tolerate ads and wont want to pay money; so instead they pay time. An advertiser can bid on slots in as precise or general way as they want. If they want to bid on a specific creator's video slots, they can. If they want to bid on a broad category of creators or videos, they can. Etc.
This won't ever be truly decentralized, and there is a whole ecosystem of services external to the network that would be needed for this to function well (payment processors, content collation and moderation -- kind of like how mastodon works, a bidding marketplace for advertisers to interface with the content network, a front end for seeders to select and re-seed, and so on).
I don't have the technical hutzpah to build out something like this. I wish I did. Is there someone out there who wants to be a technical mentor? I'd be game.
--
[edit] There should be more and less granularity in how different parties engage. Some people want to pay 10 bucks a month (Amazon Prime) and forget about the rest. Some people want to pay per view. Advertisers might want to target specific creator's channels or whole segments of created content. Creators might want to be able to never even think about ads and just check a box that says "put an ad in front of every video I upload" and then forget it. Or a creator might want to put ads in videos longer than a certain length. And so on. What I've specified above is loosely conceived and would need a lot of thought to turn it into something useful.
Decentralization is a meme, the great majority of consumers don't care. People used Morpheus because it was an easy way to watch (free) movies, not because it was decentralized. Now they use Netflix because it's an easy (and relatively cheap) way to watch movies, not because it's centralized.
Besides, Matrix already has p2p chat, IRC has always been a thing, and so on, heck even BitTorrent is technically still alive. (De)centralization is not a feature, nor is it a bug, it's simply irrelevant.
I have no idea how the experience would vary for the advertisers.
For the producer it should offer more flexibility in how they get paid. This is the major selling point I think. The ability to put out a high effort video -- say you made a full length movie -- and be able to charge theater prices (17.50USD, or close to it) and make sure the ads hit at the appropriate times? It might be worth it for larger creators to swap over.
You can also go the route of looking like a pool of users, then it's just a game of cat and mouse and one providers don't really have time to play
History has shown you can stop this well enough. Try accessing e.g. instagram; bibliogram attempted this, the project is now discontinued.
> If you perfectly mimic the GoogleBot and use google IP ranges by hosting on google cloud
In any case, my plan is to deal with it if it becomes a problem.
Edit: yea after looking at this, it definitely appears to violate the TOS. Also this seems like a huge red flag just begging reddit to shut it down: "Unlike with the Reddit API you do not need to authenticate using OAuth."
Further, old.reddit.com doesn't gate any content (even NSFW stuff) behind a sign-in page (at least for now)
Where on earth would you get that impression?
From the reddit ToS: "To use certain features of our Services, you may be required to create a Reddit account (an “Account”) and provide us with a username, password, and certain other information about yourself as set forth in the Privacy Policy."
NB: "Certain features", that clearly means that features are not gated by sign in, and that the ToS also applies to them.
Also from the Tos: " Except and solely to the extent such a restriction is impermissible under applicable law, you may not, without our written agreement:
license, sell, transfer, assign, distribute, host, or otherwise commercially exploit the Services or Content; modify, prepare derivative works of, disassemble, decompile, or reverse engineer any part of the Services or Content; or access the Services or Content in order to build a similar or competitive website, product, or service, except as permitted under the Reddit API Terms of Use."
And later: "Access, search, or collect data from the Services by any means (automated or otherwise) except as permitted in these Terms or in a separate agreement with Reddit (we conditionally grant permission to crawl the Services in accordance with the parameters set forth in our robots.txt file, but scraping the Services without Reddit’s prior consent is prohibited)"
So, while it almost certainly would not apply to our personal computers, I think it would probably apply to most commercial companies (provided they were large enough to constitute commercial commerce and provided data was used in a manner explicitly enumerated by the statute)
Edit: it's to its
This is tangential no? Third party reddit apps _already_ republish end user content.
I'm not saying third party apps powered by scraping are not illegal, I'm saying they're _no more_ illegal than those powered by the official reddit API.
> The companies that reddit (officially) wants to target, e.g. OpenAI et al., have no problem scraping.
Indeed, on their case they just need the datasets anyways
I do think it's got a lot, lot of sharp edges that need smoothing out before it's really viable. But speed isn't a problem, and it's decentralised.
Mastodon is decentralized, because there is no central server. However, it is not distributed.
Hubs can have decent uptime and thick internet connections. Hubs are relatively few so they replicate state quickly between themselves, and then more locally between spokes when they come online.
Most natural structures are more tree-form, like that.
In context of protocols like Mastodon, if the end-user devices aren't primary holders of data, then I don't call it distributed. It's just decentralized. I guess that way, "distributed" necessarily implies peer-to-peer.
The crux comes down to the consensus model for state
Everyone who played any P2P multiplayer shooter in their life knows P2P sucks... on the other hand, dedicated server based shooters and MMOs (generally) work great. You don't have to be an IT professional to know this. Similar concepts (speed of light and multiplicative effects of multiple slow connections) apply here.
P2P is only great for uses where latency doesn't matter. See: Torrents.
So for your example: One on ones over p2p are optimal, but p2p matches with more than 8 players are suboptimal (I pulled the number out of my ass it depends on the protocol and usecase)
Not generally, but many servers. Which results in me avoiding clicking on any links to mastodon because I expect it to be extremely slow.
It takes 10 seconds to load a tweet, and I have to click through the "view in app" nag, to boot.
Using Mastodon servers have been a breath of fresh air in comparison.
Mastodon page: requires unblocking JavaScript, takes one second just to load the basic layout (without post contents), then takes four to ELEVEN seconds to fully load
https://fosstodon.org/@rauschma
I expected better.
Took between 1.2 and 1.8s for a full fresh load, for me.
Lots of repos, everything is open-source
We had a P2P and open source Reddit alternative 40 years ago, called Usenet. But people fell for a centralized one run by a company once it came along, like they always do. May Reddit's fate end up looking something like Digg's and AOL's.
But a minor quibble: USENET wasn't exactly peer-to-peer. I mostly got to USENET through my university or ISP's NNTP servers. If I wanted to start a new group and get you involved, I would've had to set up my own NNTP server for you to connect to, using either your NNTP client or your own NNTP server.
A lot of people seem to believe that centralization is the problem, and ignore Wikipedia, which has been true to its mission far longer than Reddit or Twitter have, and it never needed a decentralized model. The simple explanation is that it's the profit motive that is the problem. That can be fixed with the proper organizational structure.
I wish the Mozilla Foundation would stand up Reddit and Twitter clones. I bet they would be more usable on Day 1 than the Fediverse solutions.
Well-managed centralization does allow for governance, where things like moderation and technology decisions are difficult to leave up users. It's sad what happened to the public square of Twitter. There really isn't a viable alternative at the moment which has the reputation of a neutral ground where people can exchange discourse with relatively even chance of being heard, for all its problems.
Wikipedia's original vision was 1) to free the content and the communities for forking as well as copying, and 2) to give everyone in the world a free encyclopedia in his or her own language. I have a big rant about that but the quick version is that Wikipedia.org (the web site, not the project) should have been designed like Github, so you would use it for editing and creating forks, but (mostly) not for reading articles. Instead, whenever you buy a new consumer PC, through deals set up by the WMF, Wikipedia in every language should already be on its hard drive (you can delete it if you want the space instead). You'd read articles locally instead of through a privacy invading web site and internet surveillance apparatus, you'd get updates through a nightly equivalent of git pull that got all the changes with no indication of what you had been reading, you could downstream your copy to your friends or followers, you could make a fork and downstream that if you don't like Wikipedia's policies, etc.
I agree with you that Wikipedia doesn't have the exact same problems as Reddit. It is arguably not as bad as the current version of Reddit (we're here right now because Reddit is about to get much worse), but it has its own problems, spreads propaganda that is taken more seriously than Reddit posts are, etc.
Come to think of it, we also had a perfectly good distributed, peer to peer, FOSS source control system called Git, but it got mostly supplanted by a centralized "hub" now run by Microsoft, the original closed source empire. It never stops.
And to get there first: please don't bother saying that forking Wikipedia is permitted under its licensing, since that is near irrelevant in practice. It could and should be made simple and easy. Wikipedia's main value these days seems to be AI training rather than informing humans, which was supposed to be the point of an encyclopedia.
Coming back to your first point... how often would you maintain a fork of your own encyclopedia? Even with Git, you would fork mostly in order to submit a PR to the original repo. I'm just trying to understand the use-case here since I cannot imagine a situation where I would fork Wikipedia and make my edits to be shared with my friends only and my friends would do the same.
Did you mean something like fandom.com where communities (not individuals) can start their own wikis? I'd also love that.
As a Wikipedia example, look up Joy Milne (the lady who can smell Parkinsn's disease) with a web search, then look her up on Wikipedia. Wikipedia says nothing or almost nothing. That's not an oversight, that's Wikipedia bureaucracy in action. You might be able to get an edit through by battling the bureaucracy for long enough, but if you have an interest in this illness and wanted to gather and share info about it, it would be simpler to just fork the PD article. Et cetera.
Does the decentralized nature mean that illegal content can't be easily be removed?
i think thats the obvious intention. Just scroll down the page to see what the user base is.
This would have easily been avoided by… choosing a name that doesn’t have such a storied past.
Regardless, It’s always good to see new concepts or alternatives sites, just like old days with forums.
the protocol is a light protocol over IPFS/IPNS/Libp2p Gossipsub
IPFS files (JSON/base64): https://github.com/plebbit/plebbit-js#schema Gossipsub protocol messages (CBOR): https://github.com/plebbit/plebbit-js#pubsub-message-types
There's also a FAQ https://github.com/plebbit/whitepaper/blob/master/FAQ.md
The "subplebbit" owner acts like the operator of a Mastodon or Matrix instance except as mentioned above, those were designed to aggregate data across instances.
The fundamental problem with these decentralised platforms is that they want to get rid of the human element but if you just let anyone get away with anything and everyone rehosts everyone else's data unconditionally, then you get something like zeronet which has its own Reddit clone. The problem with zeronet is that people advertise their illegal porn on your supposedly "family friendly" Reddit clone and your users involuntarily download it and make themselves criminally liable. Whether something should be legal or not is a philosophical question, someone has to make a decision and then enforce it. Any platform where users are allowed to upload arbitrary information requires moderation. There is no way around this. The only benefit of a decentralized app is that you can have smaller self hosted communities.
I spent a lot of time evaluating the technical viability of decentralised apps and honestly you should avoid it even if your use case requires it.
- Mastodon and lemmy instances can delete your account/community data (they own your data), on plebbit you own your data, it's on your device and you seed it P2P, community owners cant delete it
- Mastodon and lemmy instances can block you from accessing and interacting with other instances through their client, which forces you to use multiple clients to interact with all instances you want, on plebbit, the client accesses all content directly and p2p, community owners can't block you from accessing any content. You only have to use a single client.
- Mastodon and lemmy instances require a domain name, a static, public HTTP endpoint, an SSL certificate, DDOS protection to run. All which are complicated to set up, cost money, sometimes require KYC, sometimes require the command line and linux. Your server, domain, ssl and ddos protection provider are intermediaries that delete/block your account.
Whereas plebbit is a GUI executable like a bittorrent client, you download it, open it, that's it, you're done. No payments, no kyc, no command line, no config, no intermediaries that can shut down your account.
Other interfaces that are planned are stackexchange, discourse and vbulletin.
The protocol is completely open and the code open source. Anyone can make any number of interoperable interfaces.
> The goal is to recreate Reddit exactly, the exact same UX, except without servers, DNS, global admins, global moderators, lawyers, regulations, corporate greed, etc.
[1]: https://github.com/plebbit/whitepaper/blob/master/FAQ.md#q-w...
Reddit content is already perfectly stratified into subreddits, so I think anything that's primarily a decentralised Reddit alternative should use the same principles. Instances should host subreddits with floating user identities, and content from one instance should always stay on that instance. This solves things like moderation (it basically stays the same as on Reddit, but you get to choose your instance) and legal content attribution. In a way, it's a continuation of the "independent web forum" model that Reddit supplanted.
I think it would be better to send only small "something's new" push notifications, having the client reach out for any data or even metadata. Also, having user identities be cryptographic keys could solve the defederation problem, but I understand that the user experience would be very atypical. Restricting moderation drama to one particular instance is to me a very important feature of decentralised systems, and the possibility of being excluded from the larger community over such drama is a serious fault, even if that's probably less likely with Lemmy.
The theme itself is a good Reddit replica, but I don't think IPFS has the necessary performance to run apps like these. Maybe it can live become a Lemmy frontend for the FOSS/fediverse crowd?
Yes I know this is "my fault" for being a weirdo who doesn't blindly let his browser hit any server a website pleases. But I use this to judge the quality of a website, the care and professionalism that was put into it. This plebbit site is awful by this metric even compared to new reddit; with new reddit merely whitelisting *.reddit.com is sufficient to view a list of posts, and whitelisting 3 more domains (all clearly reddit-affiliated: redd.it, redditmedia.com and redditstatic.com) is enough to make the site behave as it should.
I kinda agree with him. There is no captcha that's going to keep out determined spammers. Even reddit is filled with bots.
You either try to eliminate bots through paid accounts, real ID verification... Or you filter based on quality of content (ie. bots are allowed to post, as long as what they say and do is indistinguishable from a human - like reddit does).
Whats the anti-spam plan?
the community owner must configure some challenge exchange between his node and the users publishing to his node. the challenge is completely arbitrary, it can be a whitelist, a password, a super complex interactive game, sending a picture of your passport, account age, minimum karma, a combination of all these things, etc. Since it is arbitrary, it will evolve with time. Any method (already invented or invented in the future) that centralized platforms use to filter AI spam can also be used, but over P2P.
User moderation in focused spaces!
Its the same reason Discord/Telegram kind of works, the same reason small niche subreddits (and Hacker News) are still OK, the same reason dedicated game servers are so much nicer than public matchmaking in video games... They have a hierarchy of users who are motivated and empowered to police their own reasonably sized niche space.
Twitter can't do this, but maybe Twitter should not exist as it does now.
Reddit the company wouldn't really like this either, as they would have rather have mainstream mega-subreddits over collectively smaller niche subs.
Remember the bots can make their own spaces too... And then lure real users in with a bunch of chatgpt-like and/or stolen content...
Paid accounts adds a significant overhead to the spamming.
Spammers just don't spam for the sake of it, they want to sell people something, legal or not.
It's one thing to buy millions of spam accounts for $100 on the dark web. It's another to have to spend $5 bucks a month per account on millions of accounts...
Depending on your definition of "free", then it becomes a government agency's problem, not the platforms.
For a business, a shill account can be one of the cheapest marketing options.
Good, that would eliminate 99.99% of the spammers.
Good enough for any social media.
The potential bank fraud and wire fraud federal charges are just the cherry on top of the ridiculous overhead of your imaginary spam operation.
If you start getting a little spam, you explicitly revoke trust in the spammers. If you get a lot of spam, revoke trust in whoever you trusted that trusted the spammers, etc. A little bit of social graph hygiene ought to go a long way against spam, so long as we're trusting people and not platforms. We're practiced at one of those and not the other.
Users can set a "trust" threshold and all content they see is filtered through that lens. It's like an automatic twitter block based on an algorithm.
Also people would ask the people they refer to the platform to please not do bad things so they can preserve their "trust" rating.
Largely we excluded these people from conversation with social cues and sold them back their prejudices via red-top newspapers, which mostly kept their brain farts and prejudices suppressed.
For better or worse, you can't do that with the internet. Facebook and twitter seem hell bent on not only encouraging their appalling behaviour but amplifying it.
The only way to stay away from them is to stay ahead of them and stay away from any medium these idiots think is "their place".
edit: what I mean by that is that facebook, twitter and reddit and 4chan should be held directly responsible for the violence they foster. "we're just a communication mechanism" doesn't cut it anymore. Their promotion algorithms are very different from being a telephone company
I'm a human but it's not clear what the "challenge" even is that needs to be completed.
And after you get it wrong, it doesn't kick you out, it just flashes a toast message like "User failed captcha".
Twilight zone material.
It’s not the technology that is hard to replace, it’s the community and it's the moderators. Reddit took what, 10+ years to sort of clean up?
You want a discussion site, but you don’t want the top posts to be from horrible alt-right incels… it’s a hard problem to solve.
Also, the load time on that site was abysmal.
Content is terrible...
Speed is atrocious...
Alas, I don't think it's yet ready for primetime. Is it fixable in a week?
We are 4 devs working full time on it. In my estimate it should be pretty smooth in around 1 year. We should also have an old.reddit interface as well for people who hate the new interface.
It's a interesting idea but I think 4chan, as the internet watering hole, has the core audience captured already and has done so for the last 15ish years. I am curious as to what the marginal value is here.
But the user experience is terrible.
I think to have any success, you need to let the p2p features take a backseat, and build a lot of centralised content moderation tooling. You can plug the p2p bit into the centralized bit with a config URL for "List of allowed moderators" and have it default to a bunch of trusted people, who effectively have whitelist/blacklist/supervote abilities.
Then you need to populate it with a bunch of good content to start off with.
That's part of the "why of new.reddit"... which makes sense for some aspects of its infrastructure. Yes, I prefer old.reddit because it renders everything at once and the layout is simpler. But if you're waiting on the network more than the CPU, new.reddit style is probably better.
The UI being slow is because loading stuff from cloudflare-ipfs.com is slow. Trying to get it all at once for a 30 card rendering would be even slower if you're waiting for ipfs.
It just can't be that hard to hack together a nested newsgroup-like system in this day and age.
Note also that many of the apps were designed to not have advertising or monthly subscription costs.
https://www.reddit.com/r/apolloapp/comments/13ws4w3/had_a_ca...
> Apollo made 7 billion requests last month
What does it cost to stand up that server? Establish the core group needed to moderate the site to not have it turn into eternal spam or content that they'd rather not serve site? Pay the lawyers needed to make sure they've got NSFW and copyright take down requests handled?
It's not impossible... but the "bunch of people start up a new site" misses a lot of the technical and social aspects of barn raising ( http://meatballwiki.org/wiki/BarnRaising ).
The advantage of reddit for the backend is:
> BarnRaising is part of the difference between SlashDot and wiki. With SlashDot the barn is already raised - the OpeningStatement already written - before you start, and everyone just sits around bitchin' about it.
I still feel like the various protests going on indicate the space is absolutely ripe for an alternative and being a bad alternative might just be good enough.
Yes, you can stand up a server that can respond to 2300r/s and handle 200GB/day (6 TB/month which is another few hundred dollars)... and the storage for it.
The problem with standing this up is the "how do you pay for it in a way that isn't out of the goodness of your own heart" - because part of the reason people are using these is to avoid paying Reddit or see advertisements.
You will also need someone to answer take down requests for whatever legal reason exists. If you do not handle the takedown and leave it to be unmoderated, there are a number of other applications that serve as an example for the community that forms there.
The difficult part is not paying for compute, egress, or storage. The difficult part is figuring out how to get the customers to pay for it and to pay for the staff needed to maintain server and legal availability.
Writing an app that acts as a conduit for data is quite a different than than being responsible for hosting a service that provides that data.
> FetchEvent.respondWith received an error: no-response: no-response :: [{"url":"https://plebbit-test.netlify.app/"}]
There are certainly overlaps in the two concepts and certain parts of them can be compatible to a small extent: a bunch of people have compared Bluesky Social's AT protocol to SSB despite the former being federated & the latter being p2p. But similarities aside they can't really be compatible at protocol level.
A capable person with different views could fork Lemmy while keeping compatibility, and that has happened many times with other Fediverse software.
At the time of writing, the 1st page load is 14 seconds and subsequent page loads are 7 seconds.
There's a GUI client so it's very easy, you just double click and leave it open.
It's also possible for centralized services to run communities for people, so it's technically possible to run a community without having to have any device run 24/7 yourself.
the idea is that if there's 100 providers eventually, and only 1 needs to not censor you to get your content, then it's unlikely you will be censored
we plan on adding IPFS webrtc and webtransport at some point, to reduce the need and load on the ipfs gateways, and whatever new that comes out that can help the browser/mobile clients. For example https://iroh.computer/ seems promising
the desktop client is fully P2P, it bundles an IPFS node, and the only requests it makes are through the IPFS node. (technically it makes one request to github for default communities, but we will remove that at some point). Also if your community or username uses a human readable name (optional), then you need to resolve that name somehow, we can technically support any name system anyone makes, and that might or might not be P2P.
Can’t call it an alternative if it browses reddits content.
Monthly capacity limit exceeded. Visit alchemyapi to upgrade your scaling policy for continued service.
Nice work, and good luck.
The quality of Reddit is created by the users. If they used a P2P platform the content would be of high quality.
P2P social media is a fantastic idea.
A big internet forum like HN gets tons of crap comments. Most eventually get moderated, but not all; and even the ones that get moderated take time before we see them.