Comment by dvasdekis | Hacker News Reader

Comment by dvasdekis | Hacker News Reader

Parent Full thread

Thank you very much for the thoughtful reply.

RE the libraries, while it is true that I can't find anything made specifically for Usenet in Go, Usenet itself is just an extension of the Telnet protocol[0], and there are Telnet clients in Go[1] and Node[2]. It probably isn't simple, but I'm sure working with OrbitDB wasn't easy either!

RE the resilience of content on Usenet, the vast majority of binaries are heavily encrypted, and doesn't make sense to anyone without the key, despite being conveyed en-masse between the world's 10-ish full-scale Usenet backbones [3]. I'm proposing that the backend of a service that makes use of Usenet could be similar, with a single 'background server' on one trusted machine enough to continuously push the history to Usenet. A regular user client could then search for the latest version of this history and quickly refresh their side from Usenet, regardless of the status of IPFS at the time.

RE democratic access to technology, at least with Superhighway84 it was very expensive for me to actually run the software, as I have a small allocation of bandwidth from my ISP and not much I can do about that in my area, and I ultimately had to delete it due to ongoing transfers of 3GB/day running the IPFS node. Quiet itself notes a limit of 30-100 individuals with its application - I'm proposing that using the one remaining federated multicast technology with some modern encryption might help with issues around blasting data everywhere from a bandwidth-constrained environment. I know that definitely in Africa, there are ongoing issues with bandwidth and networks that we forget about in the West. Usenet, with extremely lean network overheads, could be part of the answer.

I do agree with your vision of a future of truly peer-to-peer technologies, but for those of us who are bandwidth-constrained or otherwise limited in our access to those technologies, having a technology-agnostic application that just 'does magic' to do whatever it needs to do with your content is what's going to make a majority of users happy.

[0] https://www.itprotoday.com/windows-78/how-can-i-use-telnet-a... [1] https://github.com/reiver/go-telnet [2] https://www.npmjs.com/package/telnet-client [3] https://svgshare.com/i/oti.svg

Thank you for the detailed description of your idea. Indeed, if you're willing to accept the shortcomings of a dedicated USENET infrastructure, then it is definitely something that could be done. In fact, I did consider NNTP for another project of mine (https://github.com/mrusme/neonmodem), which might eventually swallow up Superhighway84 altogether. If you're interested in actually giving it a try and implement a functional NNTP library for Go I'd be more than happy to make use of it! :-)

> Superhighway84 it was very expensive for me to actually run the software

I agree with you, in terms of efficiency IPFS is still miles away from where it should be. Hence my feedback on Quiet, as I do not perceive IPFS to radically improve within the next few months or even years. And as you correctly stated it looks like Quiet uses some workarounds to improve on the overall mediocre efficiency of IPFS, which however lead to shortcomings on other ends:

> Quiet itself notes a limit of 30-100 individuals with its application

However, this is not how P2P should be. I'd be truly curious to hear from someone at OpenSea, or Fleek, or any of the services that offer high volume IPFS hosting about their experience and gut feeling on its future. I personally gave up on hosting my website via IPFS myself -- which I did for a brief period of time -- mainly for these exact reasons.

> but for those of us who are bandwidth-constrained or otherwise limited in our access to those technologies

I believe that quite on the contrary, this might benefit these people the most. Imagine not having to do the roundtrip from your phone, to a server on the internet, back to your computer, just to have a synchronized state of your address book available.

Similarly, imagine writing with someone in your city -- let's say Melbourne, Australia -- without your messages first travelling to Utah, USA, and then back again. My gut feeling is that overall congestion on the internet could even be reduced, by allowing more applications to communicate directly within small meshes rather than travel all the way across the globe and back again. That is, as soon as there are more efficient ways to deal with the overhead that is currently breaking IPFS' neck.

A quick note on Quiet's capacity limits: the 30-100 number is very conservative and not an intrinsic limit of the approach we're taking.

I'm pretty confident that we can get Quiet to the point where the practical limit of participation in any Quiet channel is storage, relative to the amount of message history that a particular community wants to hang on to for a particular channel.

It wouldn't be crazy difficult to shard storage either. Once we do, a community could store a lot of data by marshaling many nodes with low or uncorrelated downtime. Paying for storage is also an option.

RE contributing to neonmodem, I was thinking about it! But baseline NNTP, as it sits today, is a fetid pit of spam, and I don't think it would add value. In fact, spam is, I believe, a far bigger problem with these networks than the technical distribution of messages over P2P. I took a sample of a random Usenet group today: [0] - but I really struggled to find one to post an image of here because even a lower-spam group that I found (e.g. alt.politics.uk) was full of profanities in the subject lines of the posts.

I think superhighway84 remains no/low-spam because of the technical hurdle of connecting. I don't think you've got any inbuilt spam protection? Plebbit [1], full of spam. The innovation of Reddit is arguably that people love the power-trip that comes with moderating a reddit group, and will do it for free - there's been no shortage of moderators to replace the protesters in the latest rebellion [2]. Hacker News, where I get to talk to smarter people than myself - very well/heavily moderated [3]. The SomethingAwful forums just resorted to charging everyone $10 for an account when they started having a spam problem, and that happily paid for the hosting costs and a life for the main admin for years.

To deal with the spam, you need some kind of filter where users can't just create thousands of accounts, especially in the age of LLMs. Logging in with a social account is the obvious one - Github/Facebook/Google have expensive processes in place to reduce the deluge of spam accounts, but some obviously creep through. Do you then run on an algorithmic chain of trust, promoting posts based on the quality/ratings of the individual's contributions elsewhere? If you do this, you're creating a system to be gamed. Running on invites only is another potential solution, but then it's difficult to start the gravy train of quality posts - who wants to apply effort to talk to nobody? Do you instead run a pyramid scheme - charge $10 upfront, but give a share of the site's ongoing revenue to those who get their posts upvoted, Twitch/Youtube/Instagram style? This to me seems like the one solution that could potentially displace Reddit, but I lack the personal belief/gusto to make it a reality.

Even if you managed to register and motivate a thousand decent posters, I don't have a clear view of how you keep topics on track within a group without a human moderator, but some research has been done in using LLMs to pre-rate posts based on the history of the group. But if the LLM is agendaless, you obviously get a groupthink echochamber. Give it an agenda, and you start dealing with bias - not every post of value is war and peace, and sometimes you just want to thumbs up a funny cat.

Please forgive the above musings if they're low value. I feel like I have no answers, only problems and questions, and I believe I'll be posting on Hackernews for tech, Instagram for comedy, and Facebook groups for special interests (e.g. car repair) for some time to come.

[0] https://raw.githubusercontent.com/dvasdekis/images/master/20... [1] https://news.ycombinator.com/item?id=36203610 [2] https://www.forbes.com/sites/barrycollins/2023/07/21/reddit-... [3] https://news.ycombinator.com/item?id=34920400

Reply on news.ycombinator.com