Three protocols and a future of the decentralized internet | Hacker News Reader

Three protocols and a future of the decentralized internet

blog.datproject.org

137 pointsdredmorbius36 comments

The issue with federated services is that they operate from the assumption that everyone has an anarchist sysadmin in their friend group who is going to help everyone run the federated infrastructure.

I’ll use SSB as an example. If you install the software and make an identity, you’ll soon find that you have to sign up with something called a “pub server” to actually get any of your friend’s updates.

Where do you find such a pub server? There’s a wiki: https://github.com/ssbc/ssb-server/wiki/Pub-Servers. The author of the wiki has found that it is necessary to admonish people not to just pick the first pub server on the list, since there isn’t really any criteria by which to choose them. If there was a sudden influx of more than a few hundred new users, it would cause utter chaos on this list. Someone would need to spend a lot of money on hosting, with no way to earn revenue.

The hosting cost problem becomes acceptable if the pub server is fast enough and hosting is cheap enough. Look at IRC.

The cost would be zero if residential ISPs issued static ipv6 and allowed servers.

Yeah, the more I look at P2P applications the more I realize our infrastructure is not set up to support them without some serious work on the application side.

The hosting cost would be negligible.

The policing costs less so. Hygiene matters.

I don’t think static IPs are as relevant today.

What I do for my home PC (that’s for ipv4, but it doesn’t matter) is I map my current external IP in Google Cloud DNS every 15 minutes with a TTL of 0.

Of course I only make the change if needed, so I am easily below the paid threshold, too.

In a properly p2p system the idea of certain servers being offline for short amounts of time is considered a norm.

So should be OK either way.

The “allowing servers” part is a non issue.

By the time it becomes an issue, you’d already have more issues like the need of more bandwidth which would make you switch providers anyway.

>Where do you find such a pub server?

If it's anything like email, you would use the server your employer or organization you belong to created an account on for you to use.

In SSB, pubs seem like an essential (and flawed) part of the network, and this may be true at the present time, but not forever. I'm working on a few alternatives to pub servers, one of them is a server that has no storage of data, it just acts as a meeting point for clients, and sets up tunnels between them. The other is a 1-feed pub, where the idea is to setup a mirror for your data in case you are a feed that has many thousands of followers. In both cases I want to make it easy (as in Heroku one click installer button) to start a server. The idea is that the easier it is to start a server, the more servers there will be in quantity. Off from the internet, SSB already has good mechanisms for sharing data (Wi-Fi and Bluetooth), which other protocols don't seem to have. Its hard to grok how good this is unless you're living in poor internet environments. SSB on the internet is not as well designed as SSB off the internet.

Despite working with another protocol in the decentralized space, I have to say Andrew Staltz, who I assume is the person who made this comment based on the username, is doing such a great job with manyverse: https://www.manyver.se/ and I recommend downloading it to try it out. It is one of the first breakout apps (in my opinion) in the space that feels user friendly and really makes it seem like theres a viable decentralized alternative to facebook. The bluetooth/wifi option helps alot with making it user friendly when it comes to pub servers, etc.

I actually found out about manyverse last year when it was trending front page on hacker news and I'm glad I looked into it.

He is being humble by not bringing up is app here despite it being a beautiful one that feels seemless, and being extremely responsive to his users (you can even submit issues to gitlab via by clicking a button in app) but listening to his podcasts on solarpunk, and in general, he seems like a humble guy so I'm not surprised. I am very interested to see how the features manyverse plans to roll out help push the user friendly aspects needed is the SB protocol space.

That should be automated. Look at the way Bitcoin nodes find peers. (Which has nothing to do with the blockchain.) There are a few well known nodes, but they mostly redirect new nodes to other nodes.

> The issue with federated services is that they operate from the assumption that everyone has an anarchist sysadmin in their friend group who is going to help everyone run the federated infrastructure.

I'm not sure this is a real problem. Email is federated, scales to approximately everyone, and there is no shortage of email providers that all interoperate with each other.

How to fund it when no one you know wants to operate it personally is also no great mystery. Either the server provider puts ads on it, or the users pay for the service. And the individual users can decide which of the various options they prefer but can still interact with everybody who makes a different choice.

Funding community networks is one of the few use-cases I've seen that actually seem to make sense for cryptocurrency.

The Althea wireless mesh is doing this -- you "load" your router with a certain amount of credit, and it tries to get you the lowest-cost transport. People running nodes have incentive to optimize their antenna placement and signal, so they can carry more traffic and get paid for it.

I don't know if this directly applies to SSB since there's really nothing to "run", other than pubs which we've already established are only a stopgap solution anyway. Simply participating in the network (leaving your client running) is what provides the real "infrastructure", such as it is.

Pub servers are very much considered a temporary workaround in the Scuttleverse, and there is a lot of ongoing work to make onboarding both more directly user-friendly, and also less dependent on "having an anarchist sysadmin in your friend group".

Right now, without pub servers you can:

1. find friends in your immediate vicinity over WiFi/Bluetooth/etc.-type nearby protocols.

2. if you have a friend (see #1), you can connect to friends-of-friends

(#1 may seem extremely limiting at first, but it's actually only limiting for a technology which is not popular in your local area—network effect problems—so "in theory" it's not really so bad when you think about it. I've probably added a good portion of my FB friends in their presence)

However, one of the main alternatives being actively worked on is DHT invites. This is basically the "torrent magnet link" equivalent for SSB. See various links to work on this below:

[0] https://gitlab.com/staltz/ssb-dht-invite

[1] https://gitlab.com/staltz/manyverse/issues/186

[2] https://github.com/ssbc/patchwork/pull/862

[3] %r6dTZ4WG7kuLFfGlUR1DAhyat7YEuX682mGI9UJXlc4=.sha256

The biggest challenge I've seen with using decentralized systems today is with on-boarding the average user into them. As @woah mentions, the process isn't straightforward and even remotely close to signing for a facebook or ig account. I think there are a few problems that'll need to be solved before all this is actually usable.

(There's definitely a parallel here with the AOL and CompuServes of the world vs the open web, but not quite)

- It almost feels like that for a variety of decentralized protocols to take root, someone will have to build a browser-equivalent (or perhaps an extension to an existing one) that allows the normal user to partake of these new systems in one place. That means, a chrome or firefox style system that supports IPFS, but also offers tools for ActivityPub and SSB and whatnot - in the same way netscape navigator used to be bundled with a news reader, a web browser and an email client.

- The next challenge is to get the average user to use this on the mobile and then the desktop. That's a problem best solved by modeling this as an app - and there needs to be a reason for the average user to use this. The words "killer app" come to mind, but honestly, even if there was something slightly more useful that current services provide that built on top of this - it would be helpful. I'd download this new "chrome" on my phone and use it. Is there something out there that people think can be that?

- Lastly, we'll need to democratize information and app creation for this browser.

Blockstack is not based on scuttlebutt protocol, but when it comes to addressing id creation, user friendly onboarding probably needing a browser, etc

"someone will have to build a browser-equivalent"

we do have a browser that has a somewhat user friendly onboarding process: https://browser.blockstack.org/

If it is your first time, you will be asked to create an identity on Blockstack: https://docs.blockstack.org/core/naming/introduction.html

Each instance of the "browser" whether on desktop or phone, requires a unique password, associated with the id. The id is stored on the Blockchain, the data is stored in your own gaia hub which you can host wherever you like: https://docs.blockstack.org/storage/overview.html

"That's a problem best solved by modeling this as an app - and there needs to be a reason for the average user to use this"

The next browser release will have a landing page that features apps dynamically populated from https://app.co/ and will have a more user friendly onboarding process, one that explains in more detail that each instance of the browser (we have a desktop version, each login in a new browser, etc) requires its' own password.

I think my favorite app using our protocol is https://debutapp.social/ because it is the "facebook" or twitter like equivalent people can associate with to see there is a relevant use to this, and in general user end interaction with the platform can be seemless with integration of decentralised protocols when done correctly.

The founder/creator of this app is a self taught software developer who does this in his spare time outside of a full time job so if you have feedback make it constructive and feed it back to him on the help page.

Same as other protocols, we are working to make our nodes easily hostable. If you know docker at this point or how to launch a vm in digital ocean you are probably good, but we are working to automated images and one click deploys. You will see more rollouts with much less instructions and much more "one click deploy" based in the near future.

We are actually prioritising our gaia hubs being easily hosted with one click deploys by the average user over the nodes, because in our model gaia is where the user's data is stored, and the decentralized aspects of the protocol are related to identity, and authing that identity with a users data, that apps can read/write to, but the user ultimately has the ability to store that data where they want, revoke access to apps, whitelist apps, migrate the data, etc.

You can read more about blockstack, gaia (our user storage) and the rest of the platform here: https://docs.blockstack.org/

Given your very developed opinion on what the decentralised internet needs, and how it seems very aligned with what we are developing, would love to hear your feedback/initial impressions of our platform and app ecosystem.

Thanks for the pointers -- I'm pretty familiar with Blockstack and the Gaia ecosystem. I started playing around with it a couple of years ago, and registered [1] my ID in 2017 maybe? or 2018? I forget. Of course, it was much more bare bones then than now. But the owner address link doesn't work for me today [2].

Happy to give feedback directly on DebutApp, but my initial impressions of it when I tried were - it's a good proof of concept, but its a very early incarnation and so definitely rough on the edges on both product and engineering. It's commendable that this is a side project, since it shows promise.

One of my main questions with a lot of blockstack apps is exactly that - while they prove that something can be done, it's unclear to me how the next person who is going to google for "new blog" and try to pick one of squarespace, medium, or blogger - will use debutapp.

I'm curious about the host-able node comment. Who is this one click deploy targeted at? and what is the incentive for folks to do it today?

Also, any insights on why there's a general sense of lag on the ajax calls that these apps make?

[1] https://explorer.blockstack.org/name/viksit.id

[2] https://explorer.blockstack.org/address/1LMxzW9LbL6HZEo3qeLb...

Hey thank you for the feedback.

usernames expire after two years, so that could be it depending on when in 2017 you registered your username.

"One of my main questions with a lot of blockstack apps is exactly that - while they prove that something can be done, it's unclear to me how the next person who is going to google for "new blog" and try to pick one of squarespace, medium, or blogger - will use debutapp."

I think apps built with Blockstack like graphite and others: https://www.graphitedocs.com/ are gaining alot of popularity on the "normal" internet and getting recognition in places like producthunt, hackernoon etc: https://hackernoon.com/is-dweb-taking-off-a-glimpse-into-blo... , https://www.producthunt.com/posts/graphite etc etc

For now we are trying to point people to https://app.co/ to show the entire dapp ecosystem, not just blockstack based applications, but you can filter by protocol.

The next browser release will have blockstack apps from app.co dynamically loaded into the browser landing page as we know its a bit outdated based on the apps available now.

but yes there is alot more we can do, and alot of general awareness people can have on the internet. The idea is that applications like

blockusign: https://blockusign.co/signup.html

afari: https://www.afari.io/

graphite: https://www.graphitedocs.com/

sigle: https://www.sigle.io/

stealthy: https://www.stealthy.im/

travelstack: https://www.travelstack.club/

xor: https://xordrive.io/

diffuse: https://diffuse.sh/

dpages: https://dpage.io/editor/

(secret fave: https://blkbreakout.herokuapp.com/)

blockvault: https://blockvault.site/

and others listed on app.co, are also searchable on the mainstream internet, and advertised not as exclusively apart of the "new internet" but advertised based on the user value they provide that many centralised apps do as well.

"I'm curious about the host-able node comment. Who is this one click deploy targeted at? and what is the incentive for folks to do it today?" <-- in reponse to the top ranked comment on this blog about easily hostable infrastructure aka blockstack-core nodes and gaia hubs as well in our case. gaia hubs are where users store personal data (as opposed to ipfs thought gaia hubs can be technically support an ipfs driver/we have a req for this but have not prioritized it yet).

Making gaia hubs easy for users to host supports the idea that if they want to its not hosted by someone else/ a big cloud provider/another company etc, despite the api supporting they decide who has write access to their data/can delete if they want to, etc. It also allows them to host on their own server, or another cloud if they want to.

The incentive is mostly there in that already some applications are hosting gaia hubs for users and charging them for it, so there is an opportunity to make money.

In the future, we see companies dedicated to hosting gaia hubs for users and charging for it, not just applications, and you can see there are many parallel companies doing the same with IPFS type protocols like filecoin, golem computing for example (now lead by the previous founder of QubesOs, which says alot that she would leave her own operating system after a decade to focus on something like this). There is definitely money to be made in this space, I'm just a big believer in despite all of this, still making it easy for users to host their own on their own machine, a rasberry pi or something like Digital Ocean's one click deploy marketplace if they so choose to be wholly consistent with the idea that users can own their own data, manage the gaia hub themselves but not have to be an "anarchist sysadmin" skill level to do so. The incentive there right now is that Digital Ocean is also paying for 5months of one click deploy marketplace droplet hosting ($5/mo). That's something we are working on, but yes, there is definitely an economic incentive here..

The blockstack explorer actually went through some internal infrastuctural changes just this week. We are aware this is an issue and working on it. Will get back to you when you can expect faster response times for this. I would say end of not this coming week but next week, but will get back to your with more detail.

Easy user mobility should be a goal of decentralized protocols and platforms. That would help users leave one provider and join another if/when the need arises. If I join Server A today as User A, I should be able to carry my identity as User A to another server or provider in the future without losing the ability to still remain in touch with everyone I was in touch with or having to tell them that my address has changed. This would be similar to email where one owns a domain but can move it around to any provider (or self-host) for storing and managing the content.

I have generally observed that some initial servers for a decentralized platform get more users over time and remain the largest, while later and/or smaller ones may not get attention, and could possibly even close down because of low usage.

Mastodon has this. SSB doesn't, because your identity is just your message chain and key pair on your computer.

Mastodon doesn’t seem to have a way to move a user account and still have others message or tag one in the same way as before. When you switch from, say mastodon.social to mastodon.xyz, it’s almost like switching from Yahoo Mail to Gmail. There seems to be a way to export one’s following list and move that to the new instance, but this does require informing others of the address change on one’s account.

See https://medium.com/@jimpjorps/a-non-computer-persons-guide-t...

Have I misunderstood how this works?

If you've moved, your followers don't automatically follow you to your new home. But IMO that is fine. If I was following someone on cat.photo.space I might not want to follow them if they move to callgirls.xxx .

IIRC, you get a warning if you try to interact with someone who has moved.

Is it possible to operate a lightweight forwarding service, so my address is portable across servers?

That wouldn't be much different from operating a server yourself, so while it's not impossible, it's more a question of what benefits it provides.

An important feature is maintaining your social network after a fallout with your admin.

I don't want no one to tell me what to do and to prevent that I've got to prevent people from having leverage with me.

Hubzilla[1] has a a feature called "Nomadic Identity" which seems to be a solution to your use case of carrying your identity to another server.

See the "Nomadic Identity and Cloning" paragraph on the Hubzilla homepage [2]. See also "Nomadic identity, brought to you by Hubzilla," by Andrew Manning, on Medium [3].

[1] [2] https://hubzilla.org/

[3] https://medium.com/@tamanning/nomadic-identity-brought-to-yo...

Edit: Newline formatting.

This is one of the primary benefits of using Blockstack. The Blockstack naming system is used to associate an immutable user identity with the routing path to their storage, for reads and writes. The data is stored in your own gaia hub which you control which apps can write to, you can revoke access, delete your data, or take your data with you. We are working to make it more user friendly but you can read about it here: https://docs.blockstack.org/storage/overview.html

The identity is stored on the strongest blockchain and can be linked to your gaia hub.

"initial servers for a decentralized platform get more users over time and remain the largest, while later and/or smaller ones may not get attention, and could possibly even close down because of low usage" while we host our own nodes, in general the nodes in our network are selected based on a random walk graph and bias away from having a few nodes.

In what way do Dat, IPFS, SSB, Mastodon, ActivityPub, Matrix etc. counter the push for some centralizing effect, in that eventually everyone is still dependent on the same server?

It's still possible to host your own website & server, but not as easy as setting up a Facebook profile. Same goes for e.g. Mastodon: running your own server isn't as easy as creating your just an account.

How do projects like Mastodon not end up with a big server (or a few big ones) that everybody uses?

Edit: Not all of the protocols I mentioned use servers, some are P2P which don't have that issue, I think.

If you use Mastodon, hosting your own server can be hard. If you use Facebook, hosting your own server is literally impossible.

A few big servers is preferable to 1 big server (service)

Another P2P protocol that wasn't mentioned, that is already powering 2M people across Internet Archive, d.tube (decentralized YouTube), notabug.io (P2P reddit) and others is https://github.com/amark/gun (mine).

I find it interesting how people mention file stores (IPFS/DAT) a lot more than databases (SSB/GUN). Distributed file stores have been around for a long time, and are not inherently useful for creating Google/Facebook-etc. killers, where-as a P2P database is very needed.

Is it perhaps because everyone already knows about BitTorrent, and there is BitTorrent nostalgia?

I would vote for a forth protocol from matrix.org for realtime conversations, instead of the other store-forward other protocols.

The biggest problem with decentralized protocol (in practice) is still the speed. Centralized protocols are simply faster. I am hoping someone will discover a way to make decentralized protocols fast in the future. I am personally rooting for IPFS.

The speed is only an issue if you are storing all of your data in a decentralised network, but do you need to? What you want to persist is your identity, but you perhaps you don't need to replicate your data on other people's servers, for an indeterminate period of time. Furthermore, IPFS relies on DHT protocols which have shown to have scaliability issues.

At Blockstack, the name persists via the blockchain, which can be slow, but is only an initial barrier on identity creation. After that, you own your own data via gaia hub, which can be stored wherever you like, for example behind an LB with fast i/o on Digital Ocean. You just run docker-compose up on our master on an image of choice on digital ocean for example: https://docs.blockstack.org/storage/overview.html

We are looking to automate deployments for user friendly rollouts but I strongly recommend checking out gaia. Gaia is not dependent on the blockchain, and can still be persisted or duplicated via networks like IPFS is someone so chose to. We have an IPFS driver request in and are looking to get to it with some eng cycles.

I knew about Beaker and Dat (played with Beaker browser/distribute web sites) but not the other two protocols. Glad to see a discussion here.

The world of decentralized web (aka Web3) is witnessing both:

1) Transition of trust-translucent internet native centralized systems (social networks, file hosting services, instant messaging services) to decentralized systems using protocols like Dat, SSB, ActivityPub, IPFS, etc.

2) Onboarding of trust-opaque internet non-native centralized systems (legal tender, corporate law, legal citizenship) to digital decentralized systems using Bitcoin-born protocols like Ethereum, Polkadot, Cosmos, etc.

Reply on news.ycombinator.com