Pushing the Limits of IPFS and OrbitDB | Hacker News Reader

Pushing the Limits of IPFS and OrbitDB

mrh.io

104 pointsmarkhenderson33 comments

The blog post is accessible through IPFS here https://ipfs.io/ipfs/QmY2LufsW3v6AfxTTkp6SGqDGa5AeJhSZXH8RSs... in case the original link doesn't work.

Can anyone be bothered to explain to me how this works?

I know that IPFS is some sort of distributed content-addressable thingy. So I expected that ipfs.io would deliver some JS to my browser, which would get it to look-up some distributed hash table in order to find peers with the content (a la torrents).

However I'm not running JS and that page loads just fine. So how does it work?

ipfs.io is a gateway, fetching the content from the IPFS network and serving it to you over HTTP.

So sort of an intermediary between how the internet works these days, and how it should work in IPFS view.

Does the "deliverying JS to browser" thing that I described make sense at all?

There's work on running the IPFS protocol over WebRTC, so yes, it makes sense and is another way of enabling normal browsers to access it.

There is already a Javascript implementation [1], but it is not feasible to setup a full IPFS node in every browser tab that loads an IPFS resource. Instead you can use ipfs-companion [2], a browser extension that lets you easily load IPFS resources in your browser. While it can use the Javascript implementation, this is not recommended as it is currently pretty resource heavy [3].

[1] https://github.com/ipfs/js-ipfs

[2] https://github.com/ipfs-shipyard/ipfs-companion

[3] https://github.com/ipfs-shipyard/ipfs-companion/blob/master/...

The ipfs.io url is a public IPFS gateway, ie. bunch of IPFS nodes running on servers and "caching" content to be served through that domain. From that perspective, you're not accessing the content through the IPFS protocol but through http like any website. It's a public service operated by the IPFS community, but underneath it does all the "...get it to look-up some distributed hash table in order to find peers with the content (a la torrents)".

However, if you install the IPFS client (https://dist.ipfs.io/#go-ipfs), you can get the blog post peer-to-peer through IPFS by running `ipfs get QmY2LufsW3v6AfxTTkp6SGqDGa5AeJhSZXH8RSsdiao4Ds`. Note how the hash in the ipfs.io url is the hash of the content stored in IPFS.

Hope this clarifies it!

Thank you.

It's a bit sad that you require a whole new client. I think that will add enormous friciton to adoption, no?

I think it depends on what the use case for IPFS is. Many use it "server-side" to just transfer and store files, so a "native client" makes a lot of sense. The IPFS devs always recognized the importance of the browser though and as such js-ipfs (https://github.com/ipfs/js-ipfs) has always been a first-class implementation of IPFS. With js-ipfs, the IPFS protocol works in the browser and can be embedded to JavaScript applications in a way that doesn't "require a new client".

Highly recommend to read into IPFS and how it works to understand the various use cases and possibilities. A good starting point would be https://ipfs.io/ and https://github.com/ipfs/ipfs.

To summarize the relevant links and Github repos, they are:

https://tallylab.com/

https://github.com/orbitdb/orbit-db

https://github.com/orbitdb/ipfs-log

https://github.com/ipfs/js-ipfs

https://github.com/ipfs/go-ipfs

https://github.com/ipfs/ipfs

http://webcache.googleusercontent.com/search?q=cache:eh2eZNy...

Very interesting stuff , I'm following ipfs for a while now , and using the browser sessions as full nodes is an exciting idea.

One thing I wonder about , Is the browser -> ipfs node communication can be done via webrtc (I think the post refer to websocket)?

So the data sync can happen between browsers .

I did play with webtorrent for a while , and achieved a similar behavior using webrtc . I had one nodejs server , and multiple browser sessions . It worked well for video streaming peer to peer (poc level )

Browser IPFS to desktop IPFS communication can happen via websockets or webrtc right now, since those are the transports the browser currently offers us.

The stack we're using for networking is libp2p, a peer-to-peer framework, which has bunch of implementations for different transports. You can see the full list of those here: https://libp2p.io/implementations/#transports

Currently, most browser-apps using IPFS prefers websockets as it's using a lot less resources compared to webrtc, but once webrtc-quic[0] is ready for prime-time, it would be easy to write a transport for that and hopefully will be a lot more efficient than the current webrtc implementations.

Disclaimer: I work on IPFS/libp2p fulltime

- [0] https://w3c.github.io/webrtc-quic/

I doubt the usefullness of orbitDb. Ipfs-nodes only store and distribute content which they have accessed before. When you have a database with user-based data, noone except yourself will distribute it. This means you could have the same distribution with plain webrtc instead of height-ping ipfs.

It's true that availability is not "automatic" as per the software itself (rarely is in P2P networks). To make sure the data stays online (whether IPFS or OrbitDB), one needs to make sure there are enough nodes serving the data. In OrbitDB the replication protocol will make sure the full database is downloaded and served upon opening the database and as all db-data is IPFS-data, the db will be served just like any content on IPFS. So for example, for an application built with OrbitDB and IPFS, one would probably want to run some nodes themselves to "pin" (IPFS parlance) the databases and serve them to the network.

Edit: There's an open issue for a "OrbitDB pinning service" (https://github.com/orbitdb/orbit-db/issues/352), it's in the works atm.

But doesnt this mean I just distribute a big json over ipfs? Where do I need orbitDB for that?

Not quite. The data structures for the "database" part are more complex than a single json structure. Each update to the database is an "individual json" which get linked together using an append-only log CRDT (https://github.com/orbitdb/ipfs-log) and you'd essentially serve that log. So instead of "one big json", it's "many small jsons" and the state (view) of the database gets calculated from those small jsons.

Thank for clearing that. I think I have to read the docs more carefully

Although with webrtc you still need a signaling server and a turn/stun server. It would be interesting to try getting the webrtc stack to work fully serverless by using js-ipfs to exchange ip for stun and then do signaling using js-ipfs with websockets. Then you could open up peer channels to exchange data without using js-ipfs but by using it for the initiation and signaling layers you could potentially setup a pure p2p video chat system or file sharing system.

True, currently we always need some sort of signalling server to be able to connect to browser<>browser peers but once you done that, you can connect directly to other browsers (truly P2P in the browser).

Now, if we can have js-ipfs and go-ipfs nodes on desktop automatically act as signalling servers (if you start with a `--server` flag or something), we'll be able to decentralize the need for signalling servers as well, and remove another point of centralization. Getting us closer to the goal as well.

Maybe in the future we'll be able to have TCP/UDP sockets in the browser, but we'll see when/if that happens.

Disclaimer: I work fulltime on IPFS

It’s too bad that the flowering of distributed hash table / content-addressable storage research & development nearly precisely coincided with Internet Explorer’s ossification of web standards, or maybe we’d have had something like IPFS built into every browser 15 years ago.

Just a warning, the page takes 75 seconds to load.

took 5 seconds for me to load on tor, with noscript and ublock on. treat yoself with the world of no js or ads, it's great.

Instant for me.

Who pays for ipfs?

Protocol Labs - https://protocol.ai

They did a big ICO with Filecoin.

They pay for (most of) the development and running some public nodes, but in the end running IPFS is up to individual nodes.

It works like torrents: nodes only re-host content that they want to.

Original site is inaccessible for me ... is this a temporary issue?

Seems fine now, loaded quickly.

Works for me now, too. Temporary hiccup, apparently.

inaccessible even via VPN for me...

finally got through after a few minutes.

Reply on news.ycombinator.com