Hacker News Reader

Comment by rudolph9 | Hacker News Reader

There are a lot of nuanced ways a trustless distributed system can be more efficient. The article is pretty light on details but consider the content addressing mechanism. The data can be securely served by any device and independently verified by any recipient. The minimum latency of reads is theoretically much lower than the sever/client infrastructure commonly used today. It’s like edge computing with any device that can connect to one another. Obviously a lot goes into ensuring the data existing on a low latency connection to the distributed-client but the there are a number of somewhat nuanced theoretical efficiencies that can be achieved by making content addressing the core of a distributed system (caching, [as mentioned] fault-tolerance, parallel computing, etc).

View on HN

> The minimum latency of reads is theoretically much lower than the sever/client infrastructure commonly used today.

This is an oversimplification: it assumes that discovery is very low cost and that there's a peer with a copy enough closer on the network that it's faster than talking to server-class hardware in a data center with a high quality network connection. Given the number of assumptions which need to be true for that to be a net-positive I'm skeptical that it'd be easy to hit anywhere close to the best-case theoretical scenario.

> Obviously a lot goes into ensuring the data existing on a low latency connection to the distributed-client

This is admittedly a non-trivial task but perhaps an example can shed light on way it can be more efficient in practice.

Consider the Apple Photos app. It's on your phone, it's on your desktop, it's on the cloud (iCloud storage). Say you have so many photos they don't fit on your phone. When you try to access a photo that is not on your phone it gets it from icloud (these are literally features that are currently offered by photos/iCloud). Now suppose your desktop has lots of storage, you usually look at your photos while connected to your local network, why cant you get the data from there?

The key to the efficiency of file coin is not some magic cryptographic token, it's not even "blockchain", the key is the underlying protocols being unified for synchronizing devices across multiple tiers of networking. Filecoin mostly just makes sure the data doesn't disappear and gives you an optional substitute for cloud storage. Content addressing, p2p communication by default, flexible/future-proof standards for negotiated what data is and how it is formatted, and in modular plug-and-play utilization of many different communication/synchronization techniques is the key to applications being built in more effective and efficient way than conventional client/server architecture.

The word that comes to mind is "flexibility". Want you store to be S3 buckets? Great! Want to host a private network, only your devices can access? Great! Want to joint the public network? Great! Want to ensure persistence of data via a token? Great! Want to just use the address mechanism? Great! Want to run the software in the cloud, on you local machine, in a sandboxed browser tab and have them all seamlessly forming a p2p network over a multitude of network standards (tcp, udp, web-sockets, etc)? Great!

I worked extensively last year with various ipfs techs utilized in a private p2p network consisting of ARM nodes capture time-series measurement devices and cloud based persistent nodes managing the pinset of data and persisting long-term storage. There are definitely some rough edges but again it's open-source built is such a modular way it's almost difficult to develop your self into a corner.

The software coming out of protocol labs is overwhelmingly not new technology. Protocol lab large focuses on utilization of battle-tested existing standards and tech (many of which are relatively ancient like TCP, ssh, git, json, etc) unified, without breaking existing stands, via a common addressing mechanism.

tl;dr Filecoin/crypto-tokens just a side note! The vast majority of software coming out of protocol labs is independently useful and largely focuses on bridging together widely-used battle-tested tech.

> Consider the Apple Photos app. It's on your phone, it's on your desktop, it's on the cloud (iCloud storage). Say you have so many photos they don't fit on your phone. When you try to access a photo that is not on your phone it gets it from icloud (these are literally features that are currently offered by photos/iCloud). Now suppose your desktop has lots of storage, you usually look at your photos while connected to your local network, why cant you get the data from there?

Yes, what Dropbox, Crashplan, etc. offered a decade ago — it's a neat sounding idea but there are two reasons why this isn't a huge win in practice: people are mobile so the number of times where you need an uncached file and happen to be on the same network is relatively low and increasingly few people have an always-on device with a ton of local storage free (phones are really good at generating high volumes of data so this is a non-trivial problem).

Working on things like this is really interesting but it involves a lot of work to handle unreliable clients or networks and bitrot (hashes don't solve this if your client helpfully replicates the bad sector from your desktop over the pristine copy on your phone). That overhead makes it a lot harder to beat conventional services, especially in cases where the time investment is greater than the possible savings.

> Working on things like this is really interesting but it involves a lot of work to handle unreliable clients or networks and bitrot (hashes don't solve this if your client helpfully replicates the bad sector from your desktop over the pristine copy on your phone). That overhead makes it a lot harder to beat conventional services, especially in cases where the time investment is greater than the possible savings.

The idea that ipfs/related-project are trying to beat conventional services is big misconception. The overwhelming majority of the tech does not preclude usage in conventional services. For example, IPFS can 100% be configured as server/client offering roughly the same costs/reliability many are used to. The advantage is interoperability with many different "services", conventional and alternative (e.g. filecoin) alike.

Can you expand that interoperability thought? It’s unclear to me how this could change the main obstruction of vendors choosing not to encourage interoperability - anyone who keeps their API locked down is unlikely to adopt standard IPFS.

Essentially it's a separation of concerns. The content "address" (i.e. how we specify the content that is desired) and the content "location" (i.e. the connections which serve the content associated with an "address") are distinct.

An "address" is also self validating. Hash addressing, where a hash of the data is the address of the data, are use to accomplish this. This is useful in many contexts and an [extra few bits][1] on the the front allow an address to represent much much more.

"location" is by default a distributed hash table enabling the fulling distributed routing of content and by uses [Kademlia][0]. But the software could easily be configured to with fixed values for the hash table mapping any "address" to the same location which happens to be a cloud provider.

Could you elaborate on what you mean by "anyone who keeps their API locked down is unlikely to adopt standard IPFS."?

[1]: https://en.wikipedia.org/wiki/Kademlia [2]: https://github.com/multiformats/cid#how-does-it-work

Is this really an advantage of the fact that the system is trustless and the ownership of storage media is distributed? You can an imagine a system where Amazon owns all the hard drive but still distributes/duplicates files across many different hard drives, matching the theoretical performance characteristics of Filecoin. In fact storing data on S3 from my cursory understanding already distributes data across multiple drives based on their durability claims.

The advantage is the flexibility. I kind of went on a rant about this [here][1] but to answer your question more directly; the advantage is how the flexibility the addressing mechanism provides. Blockchain-tokens are frankly kind of boring, they solve the double spend problem. What's interesting is the trustless distributed system tech is being developed around them!

[1]: https://news.ycombinator.com/item?id=24218074

Reply on news.ycombinator.com