Hacker News Reader

Comment by kpcyrd | Hacker News Reader

You can rephrase this sentence:

> it really doesn't necessarily lead to any kind of true decentralized <data storage> unless someone else has decided to <store your data>

You might have an overly romantic idea of decentralization that doesn't necessarily align with the actual definition of decentralization. I would even argue that there is no solution for the idea you're suggesting. You can't have data that isn't stored anywhere.

The permanent bit in ipfs is actually referring to something else: The data isn't guaranteed to be available at all times, but the link is guaranteed to point to the correct data. Slightly anecdotal, but a while ago there was a discussion about a pdf in #cjdns. Somebody had an ipfs link, but nobody was seeding it anymore. A few hours later somebody digged out a pdf from an old archive but wasn't sure if it's the correct one, so we ran `ipfs add file.pdf` and the ipfs link started working again.

View on HN

> A few hours later somebody digged out a pdf from an old archive but wasn't sure if it's the correct one, so we ran `ipfs add file.pdf` and the ipfs link started working again.

So the file was lost until it could be recovered from a different source and IPFS served the function of a glorified hash database, that's something very different from what it promises.

Bittorrent has the same mechanics with magnet URNs, but you never see them promising permanent storage.

> Bittorrent has the same mechanics with magnet URNs, but you never see them promising permanent storage.

To be fair, IPFS does not "promise permanent storage" either.

A link pointing to the data is fairly useless if nobody is seeding it.

I have torrent files that point to the correct data but it's impossible to obtain because the data they point to isn't being seeded. But just like in IPFS, if I have the correct file I can seed it again simply by adding it to my torrent client into the torrent job.

At least a third party can prevent link rot in this situation. Link rot in the standard web is really bad these days. We do have the internet archive that is trying to prevent it, but then you still need a tuple of (URL & access time) and you have to hope that one organization cached it.

The link rot still occurs on IPFS when the content isn't being seeded, I don't think that's an improvement.

Seems way more tractable to me than solving link rot in the regular web.

Not really, if some niche content with 20 people who read it goes offline, it is very unlikely the content will come back.

And then someone will have to put in the money to pin/host the content anyway if you want to keep it online.

Link rot is solvable with tools such as WARC Proxies.

> Not really, if some niche content with 20 people who read it goes offline, it is very unlikely the content will come back.

You think these 20 people will stay offline forever?

You think these 20 people will keep that niche website they read once in their cache forever or even pinned?

Torrents die all the time because nobody bothers to donate bandwidth to strangers for content they barely care about.

How is IPFS different?

It is different in that a completely unrelated third-party can pin this content of their own volition, and it will continue to be provided under the exact same address/hash. In the web, the domain owner is the single entity who can ensure survival of the content.

Torrents do the same thing. Dat also does that. Even better in either you don't have to think about pinning. Why should IPFS be the one?

To be clear: I was referring to an overly romantic idea of IPFS I had actually had at one point, and which I acquired because lots of other people were talking about IPFS that way.

I am aware that IPFS does not actually do this, but a lot of people talk about it like it does, like IPFS means you can have websites without really needing to have/pay for any server yourself at all. (Which is, yes, technically sort of true, but only if visitors to your site reliably pin its content.)

You don't have to actually pin things. Merely visiting will cache and rehost the files, and they will only be GCed at some later time. All pinning does is disable the GC for those files, which, as of a few months ago, weren't getting GCed at all anyway, so visiting something was equivalent to pinning it.

"So unless for some reason I specifically think to help share some webpage or other, it'll just get auto-deleted from my machine when it cycles out of the cache" - my original post above.

I was thinking about that content effectively "linkrotting" away if nobody visits it for a while, say it's old posts on a blog or something. I suppose how big of an issue this really is comes down to how long stuff would tend to stay in that cache in a real, practical usage scenario. I guess I'd been assuming that it'd only be day or so, but it could be longer in practice, depending on how much space is actually allocated to the local cache and how much each individual bit of unique content takes up and how many such unique chunks of content will be downloaded per day, along with maybe some more complex factors besides first-in-first-out for deciding what to collect.

(I elaborated a little more about this below, when I thought about it kinda working like an automated torrent manager that made sure you maintained a decent seed ratio, didn't exceed an upload data cap, etc. and shuffled stuff out when it was taking up too much space or had hit a target ratio and wasn't going to be seeded anymore, with some prioritization for "is there anyone else seeding this", "is this a highly-demanded bit of content", etc.)

And in any case, even though a centralized server is still needed for p. much any practical use case, it's not like that makes IPFS useless - it's still possible in principle for any given bit of content to stick around basically forever and its link to keep working, even if the server goes down, as long as other people want to save it.

Plus, even if almost nothing is going to really be truly dcentralized, it could make hosting a site a heck of a lot easier because the traffic you personally have to deal with may be drastically reduced, as popular content gets uploaded to the first users who can then share with later users. You only have to be the exclusive host of content that's accessed more rarely than the period it'd typically stick around in the cache of the last person to ask for it. (depending on whether or not that last person is currently online.)

(It's easy to imagine a case where short-term decentralized hosting from user caches could be almost entirely sufficient - say, an IPFS-based version of 4chan, where threads are inherently short-lived and temporary objects anyway, and only in the very slowest and least-populated boards would you have threads that are checked so infrequently that you couldn't rely on the currently online user caches to contain that content. You'd still need a central server to do things like spam filtering, user authentication (to make bans stick), and update the index of what files hashes and post hashes are in which threads and in which order, and which threads are currently alive on each board)

What IPFS does do, however, is make it possible to host your website on a server with minimal capabilities - even your personal computer (provided it stays on) - because an increase in views will equally distribute the load.

That's not quite correct, the increase in views will still increase in load, especially if the increase is very sudden and not very many nodes have the content yet.

> an overly romantic idea of decentralization

For my part, I think it's more romantic that individuals in the network might affirmatively choose to pin the content that is meaningful to them.

Reply on news.ycombinator.com