Hacker News Reader

Comment by sharemywin | Hacker News Reader

I still go back to this paper for questions about decentralized storage:

http://blog.dshr.org/2017/07/is-decentralized-storage-sustai...

I think this argument fails even by its own standards. It applies equally well to Bitcoin mining, and while there is clearly some centralization there, it is still quite decentralized. At least, decentralized 'enough'. Secondly, storage is fundamentally different than other commodities. The decentralized storage market is more like AirBnB, where you're consuming the excess capacity of capital. Most people have tons of storage already that they're not using. Extracting rents from that space is found money, and therefore they have inherently lower costs than anyone supplying storage in a commercial way.

Why do you think decentralized storage could ever work on excess capacity? I cannot imagine any incentive for it. It's either profitable enough to do it on a dedicated capacity, kind of like a hosting company, but without all the hassle and marketing. Or it's a burden.

The point is most people have a bunch of free space on their hard drive they're not using anyway. I've got a few hundred free GB on my desktop, and a couple more TB on my home server. The physical hardware for those drives is sunk cost, and I'm already paying for the electricity to run those machines all day. It doesn't cost me anything extra to merely have some data sitting on the unused space on my drives, so why not make some money off of it?

On the other hand, if I wanted to create a dedicated machine just to farm filecoins, that _would_ cost me extra for hardware and electricity.

This is why decentralized storage has an advantage in this regard. Unlike mining, storing files incurs almost no extra costs for the average home user, but does incur costs for dedicated storage systems.

It costs you the bandwidth (it's not a long term cold storage) and extra processing power (so electricity cost) for running filecoin. This may well be lower cost than the gain from hosting, but it's still non-zero.

Bandwidth is effectively free for many users (no data cap), and the amount of processing power required for something simple like file storage is pretty trivial. You're right that it's non-zero, but I'd imagine it's probably on the order of pennies a month.

If you don't have data caps, be happy. But, they're still very common and even if you don't have them in your contract, you may get penalties applied after some threshold.

The cost can be also in concurrent access. Unless you apply a heavy QoS on your network, serving filecoin network may degrade your other tasks.

Now if you don't have caps and want to keep it that way, I wouldn't be excited about filecoin. Unless you're actually paying for guaranteed bandwidth and it's part of your contract, your ISP is overcommitting. The moment people actually start relying on that fact, the ISP can do one of: raise prices and buy more pipes, lower the speed, reintroduce caps. The first one is a limited resource that takes time to apply. The other two are quick and known solutions.

Is it really? Even just extra 10W over a month is just under 7 kWh and might approach a dollar[1]. Spinning 3.5" HDDs from time to time is especially costly and they often spin down when unused for a long time. Load spike might also make CPU freq scale up and consume more power. Same with bandwidth, you'll incur ISP wrath or saturate it (or keep wasting most of that free space), even 80 Mbps[1] is 'just' 36 GB / hour and under 1TB per 24h. And that's without including overhead from Filecoin, TCP/UDP and IP themselves.

It sounds lose-lose-lose situation: keep wasting your space or saturate your bandwidth or waste power on keeping HDDs spinning all the time due to accessing them every few minutes. It may vary with your usage patterns (if these disks are already always spinning and you have REALLY good bandwidth or two separate lines) but still, seems barely worth the effort.

And why couldn't it be optimized in a purpose built machine? Cheap Tibetan hydro and free cooling (altitude), low power CPUs, fast internet, fast switches and routers, 10 gigabit lan, many HDDs and SSDs (dug up from the Western trash piles even, it's not like failing once a month is a big deal). Chinese already have ASIC for bitcoin and this seems easier to make than that.

[1] I have no idea if I'm generous or stingy on USA power prices and network speed.

I guess the exact costs would depend on usage patterns, frequency of file read/writes, etc.

It's worth noting though that Sia (another decentralized storage network) is estimating their current storage costs at $2/TB-Month. That's half the price of Amazon Glacier. Whether that's the result of users farming from their home PCs or giant Chinese storage farms though I have no clue.

You could make the same argument about AirBnB. If it's profitable to rent out a room in your house, it's profitable to open a hotel. But people are still renting out AirBnB's because their homes are a sunk cost.

There is another macro-economic issue here:

With a decentralized trustless system you have to assume a higher failure rate for nodes than with a central trust based system.

If you assume a higher failure rate you have to replicate data more extensively to achieve reliability.

If you have to replicate data more extensively then decentralized will always be more expensive than centralized.

Not necessarily if the inputs are free (e.g. unused disk space). However Spotify, Skype, and Joost all found that CDNs are cheaper than borrowing free bandwidth from people so the same may apply to storage.

Did they find that out? I thought they realised it's unprofessional and unacceptable to turn your users into hosting nodes.

You achieve reliability with erasure codes, not with more replication. And failure rates are likely to be identical if not better, since incentives are the same as with hosting companies, but since it is decentralized there are a lot more availability zones.

> You achieve reliability with erasure codes, not with more replication

If nodes are not trusted then replication has to play a major roll.

It doesn't matter how you mix up the data. You still have to store it somewhere and if that somewhere is a random untrusted stranger you will need to copy your data to a lot of random untrusted strangers in order to have kind of reliability.

you could achieve some trust through a reputation system. the longer you've been reliable the more reliable you are.

It does play a major role, in fact I think not everyone figured this thing out yet, some not even taking into account countries as risky availability zones. But the storage space overhead for redundancy is at worst 2x, which could eventually be reduced by a lot.

Also, if decentralized storage systems become very popular, then people will buy lesser storage space with their computers, because they expect storage space to be cheaply available online. In that case, the number of individual hosters would reduce and big companies/pools would step, making it less decentralized.

pfraze wrote a nice response to that post the other day: https://medium.com/blue-link-labs/in-response-to-is-decentra...

Here's one easy rebuttal: on the Sia network, renters are able to choose which hosts their store data on, and they will use a number of factors to decide which hosts to use, not just price. For example, low latency is important for some applications (e.g. a CDN), so it's not acceptable for host in China to be serving content to users in Europe.

Another way of thinking about this is to realize that the centralization of all data storage in China has not happened already. Why not? A motivated company (one as big as Amazon or Google) could move their datacenters to China if price was the only consideration. But they haven't, because latency (among other things) matters.

I'm not sure if this rebuttal applies in the case of Filecoin, since it's not clear to me to what degree renters are able to choose the hosts that store their data.

Basically the same reason that bitcoin mining is becoming centralized by a small set of miners in China.

At least there is a benefit to being close to the requestor in terms of bandwidth/latency so as long as the network rewards that storage should be more decentralized than mining.

Reply on news.ycombinator.com