Filecoin, StorJ and the problem with decentralized storage (2019)
randomoracle.wordpress.com
randomoracle.wordpress.com
The whole idea of a blockchain is reputation CAN be built up over time, because every transaction is recorded and there might even be incentives to associate nodes to show X nines of pool reliability.
Is it a bad idea to use this as your only source of storage? Of course. Is it less useful than AWS if you do large local burst queries? Of course.
But none of the concerns mentioned in the post are relevant. This can be cheaper than AWS because they charge an enormous markup, and this automatically provides worldwide replication and accessibility.
Amazon already erases books from Kindles, Google scans your Docs for blasphemy against Fauci, and both have been known to throw businesses off their services. In a few years it will seem crazy not to have an extra copy of your data on here.
There are even options like S3 Intelligent Tiering that enable you to store objects on S3 Standard and it will intelligently move them to lower-cost options based on the usage patterns of the data.
There are six different pricing options of S3 storage. There are EBS volumes, EFS volumes, FSx for Lustre and many other options for storage.
I used to mine BTC, currently mine Ethereum and looked into mining Filecoin and was left wondering if the people buying into it have any idea what cloud storage options are actually out there. There is so much competition and the prices are sooooo low that I don't see any viable use case for Filecoin outside of the one you mentioned about being concerned about being shut down by a single vendor.
There are good arguments for decentralization and censorship resistance, but from a first principles perspective scale and market pressures are good bets for delivering cost efficiencies.
You must also think "scale" and "market pressures" keep down the price of Windows.
Hint: AWS has wonderful margins because everything is tied together, and bandwidth charges to get data out are huge. S3 is a great solution for data used on AWS, but costs more than 30x Storj on bandwidth alone ($90/TB for outside access, plus the cost of doing cross-region replication).
If you're going to work from "first principles" it always helps to have the first hint about how markets actually work--and that different products make different trade-offs (local transfer speed and latency vs. cheap global redundancy)
Yeah but if use AWS S3 for off site backups, those huge charges cost me exactly $0/TB.
I agree with others - Filecoin and Storj have no meaningful commercial use cases.
Who wants to understand how those things work, let alone use it for anything that involves commercial use cases?
AWS S3: put, get, delete, pay. Done.
Probably on the basis that backblaze costs an order of magnitude less for the same product.
Ps everything on aws is marked up by like 300%, it's easy to beat them on cost.
Isn't this a non-issue because you can encrypt your data prior to uploading?
What is this referencing? I'm just curious, this is something I haven't heard about.
Discussion: https://news.ycombinator.com/item?id=23275308
As usual, it may not be what you think.
And for file storage, there are no data monopolies already. Take for example AWS S3 storage -- there are multiple providers (AWS, Backblaze or local Minio) and many, many clients in all sorts of languages.
1. Economies of Scale
2. Network Properties (robustness, scalability, latency, etc.)
It seems to me that a crucial element is missing from these analyses: organizational policy. All other things being equal (which they may not be), it seems like decentralization appeals to those trying to restrict top-down control. Historically, this has been an ideological thing (e.g. bittorrent's copyleft slant), but more recently I've seen mainstream endorsement of concepts such as "flat hierarchy", "distributed workforce", "collaborative work", etc. Interestingly, I don't see this discourse being picked up by decentralization evangelists, and I'm not sure what to make of that.
Off the cuff, it seems like decentralized storage tech has a compelling story to tell about small distributed teams, working in a loose, peer-to-peer organizational structure. In this context, my mind immediately goes to the functional centralization of cloud storage. In general, the Cloud tries to concentrate control over computing resources in a single department (i.e. the devops team). But what happens when a bunch of freelancers collaborate on a project-by-project basis? From what I've seen, either (1) they use cloud providers, but the account is under the client's name or (2) they use some kind of SaaS solution with support for sharing or other forms of collaboration. With decentralized storage, it seems like a "bring your own cloud" approach is possible in principle, where these freelancers would pool resources (storage, compute, etc.). Are there any pain-points here? Is there perhaps a business problem that needs solving?
The question for me is whether these new patterns of collaboration are really becoming a thing, or whether it's a fad amongst business-school graduates.
At any rate, this comment is an invitation for y'all to opine on the subject. I feel like a good analysis of organizational policy could help us decide whether a decentralized web is economically viable, but I'm kinda stumped.
though most of the punches land toward the end, it is a fascinating and short read, in which she dissects, with the sober disappointment of a former optimistic evangelist, exactly why structureless ("decentralized" before that term became vogue) movements are never what they claim to be, and indeed cannot be.
why? because structureless coordination of groups above a handful members doesn't scale beyond the achievement of the simplest kinds of goals, such as consciousness-raising (or, in crypto-promoter terms, spreading "adoption").
any group objective requiring specialization of labor, coordination of resources and responsibility inevitably succumbs to the insidious creep of informal, but officially disavaowed structural networks of insiders.
it is especially pernicious for newcomers, who have been heavily messaged about "decentralization" or "structurelessness" and engage themselves with the groups identity without any awareness of these cryptic back channel networks that actually govern its operation. only after they are committed do they slowly find out there really is a hierarchy and a command structure.
i highly recommend a quick read for anyone considering the merits of a given groups' claims of being decentralized.
The confusion between structurelessness and decentralization is interesting in and of itself, though. It seems like it is exactly this conflation that prevents people from asking the interesting question: when does a centrally-organized social system (e.g. a business) benefit from a decentralized protocol?
It seems like this should have something to do with usage/access patterns within the business, yet we never hear about this.
"Cash and carry" grocery outlets were a 20th century innovation. [0] Before that, the norm was to have a line of credit with the business, and in many cases, to accept their terms for delivery. Cash transactions anonymize, since settlement is done at the counter. You don't have to assess the buyer, you just need to verify the bills and change are real.
However, that isn't the entire story. While there were businesses using cash before this, they faced difficulties with accounting, supply logistics, and other elements that made it hard to conceive of something like a "supermarket", carrying a vast variety and quantity of goods on a daily basis. So then we have to look at all the pieces that fell into place to make it possible.
The automobile decentralized access to mobility, making carry-away a real possibility for more people, and thus making it possible to apply cash-and-carry in more places. Supporting elements like cash registers and refrigeration were becoming mature enough to support new forms of retail and allow more parts of the transaction to be delegated to local outlets and low-wage employees. The inter-war years really saw a whole set of technological innovations that were used in combinations to propel social changes and different categories of business(e.g. fast food), many of them decentralized in some respects but centralized in others - supermarket chains, as opposed to local markets.
These are the kinds of changes that are hardest to assess in full; when you decentralize one thing, centralization is "squeezed" into other parts of the economy, it seems. The obvious example for this phenomenon is Amazon, leveraging an apparently decentralizing mechanism(online retail - premised on an internet with sufficient bandwidth and security to list goods and take payments) into becoming the world's largest retailer. So it's centralized on one axis, but decentralized on others - a buyer no longer has to go to a particular physical location to purchase something, when all of it can be delivered to the doorstep.
[0] https://en.wikipedia.org/wiki/Cash_and_carry_(wholesale)
To give more detail about the sort of system I mean, here is an excellent description of the approach favoured by the team behind the Matrix communication protocol:
https://matrix.org/blog/2020/10/19/combating-abuse-in-matrix...
However, I am not convinced it is exactly applicable to the situation I have in mind. Freeman's point pertains to extreme decentralization of organizational policy, i.e. the flat hierarchy. These are arguably not even organizations; the essence of organization is precisely hierarchy!
I'm more interested in organizations that do have some degree of centralization. My hypothetical freelancers are certainly not disorganized. What I'm trying to grasp is the conditions under which any group -- especially one with centralized policymaking -- may seek decentralized infrastructure.
In other words: when does decentralized infrastructure serve the goals of a well-organized group?
the paradox is simply this: decentralization implies the absence of a decentralizer. there can be no "center of decentralization."
I contend that the term is much closer to incantation than implementation in practice.
Hmm, I'm not sure I follow. On a technical level, a pure p2p system continues to function without central coordination even if each individual node is "owned" by the same entity. These seem to be two very different levels of analysis. If I understand your claim correctly, you are pointing out that this hypothetical entity is centralized -- and I agree -- but that doesn't magically turn the p2p system in question into something else.
To reformulate the question: when is it advantageous for this abstract entity to use a p2p system in this way rather than a client-server system?
Is it e.g. a question of scale?
I have a nagging suspicion that it's partly a question of usage-patterns, though I'm struggling to put my finger on it. This is what I mean by "organizational policy" (which is admittedly an opaque term...)
p2p requires peers, so called for good reason, because their interests and cost/benefits must align sufficiently for them to engage in using the same protocol. thus, "who these people are" dominates the question of control over the network.
tl;dr behavior of users networked by a putatively decentralized protocol will ultimately conform to the social organization of those users, not to the rules of the protocol. these will be routed around or backgrounded as needed to achieve the objectives of the dominant members of the group hierarchy.
I think I could get excited about FileCoin if it allowed me to pool storage with friends and colleagues. I've been wanting to have a distributed hacker-garage for a while [0].
Importantly, I want it to be independent of any cloud provider because (1) it's hard to freely experiment/prototype when you have to keep track of billing and (2) functional centralization makes the AWS/GCE admin de facto responsible for everything running under the account. This is great for large enterprises that want to virtualize their IT department, but it really goes against the grain for hackers, freelancers, and -- IMHO -- startup founders.
Returning to Filecoin, I feel like all the *coin ventures make the same fundamental mistake of over-estimating how much we care about money. I really don't care about pimping out my hard-drive for a couple of cents. I'm motivated by gains on the order of several hundred to a couple of thousand dollars. The counter-argument I usually hear is that there will be more interest in developing nations, but then I don't want to store my data in e.g. Pakistan. I want data for pet projects in my own hacker-garage, and I want production data close to the end-user.
/rant
[0] Incidentally, I'm working on a project to do exactly this. It's a way of clustering a set of computers over the internet to produce a virtual cloud. It would be super cool to use FileCoin as a block storage layer in this context, but restricted to the hardware we own.
The alpha already lets you do what you're talking about, but I haven't used it yet and so I can't speak to its current stability.
Edit: One thing I liked about Storj was that geographic redundancy is built in. I know Amazon has that too, but if three data centers caught fire on the same day, Amazon could lose your data. Super unlikely with Amazon, but virtually impossible with Storj, and most S3 providers have less geographic redundancy than Amazon.
> knowing your data is there does not mean that you can get it back when needed. This is the subject of the next blog post.
This is something I'd love to hear more about.
Instead, decentralized storage gives you the ability to store your data with dozens of independent entities; as long as the behavior of these entities is sufficiently uncorrelated, then it is highly likely that you will always be able to retrieve your data from some subset of them.
or one entity operating dozens of nodes? droplets are cheap, pwned windows boxes even cheaper. they don't even have to be cheap by regular Joe standards, they could be cheap by nation state, or ransomware gang standards. the point is there's no way to prove they are decentralized. this is an intrinsic, structural property of the cryptocurrency model, thus it is not a mere technical defect that will get solved "in the future"
proving decentralization would require zero privacy, because you'd have to prove independent nodes are indeed independent and not part of a cartel, which means you'd have to somehow prove who operates each one.
nothing but faith-based decentralization here, as with all cryptocurrencies.
Hold on there, SLAs are actually something that blockchains are great at. Smart contracts provide cryptographic guarantees of service. In several of these types of systems, node runners must put up collateral as "stake" in exchange for collecting fees for their services. If a participant in the network identifies a misbehaving node (violation of SLA) then their stake can get slashed, or burned completely.
If a Filecoin node claims to be holding data for someone, but is unable to provide it in a timely manner when legitimately requested to do so, they're in for some serious penalties.
The great thing is that the consumer doesn't need to worry about some big company hiring expensive lawyers to weasel out of their contractual obligations. The rules are all readily transparent and enshrined as code that anyone can read and verify the validity of before participating.
It's not _likely_, as that would negate the usefulness of the whole system if it became an issue, and therefore negate the cash flow into the system.
Ultimately what's keeping the storage providers honest, whether they're centralized or decentralized, is that their future profit stream is based on their current behavior, and they presumably care about future profits. Code and SLAs ain't nothing but words.
Disclaimer: I have no idea how these systems currently work.
So the code shows file, all my proof of storage is correct and users are hosed. There’s no “proof of bandwidth” (yet)
It's still quite expensive if you ever need to retrieve data in a hurry, but it used to also have a complicated and unintuitive pricing model that made it extremely easy to accidentally spend orders of magnitude more money than you intended to. They redesigned the pricing structure in 2016 to make it a lot more straightforward.
The real killer is the $0.09/GB egress fees. That dwarfs the $0.00099/GB/month storage fee and the $0.0025/GB bulk retrieval fee.
I consider it as a mitigation against disaster and expect to never retrieve other than for testing the process. So it seems good for extremely low probability, high impact events. And I trust amazon to meet their SLA.
* not counting usd inflation, because they don't lock storage price to coin value (with crazy volatility), they fix it against usd and covert it to their coin when paying their nodes.
Blockchain is not used for distributed storage itself. Existing concepts like IPFS are used for this. Blockchain is used to confirm that someone is storing certain data and to verify it's integrity through an additional protocol.
From ipfs.io about Blockchain
>With IPFS, you can address large amounts of data and put immutable, permanent links in transactions — timestamping and securing content without having to put the data itself on-chain.
Please correct me if I am wrong.
edit: Wouldn't erasure codes just completely fix it (as described by the Storj guy)?
A small filecoin operation would not have the legal funds to defend against an aggressive prosecutor trying to “take down a network of dangerous pornographers” and would get little sympathy from a general public that doesn’t understand the tech.
Until there is strong legal precedent, I wouldn’t entertain a mining operation. Maybe I’m paranoid but it just doesn’t seem worth the risk.
>it just doesn’t seem worth the risk.
Well, the concern is only yours.
This does raise the point that if police contact you to remove some data you are hosting for someone, you lose money, and they will likely not give you back whatever money you lost (all the more reason it should be fully anonymous like Freenet but IPFS and co are obsessed with trying to solve every use case in the world including legal compliance).
FileCoin is, at best, an inefficient way to incentivize people to store your data in an imaginary trustless world. (At worst, it's an obvious scam.)
You can imagine any hosting provider offering regular IPFS hosting. Sign up on our website, send us a list of data to store, pay us money, and we'll guarantee you can get it back or you can sue us - same as the guarantees by any conventional hosting provider. This works fine and well without gluing Filecoin to it.
I'm not sure there's a huge difference between that and this, other than anonymity for providers (which is the major critique of this blogpost.)
In fact the two technologies would complement each other really well. Imagine that you could pay someone to make content available on IPFS, basically a distributed pinning service. However this isn't trivial because IPFS generally assumes that people with files would like to share them, however a for-profit storage node would be incentivized to use as little bandwidth as possible.
I outlined in my post a specific, trivial way to pay somebody to make data available on IPFS.
This is like saying HTTP provides storage because you can pay someone to host your files and allow you to access them over HTTP.
I dunno about that. Amazon and Google, at least, charge significantly more than I could go out and buy a few drives for on sale. (AND, of course, they charge it perpetually. Dunno how StorJ works.)
As someone responsible for my company's data, I cannot make a sound argument to convince management (or myself) to use anything but Amazon S3 (or Google or Microsoft) cloud. The data is simply too important to trust to a smaller entity.
Maybe coin storage can boost Storj's reputation. And I certainly favor decentralization for Crypto.
If that's the case, I think you could argue that open source and distributed solutions are much easier to make more reliable than centralized solutions. With a centralized solution, you're stuck with trusting their reliability or implement some reusable API (or use some 3rd party's API that abstract that away, introducing more error surface) so you can also store the data in multiple places with the same API.
Or, you can just use a open source decentralized solutions and run it locally geographically, hosted on dedicated servers and "in the cloud" (meaning other users who host it) with the same API without really caring about _where_ the files are, just _which_ files you need.
And that's the beauty of content-addressable networks like many of these are. It doesn't really matter where the data is as long as you know what data you need. That makes it easier to provide reliability for yourself, not harder.
Once you dig down in the details and compare them both carefully, I think you can make sound arguments both ways, it's not as black & white as you make it. Ultimately it depends on one's use case.
How do I communicate the reliability of a distributed cloud that is essentially a collection of random servers? Is there a way that each of these server's reliability is accounted for by the high-level service?
First, on the reasonable: absolutely, privacy and security should be layered on top of whatever storage platform you use using your own encryption. No objection! That's the right approach. It is always better to bring your own encryption, and having it layered so your storage provider can't have access is great. I have much love for @cperciva's Tarsnap. It's awesome.
The downside with a product that punts encryption to the user is that users /typically don't do this/. In fact, users like to share things and serve content to others. Bringing your own encryption that the sharing infrastructure doesn't understand means it is much harder to actually use the product for sharing. This is why Storj includes hierarchical encryption for delegated sharing. It's not that embedding encryption into the framework is better encryption by any means, but it is a better default. Users are by default protected by their own keys that we don't have access to, which is a better default than the cloud. Should people use something else also? Sure! If it suits them.
On the unreasonable:
The author says:
> [I]t is very unlikely that a decentralized storage market can offer an alternative that can compete against centralized providers— AWS, Google, Azure— when measured on any of these [cost, speed, reliability] dimensions.
This is a testable assumption, and the author didn't test it (post is from December of 2019, we were in late beta then, working very well).
Storj is cheaper ($4/TB/mo, was $10/TB/mo in December of 2019), provides 11 9s of durability (no SLA offered until March 2020, but we had and still have never lost a single object out of billions), and is equivalent in speed to providers like Backblaze (and getting faster). It's also multiregion by default (obviously). This is a screaming deal on all dimensions. Try it out for yourself. Our metrics are delivering on all three dimensions.
This author is picking and choosing between architectures of Filecoin and Storj, which is fine, but each should be evaluated in isolation. Filecoin and Storj make very different design decisions with very different payoffs. The author argues Storj can't deliver on its promises with appeals to Filecoin's architecture.
Filecoin absolutely uses blockchains and proof of storage and whatever else. In fact, Filecoin does fall victim to requiring lots of resources (see https://docs.filecoin.io/mine/hardware-requirements/).
But this is just true of Filecoin, this isn't a problem with decentralized storage in general, and Storj is an excellent counter example. Storj does not require lots of resources (and in fact many of our node operators use Raspberry Pis). Storj is not a blockchain, uses statistical audits that are low-effort and low-CPU for storage node operators, and works great for idle capacity.
If you're interested in how we can achieve these things, you might take a look at an older blog post I wrote explaining (without naming) why Filecoin's architecture is fundamentally flawed: https://www.storj.io/blog/replication-is-bad-for-decentraliz...
Storj and Filecoin are both decentralized storage products, but they are really fundamentally very different, and their differences are worth understanding.
You presumably agree with the node to pay for the transfer out, but if both counterparties are anonymous and do not trust each other, how is this accomplished?
> Whenever a Satellite on the Storj network has a less than stellar payment, demand generation, or performance history, there is a strong incentive for the storage nodes to avoid accepting its data. When a new Satellite joins the network, the participating storage nodes will commence their own vetting process. This process limits their exposure to the new and unknown Satellite, while building trust over time to highlight which of the Satellites have the best payment record.
They also incentivize complete file delivery by paying for chunks of bandwidth as they're used to deliver the file, rather than all up front.
If they send you the data first, then why should you pay them? If you send them the payment first, then why should they send it?
I'm sure there's a piece I'm missing, but have I missed something obvious?
AFAIK Filecoin uses payment channels so you probably lock some money, retrieve data, then unlock the payment.
If I pay you upfront for the first block, what stops you from then saying: actually I still want more money to start transfering the first block?
If bonds solve the bad actor problem, then this use of capital would directly enable safe data storage. I guess it depends on your definition of “productive”, but that definitely feels like a good use of capital to me.
Let's say the entirety of YouTube was decentralized in a way that every internet connected device plugged in to the wall starts sharing watched content to other viewers who want to watch that content. If it's your neighbor, they'll download it straight from you, via the closest router/hub. Suddenly the company's hosting is more providing a "core archive" of data, and every user becomes a edge node of that content. Internet providers can, if they chose, pre-share content they think will be popular, in order to speed up their own infrastructure, and so on.
I'm not saying this is bullet proof or anything, I just think you're too quick to dismiss it without really saying why you think it's impossible (or "virtually certain").
Most home networks are wireless only at this point which is pretty bad for throughput, latency, and reliability if serving other homes. Maybe ISPs could start building some storage and IPFS into modems? I think handling the metadata of a p2p system might still be too costly. Imagine every neighborhood having to run a torrent tracker for every media file produced in the last ~year.
I assume all these decentralized storage systems are datacenter-based so I'm comparing random Chinese datacenters vs. FAANG datacenters.
If you pay attention to the naysayers, you're gonna have fun staying poor.