Filecoin's Cryptoeconomic Constructions
filecoin.io
filecoin.io
1. How does this system deal with the "data withholding" problem? In other words, when people provide "storage power" their data will be repeatedly sampled to make sure it is available... but when an entity claims that samples aren't being provided as required by the protocol, how does the system determine that that person wasn't lying, if the sampled data is still provided correctly in a followup request? If the answer is "through arbitration", what prevent the arbitration system from being DDOSed?
2. The "verified clients" are certified by "a decentralized network of verifiers". How does this system prevent a sibyl attack, i.e how does it prevent verifiers from repeatedly verifying themselves using multiple accounts?
3. I notice this system doesn't mention the use of erasure coding, which is usually a common feature of similar schemes by other projects. Why is it that erasure coding isn't necessary in this system? In other words, if data is randomly sampled, how does a client make sure 0.001% of the data isn't missing if only 99.999% or less of the data has been sampled so far?
4. The filecoin organization has a ton of funds due to their successful ICO. This makes it hard for users of the filecoin network to know if it is truly scalable (since the filecoin org could just run a bunch of anonymous server farms with their funds that provide free storage to paper over flaws in the cryptoeconomic incentives) How can a user of filecoin get some assurance that the files they are storing aren't just sitting on a server run by the filecoin organization & are truly running on a decentralized system functioning through the specified cryptoeconomic mechanism?
isn't most of ipfs running on some combination of ipfs.io and infura? Will Filecoin end up in the same situation?
> How does this system deal with the "data withholding" problem? In other words, when people provide "storage power" their data will be repeatedly sampled to make sure it is available... but when an entity claims that samples aren't being provided as required by the protocol, how does the system determine that that person wasn't lying, if the sampled data is still provided correctly in a followup request?
Filecoin sort of splits this problem into two parts – "data withholding" from Filecoin's proof-of-spacetime consensus mechanism (a "storage fault" in Filecoin terminology, yes I know there's a lot of new terminology here!), and "data withholding" from a client that's requesting stored data.
Storage miners are required to prove to the network itself, not to any specific challenger entity, that they're storing files. Each storage miner is (basically) randomly challenged once per [short interval] to provide a compressed cryptographic proof in response to a challenge. The proof conclusively confirms that, during that period, the miner was storing the data being they'd previously promised to store. You can ctrl-f "if a miner goes offline" in the linked post for a surface-level description of how the network deals with storage faults. Ditching the data and recovering it later is economically irrational for pretty involved reasons – basically, recovery is more expensive than just storing the data over the short-ish intervals during which faults are recoverable.
When it comes to "withholding data" from clients – retrieval on Filecoin is just a market-based system for bandwidth. The solution to holding data "hostage," i.e. refusing to serve it at reasonable prices, is to store a few replicated copies (just like centralized storage services do for you today behind the scenes). There's really no upside to miners refusing to profitably serve you a file when they know or suspect you can get it from another source.
> The "verified clients" are certified by "a decentralized network of verifiers". How does this system prevent a sibyl attack, i.e how does it prevent verifiers from repeatedly verifying themselves using multiple accounts?
The short answer here – with apologies for the brevity; details forthcoming – is that verified data isn't meant to be scarce, and some degree of over-verification is expected. There will be a decentralized group of folks responsible for (quite permissively) verifying and renewing clients for fixed amounts of data, and declining to renew allocations for clients who seem to be abusively verifying data. We're optimistic that this will dramatically decrease the rate at which "fake" data is stored and (most importantly) ensure that there's always storage available for client data.
> Why is it that erasure coding isn't necessary in this system?
Basically: cool, novel cryptography! In particular, this is where proofs-of-replication and proofs-of-spacetime kick in. Check out this podcast with Juan to learn much more: https://filecoin.io/blog/filecoin-proof-system/
(Also – if you like erasure coding, it is totally compatible with Filecoin whether you're a miner or a client! I would be surprised if this feature isn't developed by the community in Filecoin's early days.)
> How can a user of filecoin get some assurance that the files they are storing aren't just sitting on a server run by the filecoin organization?
Really fair question. First and foremost, as a client, you get to choose your storage miner if you want to. You then have to solve another problem, of course, which is how to map a Filecoin peer ID to a real-world actor (or prove that it's not being run by Filecoin, or whatever). This is solvable in a bunch of different ways, which I won't get into here, but the high-level takeaway is that you're not just throwing your data at an undifferentiated storage interface with obscure inner workings.
More fundamentally – Filecoin is part of a huge ecosystem of open source projects. Transparency is a key value – highlighting the success of the community, including the many decentralized storage miners participating in Filecoin, is really important to us and the only way the network can succeed. You can hop on our Slack any time (https://join.slack.com/t/filecoinproject/shared_invite/zt-dj...) to chat with the many folks already building on Filecoin. If you have other ideas on how we can establish that there are lots of groups operating on the network, not just us, let us know and I'll see what we can do :)
The problem is, people have been trying to build this sort of system for many years, so any proposed system is likely to fail when put into operation & therefore has an enormous burden to prove they've overcome the many fatal flaws that existed in previous systems.
The block chain world is absolutely overrun with over-funded over-specced over-engineered boil the ocean projects that will never ship anything usable, solutions in search of problems, stuff that just plain doesn't work for fundamental math/algorithmic reasons, and of course a ton of outright scams. All of these are loud and lean heavy on the hype because they have nothing else. Just keep working and you will emerge once the dust settles. Not saying forget about marketing, but focus more on product.
* private encrypted backups
* public distributed files
* public encrypted distributed files
It’s contrary to parliamentary procedure and rules of debate to make a statement in order to back up your own point and not admit counterexamples and discussion of said statement and it’s implications as well as it’s intent and factual basis in the current reality we all share, regardless of which parts of it are presently up for debate.
You would have us not be informed about relevant contextual information regarding a point you yourself brought up, just so we don’t derail a discussion you yourself are replying to? We’re all valid commenters with valid comments provided we have something relevant to say.
Be that running an obscure website that gets flash traffic, or pushing out backups you hope you never need, until you need them all at once.
So the question is does your cloud provider end up charging you for your average traffic per year or your peak traffic, and that depends on base rates and overage rates. Even if protocols like this just exert downward pressure on those rate tables, we all win.
Unfortunately for the dream, centralisation offers efficiencies and reliability that are hard or impossible to match.
It may help drive their prices down though.
If it costs me $1 to store my stuff per month on S3, and you reduce that... so what? It's so cheap it's not going to help my wallet much. This isn't like going from $100 to $2- or even $50 to $20. It's going from like $10/year to... I dunno, $4. I might as well stay on AWS.
That leaves the enterprise market, which, naturally, loves S3 100x more than the random individual, because S3 is a solid enterprise choice, and will always have the enterprise advantage, by a crushing margin.
AWS is itself hard to compete with, but of all the services you could compete with on AWS, S3 is probably the worst. So you're going up against the worst of the worst, here. I can't say it's impossible, but it's like an extra double hard market to compete in, in an already tough market.
I recently had a RAID array rebuild fail and had to restore 10 TB from cloud backups. That would've cost me $900 on S3.
Arweave has developed a new type of blockchain based on Moore’s Law of the declining cost of data storage. Users pay upfront (one time payment) for a hundred years of storage at less than a cent per megabyte, and the interest that accrues will cover the dwindling storage cost forever. More than one million pieces of data are now stored on the permaweb...
Source: https://techcrunch.com/2020/03/05/coronavirus-censorship-arw...
The issue you're forgetting to mention is the fact that physical DVD drives won't be around for more than a few more years, you then need to make sure you have one in working order on a system that can use it
The basic reason why is because the protocol to request a file from Filecoin is a lot more complicated than simply fetching a URL, so IPFS can't simply sit atop, at least not without other changes.