The thing I never understood with private blockchain tech is that the "traditional" blockchain (i.e. Bitcoin) relies on proof of work, and the only way this is viable is to have tons of resources working on these proofs so that you don't get a 51% attack (there have even been a bunch of articles about how smaller coins actually are very susceptible to a 51% attack by a decently funded attacker).
For a private blockchain, though, it never made sense to me as to who would serve the role of the miners with sufficient incentive to prevent a nefarious attacker. If on the other hand you are in a system where the participants agree as to how they will trust each other, well then you'd be back to a situation where the byzantine model isn't really necessary and you can just go back to a cryptographically signed ledger a la the Quantum Ledger DB that AWS just announced.
Would really appreciate someone explaining this one to me!
Another issue against the DLT hype is that the blockchain does not provide security where 90% of the whole architecture is outside the blockchain, less if there are many few nodes.
Until we don't see a community working on adversarial attacks, we will not know if it is secure of not. There is concrete research about BFT since 1982.
I think it's very unlikely that any of the theoretical ideas behind Shasper are incorrect. Casper itself is simple and comes with a simple proof, and it's similar to an old algorithm by DLS [1], which also comes with a proof. Sharding does introduce some other machinery, like VDFs for randomness (specifically [2]), but that has been vetted by plenty of cryptographers such as Dan Boneh's group.
So even though there aren't any large-scale deployments of BFT algorithms yet, the approach is widely thought to be sound. I'm working on a blockchain based on sharded BFT, as is Ethereum, RapidChain, NEAR Protocol, and others.
It's always possible that our economic assumptions will fail, but they're not radically different from Bitcoin's. You can attack Bitcoin by buying 51% of all hash power, or you can attack a BFT system by buying 34% of all stake. Either way it comes down to making an assumption about the attacker's funding.
[1] https://groups.csail.mit.edu/tds/papers/Lynch/jacm88.pdf
When I read the updates about Casper[1] there are plenty of issues presented, so don't sure why you are saying Casper itself is simple.
Fine you are working in the "Sigma Network", we are working with one of the projects mentioned there.
And I think you're right -- adding Nakamoto consensus to private blockchains makes no sense, because by definition they don't have arbitrary validators. And without Nakamoto consensus, "blockchain" is just rebranding of old and boring tech.
A distributed merkle-chain database is still pretty innovative. A good example is git. I think what many private groups want is a "binary git for transaction data". Everybody can review their own copy of the shared chain with signing of new links in the chain.
Now that isn't really a full blockchain, but is a different kind of thing from a central RDMS.
The point being that all the hype about "blockchain everywhere" came about because of bitcoin. If people are now starting to refer to just the "hash of blocks with previous blocks" (i.e. the "Timestamp Server" section of the original bitcoin whitepaper) as blockchain withOUT the proof of work/stake part, well then you really are just rebranding old tech as blockchain, because it was the proof of work part that was really the new thing in bitcoin.
Proof of work is required for distributed consensus; without that you need a centralized repository like GitHub or Linus to bless the current state of the database?
With Git for code you don't really need consensus - I can have my fork and you can have yours.
It is not 'trustless', but it is decentralized. You're basically relying on a majority of participants to behave ethically and not collude. In a situation where collusion is impractical or has little reward, this can function fine.
An example of this is Blockstream's 'Liquid' crypto which is validated by crypto exchanges. The security model of this coin isn't great compared to Bitcoin, but it arguably has advantages over the simple custodian model.
If you're operating a legal business, a centralized database hosted by an industry mutual or regulator beats a blockchain.
If you're at the scale where broad co-ordination is a problem, you're at the scale where the big boys can arrange a meeting. Alternatively, if you're at a scale where someone can get everyone on a blockchain, you're at the scale to create an industry organization.
Get the actors together to standardize a format for the messages everyone's emitting, and then make some software (one shared implementation, many different ones, doesn't matter) to parse the log stream into a point-in-time representation you can load into an analytics tool.
Seems to me that that sort of technology would fit this use-case a lot more closely than an Actual Blockchain™ would.
That's the idea. Just add a proof of causality for the transactions that span more than one peer, and you've got a blockchain.
A blockchain is a chain of blocks. A block is a persistent record of all the information required for the consensus process, which is held onto by a node after the consensus process has completed for that node. A new blockchain node bootstrap-syncs to the network by just receiving blocks at random from peers and then evaluating the consensus rules against those blocks in order to decide what its deduced copy of the "chain" shall look like. And you can never throw those blocks away (just keeping the transaction log), either, because a new peer might want to bootstrap itself from you.
This is not how multi-master sync in e.g. etcd or Postgres works. In those, consensus is a process that happens between all the nodes registered to a given cluster at any point in time. Each consensus-step happens between a known, fixed set of peers (fixed for the duration of that step, that is), consuming a fixed set of data that must be the same on all peers (the database before the update), and producing a new artifact (the updated database) that should be identical on all peers. After the consensus step completes, the inputs required to re-evaluate that particular consensus step are discarded by all peers. You can't go back and "watch history" happen again. You can only know what you've got right now.
But, luckily, since what you've got right now is an append-only file, you can just read it and see everything that's happened historically. It's just a logical history, though, not the history of the consensus process itself.
New nodes in such a system don't re-evaluate history to "reach consensus." They just pick a bootstrap peer to trust and slave themselves to it, synchronizing until they're an exact mirror of it. But—because all the nodes in the cluster keep their state in lockstep under consensus anyway—every node that is "in consensus" is as good as any other node that is "in consensus" for bootstrapping from. (And if you happen to bootstrap from a node that is lying about being "in consensus", then you'll quickly find that you can't obey the consensus protocol with the rest of the cluster using the data you bootstrapped.)
A blockchain is a very specific kind of distributed database architecture. Just having a distributed append-only database does not automatically make something a blockchain. You can have distributed append-only databases that aren't blockchains.
(Heck, there's an even more trivial case: a distributed append-only database owned by one party. How do you build that? Just deploy a master and some read-replicas, and tell the master to be append-only by policy! It should be pretty obvious, I hope, that that is not a blockchain.)
Ok, it's not. It also does not have verifiable causality, because it's formed by a set of registries with no explicit relation between them.
Picture instead, something more like a document synced in realtime using Operational Transformations (via SubEthaEdit, Etherpad, Google Docs, etc.), purely peer-to-peer (so specifically like SubEthaEdit) and without any peer expected to retain any history of the OT sync process itself (so you can't "wind back" the document like you can on Google Docs.) Except, now, also picture that if you try to broadcast any OT events other than an insert OT, the other nodes will just ignore those OT events, causing your document to fall out of sync with everyone else's. So the document itself can only ever grow, and nothing can be redacted or changed once it has been inserted.
So now you've got this single document that everyone can "write" to (but never change anything that's already been "written.") Now just make everyone always put their new writes at the end, so they're not breaking up anyone else's message. Now you've got a durable, distributed message bus.
(If you've ever used a wiki Talk page—or the old original C2 wiki where every page was a Talk page—the rule there for posting a new "chat message" to a page, is that you should append them onto the end of the existing "chat messages" of everyone else. If you put it anywhere else, another editor will revert your edit. This rule by itself is enough to allow for a functional chat system/forum with a complete logical "conversation" history! Same idea here—just that it's the database-node software that's automatically "reverting" bad edits. In all other senses, it's just a regular distributed file.)
From my perspective, blockchains are suitable for two things: People who want to misbehave(therefore they don't have legal recourse, independent of the morality of the extralegal actions) and people at war with each other that still need to have a relationship with each other.
It's kind of the perfect technology for a collapsed civilization or the revolutionaries and criminals in the current world order.
So Wall Street banks?
Then you have a people & process problem, not a technical one. The Blockchain, A Real Database™ or even a CSV file won't solve your problems here.
Do you have an actual example?
But never-the-less I'd echo the interest in reading details of a successfully implemented use-case for public blockchain technology, beyond cryptocurrencies. Based on some of the comments here hinting that there might be some, and assuming they do indeed exist, I suspect that they are not solutions that could be achieved only with blockchain technology (like cryptocurrencies), or even solutions that are significantly better (in terms of cost, effort, time to market or some other metric) with blockchain technology, but simply solutions where blockchain has been made to work (i.e. other approaches could have been too and some of those may have been as good if not better).
[0] http://merltech.org/blockchain-for-international-development...
Is NIST correct?
Hasidim don't seem like they have that problem.
Technology can't solve this problem.
If multiple entities need to coordinate inside a mutually used product don't trust each other, then the problem is structural and needs to be solved with interpersonal, regulatory or political action.
If you're in a situation where cryptocurrency is the best solution for currency, then you're probably in a lawless wasteland with abjectly destructive governance systems.
Seems like in that circumstance moving is the right answer.
It also enables otherwise untrusted actors to make trustable claims, bypassing the gatekeepers that would otherwise have mediated those claims. One tangible example is ICOs bypassing not just regulators, but banks and the financial industry that would normally have had to underwrite them. Obviously this particular example has some serious kinks to work out, but the ability to bypass the corporate gatekeepers (not so much the regulators) has value, I think.
So I personally think DLT is going to be one of those multi-quadrillion dollar technologies that come around every few hundred years. The real benefit of DLT is that it enables new types of human relationships. So thinking about it in terms of what percentage of your existing databases should be replaced by it isn't going to give an especially impressive result, because by definition your existing databases are going to model your existing relationships.
Think about what percentage of the stuff in your house you would own without the invention of double entry accounting. Unless you happen to have some veggies from the farmer's market or a sweater one of your relatives knit you, the answer is probably 0.00%. It simply isn't possible to manufacture things like the iPhone without double entry accounting, because without double entry it would be impossible to form the sorts of human relationships needed to produce such a complex product. And if we asked someone to take a look at all the stuff in their house a couple hundred years from now, I'm guessing that a similar 0.00 percentage of the stuff they own will have not have been created as the result of DLT.
The fact is that once DLT becomes ubiquitous it will no longer be cost competitive to manufacture and distribute products using only the sorts of relationships and techniques enabled by double entry accounting. And if anyone even tries it's going to be like bringing a knife to a gunfight.
Can you share some examples of use cases where DLT is going to be transformative?
The tampering comes after the fact when a legal issue arises. The immutable nature of the ledger makes it an improvement over the past by preventing later tampering.
If you look at Bitcoin and the Proof of Work mining algorithm that was created to solve its BFT problem, what you have is a globally dispersed group of participants driving the cost of SHA2 hashing to as close to zero as possible. Electricity combined with computers can be converted to Bitcoin, which has value because a group of people believe it has value. You are optimizing everyone to solve that problem, and people are responding naturally to economic incentives.
I believe other problems can be reduced to a Proof of Work distributed problem to drive the cost to zero. LivePeer ( https://livepeer.org/ ) is a recently launched protocol built on Ethereum to drive down the cost of transcoding live video. This is a computationally intensive process full of proprietary technology and patents that forces video streamers to become beholden to centralized third-parties such as YouTube, Twitch, etc. LivePeer solves this problem by incentivizing global participants to commit transcoding resources to an open network, ultimately driving the cost of this to zero. This will reduce censorship and costs.
Beyond computationally intensive work and protocols, I believe that smart contracts will open up a range on novel use cases. For starters, pure digital assets will become a way to fund development of videogames. Free to play games like League of Legends and Fortnite sell in-game assets, but these are not unique - they sell as many copies as people buy. While this works for big players as a business model, I believe new, novel games will come from making game assets provably unique.
These are just a few examples, but many very smart, non-crazy non-scammy people are working in this industry and creating new technologies. It's sad that the fraud has poisoned the conversation so much, especially on tech places like HN.
Then after bitcoin had gone up the marketing guys saw visions of billions in free money and started saying blockchain, blockchain!
I've seen lots of vague "blockchain for supply chains is a great idea" pronouncements but I don't get how it would actually work.
Then as each head of lettuce is picked, it goes into a crate that's securely sealed and then signed with the farmer's private key in a way that originates that crate of produce on the ledger. When the farmer sells their produce to the middleman that transaction is also recorded, and so on, all the way to the end consumer who buys the produce in a grocery store or in a restaurant. (And consumers would just use their phones or credit cards for this, rather than using any sort of external fob.)
Then when the first person gets sick they report their illness as per usual. Nothing happens at this stage, because there's no way to narrow down what made the person sick. By by the time the second person gets sick (with E. coli of the same genetic signature), now you can find the furthest place back in the supply chain where both people's purchases intercept. So you can now see if e.g. the contamination came from a single farm, and if so only recall lettuce from that farm rather than all romaine lettuce produced worldwide.
Because the database is open anyone can download a copy, and there is no risk of a single entity imposing a 30% Apple tax on each head of lettuce or whatever. And each person benefits from participating, because it's a pareto improvement in terms of their profitability. (Now their products only get recalled when they are at fault, rather than their products getting recalled when anyone is at fault.)
Trust scales with something like Metcalfe's Law. E.g. a consortium of ten independent banks is probably 99% less likely to steal my money than just Wells Fargo. The idea that we need millions of independent entities to get substantially better security than the status quo is just propaganda that gets spread by Bitcoin maximalists.
There is a cost of duplication, but at the level of duplication you actually need the cost isn't that much compared to the benefit.
So to apply that, wouldn't we have to somehow register each individual head of lettuce with a tamperproof fingerprint? Just registering the bags the lettuce came in isn't good enough; the "farm" selling you their romaine might actually be selling you romaine from somewhere else that they've repackaged. (If that wasn't the case, we wouldn't be having this lettuce problem to start with.) And this is setting aside the possibility that lettuce might be leaved/shredded before packing and shipping, because now we have to put that tamperproof fingerprint on each and every leaf.
Oh, also there's the thing about all the leaves actually being in contact with one another as they're shipped, so by the time the consumer actually eats the leaf and gets sick, the best you could possibly do is say "we think it's from one of these packagers."
Maybe you're envisioning some other way entirely for Distributed Lettuce Technology* to solve this problem, I dunno. But I have trouble seeing it.
*I'm sorry, but cut me some slack, the joke is RIGHT THERE
Think about it from a game theory perspective. Under the status quo, each party in the supply chain maximizes their profit by lying about where their supplies came from. Whereas with DLT, each party maximizes their profit by being honest about where their supplies came from.
As an example, let's say you hijack you a truck and swap out expensive lettuce for cheap lettuce in a way that circumvents whatever tamper proofing technology has been applied. Stuff like this happens all the time currently, and is wildly profitable. (That's why if you go to a sushi restaurant and get a bunch of assorted sushi pieces or rolls, there is roughly a 0% chance that your meal will actually consist of what you ordered.)
With DLT though this is no longer profitable, because if you sell the cheap lettuce as expensive lettuce then you now have to sell the expensive lettuce as cheap lettuce, because each head or crate or whatever still needs to be traceable back to the source. This means that your total profit from the transaction is just whatever you would have made without cheating, plus your costs of hijacking the truck. So all in all, a substantial net loss.
It allows you to know the source, which enables you to make a data-informed decision about whether or not you trust the source.
As opposed to the current status quo, where you can never even know the source so trust doesn't even come into the picture.
You don't really need a distributed ledger for that,a centralised database run by an industry clearinghouse could serve the purpose just fine.
But maybe there are cases where a distributed ledger is easier or cheaper to establish than such a clearinghouse.