Hyperledger – Linux’s open source approach to blockchain building
litepaper.com
litepaper.com
The LF is a huge disappointment to me...just selling the linux brand to whoever wants it.
- Ethereum has a bigger community that is more open and inclusive. While Hyperledger is mainly an IBM and friends like project and a lot of decisions and information is kept inside. Goodluck getting things done without involving consultants from these "supporting" companies.
- Consensus algorithm is what makes a Blockchain a blockchain. Hyperledger says something like you can use whatever consensus mechanism you want but in practice there is not a real alternative to proof of work. Relying on a in public production battle tested algorithm will save a lot of effort and headache.
- crypto economic incentive is what makes a Blockchain a blockchain. Without having a coin to motivate participants in a balanced way to act in a specific desired way, there will not be any real advantages to use a blockchain platform and it is in most cases smarter to just use a distributed database. Hyperledger doesn't provide a practical way to create an incentive mechanism.
- The underlying philosophy of hyperledger is around "federated networks" which means that it is trying to create a controlled environment where transactions take place between authorized participants. This leads to centralization by authority which leads to data immutability according to the degree you trust this entity. This defeats mainly the purpose of Blockchain to create a trustless environment of immutable data record.
With the latest crop of PBFT consensus algorithms (notably Tendermint/Cosmos), proof-of-stake is shaping up to be a real alternative. Though I'm a bit worried about the inherent complexity.
It's already being done to smaller proof of work chains.
So in this regard you have similar issues.
For public use maybe but for commercial/private use cases it can often be assumed that all parties are trusted in some manner. I think their approach to different consensus algorithms makes sense when you're solving for use cases beyond financial transfers between untrusted parties.
The adage that 'if parties are trusted, than go with a conventional DB' doesn't hold water anymore. If you need to share a database with other companies, it inevitably leads to contracting a third party solution to manage the data entry between all parties, which means you now have to trust this one entity and there's a single point of failure.
Hyperledger, blockchain, and the like eliminate the need for third party management. A few engineers at each party needs to set up the new service, and now you don't have to worry about central points of failure, you have built in redundancy, and you save money by not having to use a data management service that exists to make money off of you.
Not practical how? It seems like this criticism can only be applied to usability, but that's not a technical shortcoming.
> If you need to share a database with other companies, it inevitably leads to contracting a third party solution to manage the data entry between all parties, which means you now have to trust this one entity and there's a single point of failure.
This criticism seems a bit underspecified. While it's true that having a single entity to handle data entry results in a single point of failure, but what's the failure model? Do we assume that this single entity might be malicious? Or are we assuming that the entity handling data entry isn't familiar with the types of failures they may encounter, and thus hasn't taken steps to mitigate the effects of those failures? Both of those scenarios, given the context of sharing data among organizations, seems unrealistic.
> Hyperledger, blockchain, and the like eliminate the need for third party management. A few engineers at each party needs to set up the new service, and now you don't have to worry about central points of failure, you have built in redundancy, and you save money by not having to use a data management service that exists to make money off of you.
The claims you make here are correct: (1) the consensus protocol provides a decent mechanism for writing to the database, (2) redundancy insulates against loss of data, and (3) the open nature of the protocol means you don't have to use a for-profit service. However, there are far more efficient ways to achieve these things without relying on blockchains. If everyone that could write to the database is known, and what you want is simply to guarantee the integrity of the database, there are better data structures than a linear chain of hashed, batched events (which is all that a blockchain is, at its core).
One of the key questions being asked here, it seems, is: what problem (if any!) do permissioned blockchains solve? Put another way: if you're not trying to build a censorship-resistant, decentralised payments network, for example, why do you need any of this stuff?
To be frank, where I think _some_ private blockchain platforms have gone wrong is that they have never satisfactorily answered this question. Instead, there has been a leap of magical thinking from "Bitcoin is amazing" to "let's cargo-cult some of the same ideas and use them to solve (unspecified) problems in business." This is not a critique of Hyperledger btw... It's true I've made some high-profile critiques of Hyperledger Fabric's design choices in the past but that isn't my point here. Indeed, I'd hope most of the Hyperledger community would agree with this post.
The thought process we went through when we designed Corda at its most simple was: how can we build a system that allows parties who wish to transact (but who don't fully trust each other) to maintain accurate shared records of their dealings with each other without reliance on a third party?
I know that _sounds_ either vacuous or trivial but I really don't believe it is. I still don't think I've written a totally satisfactory description of my vision but the article linked to below probably comes closest (scroll quickly past the shilling I do for our commercial distribution at the top and tail... the meat is in the middle!)
https://medium.com/corda/markets-are-decentralised-and-the-s...
tl:dr of that piece: firms who transact with each other in the real world - with paper and phone calls and faxes - don't need a centralised third party toll-taker to manage and record their transactions for them. So why is our instinctive response to introduce one when those same firms decide to transact electronically? Whether it's a centralised database (who runs it? what do they charge? what power do they have? who holds them accountable? what happens if it goes down?) or a formally constituted infrastructure firm like in the financial markets, adding a centralised entity where previously there were none feels like inadequacies in technology forcing industries to change their market structures. Tail wagging the dog.
And yet, without such a thing, you end up in a total mess with each firm holding and maintaining their own records, which are invariably out of sync (out of consensus) with their counterparts.
A key problem we try to solve with Corda is to enable trading partners to connect their applications to each other in a way that ensures they're in sync at all times. And this requires far more than just conveying data from one side to the other... it's ensuring it's interpreted and processed in the same way - deterministically, in the same order. Indeed, the abstract for the Corda technical whitepaper actually calls it a "decentralised database" (note: not "distributed database").
This takes you into a completely different design space where many (but, crucially, not all) of the same principles underpinning public blockchains are needed... but where requirements such as transaction finality, strong identity, interop/integration with existing systems, reuse of existing codebases, developer productivity, ability to deploy behind a firewall yet connect to peers across the internet, etc., come into play.
The result is that platforms like Corda look similar to public blockchains on one level (chains of transactions, deterministic execution of business logic, digital signing of transactions, decentralised consensus algorithm to confirm and order transactions) but also very different (no crypto economic incentive, settlement finality, runs on the JVM so the world's 10m Java developers can use it easily and so on)
I think it is very important for Ethereum to research other consensus mechanism as an alternative for Pow. Otherwise it will be the bottleneck if it wants to realize the vision around "worldwide computer".
Vitalik has statements related to usefulness of use cases around private/permissioned blockchain networks. I think that will be only useful if combined with a layer 1 network which is public.
IBM must have a lot of friends then, including direct competitors like Oracle and SAP, and lots of small startups. :) IBM is one of over 250 members in the consortium, but in the developer community (where it really counts), they are a substantial contributor - I have no idea how many FTEs but there are dozens of IBM developer names in the changelogs for Fabric, Composer, and others. But there are hundreds of other developers contributing to the 10 different project at Hyperledger, and even on Fabric, they are less than 50% of the contributor base by individual and by pull requests. The development is all done publicly (I challenge the "lot of decisions and information is kept inside" claim) and who you work for does not matter, so it's really on the other companies in this space (large and small) to increase their contribution levels, not on IBM's shoulders to decrease theirs, if this is an important metric.
"in practice there is not a real alternative to proof of work"
Come back when you've learned something about how blockchains work somewhere other than with cryptocurrencies. There are many other effective consensus mechanisms out there, especially when you're not trying to solve the anonymous participation goal - and much more energy efficient, and with better speed and finality to boot.
"crypto economic incentive is what makes a Blockchain a blockchain"
No. Perhaps this is a semantics game, as lots of people pretend to be authoritative about this, but there are plenty of intrinsic motivations to developing a common system of record amongst a population of rivalrous nodes, even amongst parties who don't trust each other. If ten banks want to maintain a shared DLT to exchange bank wires, they don't need to pay each other a token to record a transaction. They just don't - the costs of running such an infrastructure are de minimus compared to all the other costs in adoption.
"The underlying philosophy of hyperledger is around federated networks"
False. Hyperledger, as a project and as a community, is a place for the collaborative software development around open source blockchain technologies. There are five different frameworks, including Fabric, but including others focused on different approaches. All projects have focused on non-POW consensus mechanisms, and most are an implementation of consortium models, what's sometimes called "permissioned" blockchains - pick your metaphor, but a consortium of entities governed by a lightweight governing mechanism, or be it a referee on a football field, is a much closer conceptual model than a "federated" network, for which there are already plenty of options.
Again, please do research, talk to people, if you have questions or concerns air them with the project, but please don't just spout what you've heard, or made up after you saw the announcement press release in December of 2015 and paid no further attention to.
A lot of people on HN do exactly understand the nuanced differences between open source projects and communities. It is I guess enough to mention that Hyperledger is a IBM kind of enterprisey setting.
I will not try to explain in depth why proof of work is really difficult to substitute. But in summary: you will need to create an algorithm that prevents sybil attacks + creates incentives for participants to keep the network up + mechanism for byzatine consensus. Anonymous participation goal is not a goal on itself but a prerequisite for decentralization. Otherwise someone needs to manage and control who is eligible to access the network...
You are really wrong about incentive motives too. The example involving wire transfers is exactly the point. Blockchain is not about a record in a database of a money amount, it is directly the value representation. This creates a big difference. I'm sure you will understand this if you give it a more open minded thought.
I did a lot of research and talked to enough people. I will not respond to more ad hominem reactions.
We really wanted it to work since the chain code model was cool however we ran into stumbling block after stumbling block and in the end ditched it for something else.
List of pain points (no idea if any have been solved since we gave up on it)
1) Distributing chaincode updates to participants was an exercise in coordination that is unscalable
2) Kafka as consensus. At the time you had to use an infinite kafka queue since pruning it would throw fabric nodes into a panic and they'd drop off the network. We thought for sure we were doing something wrong but the need for an infinite queue was verified by someone at IBM
3) Community. The community around HL F was mostly IBM people or folks off doing their own weird thing and adding features or pull requests that were bizarre (a custom fork of postgres to act as state dB??). With that said there were a handful of _very_ helpful people there just never was a good mass of them
4) Performance. We wanted to run a private blockchain but amongst nodes distributed across the globe in whatever datacenter they chose. Running fabic in a truly distributed way just kills performance. There is a lot of cross talk and waiting for the ordering service to get sorted
5) Not really a blockchain. Fabric relied on the centralized ordering service and each node would contain its own state. If the ordering service pruned logs (which I guess is a feature now) then the distributed nodes could have ledgers that have content the central order does not know about. If that node wanted to rebuild it would need to rebuild from other nodes without a way to truly verify the content of the chain. This may have been solved but it was a gaping wound of a question when we raised it
There are others, in the end it was just way to much work and we moved off to a more popular blockchain and brought our proof of concept to feature parity and beyond in just two weeks.
We reached out to IBM for help several times and the answer was mostly "just buy bluemix" or "pay us and we'll implement it for you"
To use HyperLedger Fabric in any sense of scale and operational readiness you must run it all within one data center. If you have external participants in your network they must also be customers of that service or you can just run it all yourself in y our own DC for your own needs.
IF you go that route, it begs the question of "why bother?" You're putting everything in the hands of one entity and then adding in a ton of complexity and slowness. Just use a database or something right?
*edit format
I was put on a team to research using it on an upcoming project. Once we started digging in and asking questions, it became apparent very quickly IBM wants to control everything from top to bottom. It's on their network, they build it, they maintain it, and there's not a lot of options outside of what they've already built.
We had some very specific functionality we needed and were told by the IBM team it wouldn't be a problem. When we got down to delivery and talking actuals, they started to balk if they could even deliver what we really needed. After trying to nail several things down and not getting anywhere, we pivoted and are currently looking at other blockchain platforms.
After reading your response, it sounds like we may have dodged a bullet and made the right choice.
So what's the justification your company used for a private blockchain in lieu of a database?
If you really want to record transactions without a consensus mechanism, use a database.
For every new record, add it to the tree. When your tree gets big enough (i.e. hits maximum block size), sign it along with the hash of the last one, and then start a new Merkle tree. Congrats, you have a blockchain.
If you want a good ready to use development tool look at https://truffleframework.com/ - it's literally a wrapper around ethereum-js along with some friendly programming tools for smart contracts, etc.
Brian Behlendorf, Exec Director, Hyperledger
Hyperledger Fabric: A Distributed Operating System for Permissioned Blockchains