An overview of the Linux Foundation's open source approach to blockchain. LF is involved with lots of projects which aren't all explicitly Linux these days.
It could be changed to "The Linux Foundation's approach", sure, but it's very much a Linux Foundation project, and is developed and managed using the same template by which the Foundation manages the kernel project and others. We have some differences - there is no central kernel, but a family of different DLT systems within the Hyperledger community, reflecting geniunely different approaches. But other parts are similar, from technical governance to everything being under an open source license to legal compliance.
I did research Hyperledger Fabric and compared to other "Blockchain platforms". My conclusion was that it made a lot more sense to use Ethereum in most cases due to:
- Ethereum has a bigger community that is more open and inclusive. While Hyperledger is mainly an IBM and friends like project and a lot of decisions and information is kept inside. Goodluck getting things done without involving consultants from these "supporting" companies.
- Consensus algorithm is what makes a Blockchain a blockchain. Hyperledger says something like you can use whatever consensus mechanism you want but in practice there is not a real alternative to proof of work. Relying on a in public production battle tested algorithm will save a lot of effort and headache.
- crypto economic incentive is what makes a Blockchain a blockchain. Without having a coin to motivate participants in a balanced way to act in a specific desired way, there will not be any real advantages to use a blockchain platform and it is in most cases smarter to just use a distributed database. Hyperledger doesn't provide a practical way to create an incentive mechanism.
- The underlying philosophy of hyperledger is around "federated networks" which means that it is trying to create a controlled environment where transactions take place between authorized participants. This leads to centralization by authority which leads to data immutability according to the degree you trust this entity. This defeats mainly the purpose of Blockchain to create a trustless environment of immutable data record.
> Hyperledger says something like you can use whatever consensus mechanism you want but in practice there is not a real alternative to proof of work
With the latest crop of PBFT consensus algorithms (notably Tendermint/Cosmos), proof-of-stake is shaping up to be a real alternative. Though I'm a bit worried about the inherent complexity.
This is actually worse for proof-of-work chains, though. The cost of a 50% attack on most cryptocurrencies (including bitcoin!) is much less than half the market cap. The main thing which keeps such attacks from happening is that they are hard enough to profit from (it's quite hard to double-spend enough) while being expensive enough to that such an attack is a very risky and almost certainly unprofitable endeavour.
Not only is it hard to double spend, but if your double spend is found, the liquidation value is likely to plummet from under you in or before the process of attempting to realize the theft.
> Consensus algorithm is what makes a Blockchain a blockchain. Hyperledger says something like you can use whatever consensus mechanism you want but in practice there is not a real alternative to proof of work.
For public use maybe but for commercial/private use cases it can often be assumed that all parties are trusted in some manner. I think their approach to different consensus algorithms makes sense when you're solving for use cases beyond financial transfers between untrusted parties.
If parties are trusted, don't bother with Blockchain. Huge added complexity (which brings security issues) without any benefit. Use for example git or a conventional database.
Ever try to share a conventional database between multiple different companies or organizations? It's not practical.
It seems more complicated than going with a blockchain solution (or will be as blockchain deployment tools mature).
The adage that 'if parties are trusted, than go with a conventional DB' doesn't hold water anymore. If you need to share a database with other companies, it inevitably leads to contracting a third party solution to manage the data entry between all parties, which means you now have to trust this one entity and there's a single point of failure.
Hyperledger, blockchain, and the like eliminate the need for third party management. A few engineers at each party needs to set up the new service, and now you don't have to worry about central points of failure, you have built in redundancy, and you save money by not having to use a data management service that exists to make money off of you.
> Ever try to share a conventional database between multiple different companies or organizations? It's not practical.
Not practical how? It seems like this criticism can only be applied to usability, but that's not a technical shortcoming.
> If you need to share a database with other companies, it inevitably leads to contracting a third party solution to manage the data entry between all parties, which means you now have to trust this one entity and there's a single point of failure.
This criticism seems a bit underspecified. While it's true that having a single entity to handle data entry results in a single point of failure, but what's the failure model? Do we assume that this single entity might be malicious? Or are we assuming that the entity handling data entry isn't familiar with the types of failures they may encounter, and thus hasn't taken steps to mitigate the effects of those failures? Both of those scenarios, given the context of sharing data among organizations, seems unrealistic.
> Hyperledger, blockchain, and the like eliminate the need for third party management. A few engineers at each party needs to set up the new service, and now you don't have to worry about central points of failure, you have built in redundancy, and you save money by not having to use a data management service that exists to make money off of you.
The claims you make here are correct: (1) the consensus protocol provides a decent mechanism for writing to the database, (2) redundancy insulates against loss of data, and (3) the open nature of the protocol means you don't have to use a for-profit service. However, there are far more efficient ways to achieve these things without relying on blockchains. If everyone that could write to the database is known, and what you want is simply to guarantee the integrity of the database, there are better data structures than a linear chain of hashed, batched events (which is all that a blockchain is, at its core).
Hi there - Richard Brown, CTO of R3 here. We support and maintain Corda, an open source permissioned blockchain (that is not currently part of Hyperledger). Brian has done a good job of putting Hyperledger's case and I agree with him in a lot of his responses. But I wonder if there's a broader point to be made too.
One of the key questions being asked here, it seems, is: what problem (if any!) do permissioned blockchains solve? Put another way: if you're not trying to build a censorship-resistant, decentralised payments network, for example, why do you need any of this stuff?
To be frank, where I think _some_ private blockchain platforms have gone wrong is that they have never satisfactorily answered this question. Instead, there has been a leap of magical thinking from "Bitcoin is amazing" to "let's cargo-cult some of the same ideas and use them to solve (unspecified) problems in business." This is not a critique of Hyperledger btw... It's true I've made some high-profile critiques of Hyperledger Fabric's design choices in the past but that isn't my point here. Indeed, I'd hope most of the Hyperledger community would agree with this post.
The thought process we went through when we designed Corda at its most simple was: how can we build a system that allows parties who wish to transact (but who don't fully trust each other) to maintain accurate shared records of their dealings with each other without reliance on a third party?
I know that _sounds_ either vacuous or trivial but I really don't believe it is. I still don't think I've written a totally satisfactory description of my vision but the article linked to below probably comes closest (scroll quickly past the shilling I do for our commercial distribution at the top and tail... the meat is in the middle!)
tl:dr of that piece: firms who transact with each other in the real world - with paper and phone calls and faxes - don't need a centralised third party toll-taker to manage and record their transactions for them. So why is our instinctive response to introduce one when those same firms decide to transact electronically? Whether it's a centralised database (who runs it? what do they charge? what power do they have? who holds them accountable? what happens if it goes down?) or a formally constituted infrastructure firm like in the financial markets, adding a centralised entity where previously there were none feels like inadequacies in technology forcing industries to change their market structures. Tail wagging the dog.
And yet, without such a thing, you end up in a total mess with each firm holding and maintaining their own records, which are invariably out of sync (out of consensus) with their counterparts.
A key problem we try to solve with Corda is to enable trading partners to connect their applications to each other in a way that ensures they're in sync at all times. And this requires far more than just conveying data from one side to the other... it's ensuring it's interpreted and processed in the same way - deterministically, in the same order. Indeed, the abstract for the Corda technical whitepaper actually calls it a "decentralised database" (note: not "distributed database").
This takes you into a completely different design space where many (but, crucially, not all) of the same principles underpinning public blockchains are needed... but where requirements such as transaction finality, strong identity, interop/integration with existing systems, reuse of existing codebases, developer productivity, ability to deploy behind a firewall yet connect to peers across the internet, etc., come into play.
The result is that platforms like Corda look similar to public blockchains on one level (chains of transactions, deterministic execution of business logic, digital signing of transactions, decentralised consensus algorithm to confirm and order transactions) but also very different (no crypto economic incentive, settlement finality, runs on the JVM so the world's 10m Java developers can use it easily and so on)
Parties do _not_ have to trust each other to work in a permissioned or consortium block chain. They just don't. They do have to be non-anonymous to each other, and they might have to sign up to a legal agreement between each other (brokered by the governing entity convening the ledger, who issues certs to organizations so they can participate). But they don't have to be trusted. All you need is intrinsic interest in being on the ledger - some kind of pain (be it staked value, getting kicked out of a market, etc) if they are found to be a bad actor, which would be pretty obvious, and many bad actions can be prevented in the first place (e.g. prevention of double spend).
"While Hyperledger is mainly an IBM and friends like project"
IBM must have a lot of friends then, including direct competitors like Oracle and SAP, and lots of small startups. :) IBM is one of over 250 members in the consortium, but in the developer community (where it really counts), they are a substantial contributor - I have no idea how many FTEs but there are dozens of IBM developer names in the changelogs for Fabric, Composer, and others. But there are hundreds of other developers contributing to the 10 different project at Hyperledger, and even on Fabric, they are less than 50% of the contributor base by individual and by pull requests. The development is all done publicly (I challenge the "lot of decisions and information is kept inside" claim) and who you work for does not matter, so it's really on the other companies in this space (large and small) to increase their contribution levels, not on IBM's shoulders to decrease theirs, if this is an important metric.
"in practice there is not a real alternative to proof of work"
Come back when you've learned something about how blockchains work somewhere other than with cryptocurrencies. There are many other effective consensus mechanisms out there, especially when you're not trying to solve the anonymous participation goal - and much more energy efficient, and with better speed and finality to boot.
"crypto economic incentive is what makes a Blockchain a blockchain"
No. Perhaps this is a semantics game, as lots of people pretend to be authoritative about this, but there are plenty of intrinsic motivations to developing a common system of record amongst a population of rivalrous nodes, even amongst parties who don't trust each other. If ten banks want to maintain a shared DLT to exchange bank wires, they don't need to pay each other a token to record a transaction. They just don't - the costs of running such an infrastructure are de minimus compared to all the other costs in adoption.
"The underlying philosophy of hyperledger is around federated networks"
False. Hyperledger, as a project and as a community, is a place for the collaborative software development around open source blockchain technologies. There are five different frameworks, including Fabric, but including others focused on different approaches. All projects have focused on non-POW consensus mechanisms, and most are an implementation of consortium models, what's sometimes called "permissioned" blockchains - pick your metaphor, but a consortium of entities governed by a lightweight governing mechanism, or be it a referee on a football field, is a much closer conceptual model than a "federated" network, for which there are already plenty of options.
Again, please do research, talk to people, if you have questions or concerns air them with the project, but please don't just spout what you've heard, or made up after you saw the announcement press release in December of 2015 and paid no further attention to.
This is the "not too inclusive" attitude that I referenced. It is not only rude to provide your opinion about a subject with this voice of tone. It is also problematic because it doesn't make it possible to have a positive discussion and learn from each other.
A lot of people on HN do exactly understand the nuanced differences between open source projects and communities. It is I guess enough to mention that Hyperledger is a IBM kind of enterprisey setting.
I will not try to explain in depth why proof of work is really difficult to substitute. But in summary: you will need to create an algorithm that prevents sybil attacks + creates incentives for participants to keep the network up + mechanism for byzatine consensus. Anonymous participation goal is not a goal on itself but a prerequisite for decentralization. Otherwise someone needs to manage and control who is eligible to access the network...
You are really wrong about incentive motives too. The example involving wire transfers is exactly the point. Blockchain is not about a record in a database of a money amount, it is directly the value representation. This creates a big difference. I'm sure you will understand this if you give it a more open minded thought.
I did a lot of research and talked to enough people. I will not respond to more ad hominem reactions.
Where I said above that people need to do research, it wasn't necessarily directed at you. Perhaps you have, though even the above statements suggest you've not looking at this the same way many others do, so we can likely just punt with we are trying to solve different problems. And at the very least, have different definitions of decentralized.
No, I think he is smart enough to avoid making statements around depreciating CAP theorem. He actually emphasizes repeatedly the trilemma of blockchain networks with the factors scalability, security and decentralization and talking about that this would bending physics if it could be realized.
I think it is very important for Ethereum to research other consensus mechanism as an alternative for Pow. Otherwise it will be the bottleneck if it wants to realize the vision around "worldwide computer".
Vitalik has statements related to usefulness of use cases around private/permissioned blockchain networks. I think that will be only useful if combined with a layer 1 network which is public.
Was good to work with HLF but the ops background required to run it in production was staggering: Replicated Kafka + ZK backing the consensus algorithm. I put it all in Kubernetes with StatefulSets. Even with my background running those systems it felt unreasonable. I quit that job in Nov so I don’t know if the situation has improved since then. I liked the architecture and the team was responsive and put care into docs. Would use again.
Thank you! Folks are working hard to get deployment and management via Kube (and Helm Charts, etc) to be first-class supported and consistent. It's about where one would expect a 1.x level project to be, which means improvements like you point out are needed. Lots has happened since November. :) Hope you consider coming back and contributing.
we did a long evaluation of HyperLedger Fabric (0.6 - 1.x) for a project at my company.
We really wanted it to work since the chain code model was cool however we ran into stumbling block after stumbling block and in the end ditched it for something else.
List of pain points (no idea if any have been solved since we gave up on it)
1) Distributing chaincode updates to participants was an exercise in coordination that is unscalable
2) Kafka as consensus. At the time you had to use an infinite kafka queue since pruning it would throw fabric nodes into a panic and they'd drop off the network. We thought for sure we were doing something wrong but the need for an infinite queue was verified by someone at IBM
3) Community. The community around HL F was mostly IBM people or folks off doing their own weird thing and adding features or pull requests that were bizarre (a custom fork of postgres to act as state dB??). With that said there were a handful of _very_ helpful people there just never was a good mass of them
4) Performance. We wanted to run a private blockchain but amongst nodes distributed across the globe in whatever datacenter they chose. Running fabic in a truly distributed way just kills performance. There is a lot of cross talk and waiting for the ordering service to get sorted
5) Not really a blockchain. Fabric relied on the centralized ordering service and each node would contain its own state. If the ordering service pruned logs (which I guess is a feature now) then the distributed nodes could have ledgers that have content the central order does not know about. If that node wanted to rebuild it would need to rebuild from other nodes without a way to truly verify the content of the chain. This may have been solved but it was a gaping wound of a question when we raised it
There are others, in the end it was just way to much work and we moved off to a more popular blockchain and brought our proof of concept to feature parity and beyond in just two weeks.
We reached out to IBM for help several times and the answer was mostly "just buy bluemix" or "pay us and we'll implement it for you"
To use HyperLedger Fabric in any sense of scale and operational readiness you must run it all within one data center. If you have external participants in your network they must also be customers of that service or you can just run it all yourself in y our own DC for your own needs.
IF you go that route, it begs the question of "why bother?" You're putting everything in the hands of one entity and then adding in a ton of complexity and slowness. Just use a database or something right?
We had a large presentation from IBM about hyperledger (I work for a healthcare company) and it seemed like something our industry could really leverage.
I was put on a team to research using it on an upcoming project. Once we started digging in and asking questions, it became apparent very quickly IBM wants to control everything from top to bottom. It's on their network, they build it, they maintain it, and there's not a lot of options outside of what they've already built.
We had some very specific functionality we needed and were told by the IBM team it wouldn't be a problem. When we got down to delivery and talking actuals, they started to balk if they could even deliver what we really needed. After trying to nail several things down and not getting anywhere, we pivoted and are currently looking at other blockchain platforms.
After reading your response, it sounds like we may have dodged a bullet and made the right choice.
IBM is not the only company that can deliver a network built using Hyperledger Fabric, or any of the other frameworks. Did you take a look at Sawtooth, or Iroha? Did your devs engage with the actual Hyperledger community, or only through IBM? There are several healthcare companies (Change Healthcare is one, managing insurance claims) who have engaged directly and are in production.
IF you go that route, it begs the question of "why bother?" You're putting everything in the hands of one entity and then adding in a ton of complexity and slowness. Just use a database or something right?
So what's the justification your company used for a private blockchain in lieu of a database?
There are many, but one of the points above mentions that participants are distributed globally and that saying that everyone must use the same platform was a nonstarter. I'll add that trust is not necessarily granted by inclusion. The needs for privacy do not have to include explicit trust and that a blockchain still solves very specific problems in this scenario.
I am looking for a library that provides a blockchain, without needing a consensus mechanism. 50% of the the need is to gain experience and 50% is for a couple of light weight projects. I was hoping this was something I could try out but from the comments it looks like not. I use nodejs so extra points for that, but I am open to other alternatives. An example project would be to set up a local swap market, where users can swap appliances for swap-coins, then spend those for other appliances at a later time.
IMHO a "blockchain without a consensus mechanism" is tantamount to a security door without a lock. The whole point of "blockchain" being that all transactions on it are correct and verifiable via a consensus mechanism, i.e. a distributed, non-centralized trust mechanism where you don't have to trust one individual or company to get it right.
If you really want to record transactions without a consensus mechanism, use a database.
The only mildly sophisticated part of the blockchain itself is the Merkle tree. You can find lots of Node libraries for that.
For every new record, add it to the tree. When your tree gets big enough (i.e. hits maximum block size), sign it along with the hash of the last one, and then start a new Merkle tree. Congrats, you have a blockchain.
A blockchain "without a consensus mechanism" is git: every commit has at least one parent. There is no consensus mechanism, as everybody can commit "to the blockchain".
If you want a good ready to use development tool look at https://truffleframework.com/ - it's literally a wrapper around ethereum-js along with some friendly programming tools for smart contracts, etc.
There is a lot of misinformation circulating in the comments here that I'll be addressing as replies to some of these threads, though inevitably won't be able to get to each one. I'd ask each of you to not just repeat misinformation you've heard from others, but do first-hand research by going to the Hyperledger website, wiki, discussion forums, or other places and learn about the tech before taking the snarky way out. Thanks,
Love the format and illustration, though I feel having a high-level comparison between different Hyperledger projects (especially Fabric versus Sawtooth) is probably important here. Happy to contribute/collaborate on this.
Can someone give me a concrete example of a situation in which a private blockchain makes sense for a company or industry instead of a shared, permissioned database?
