The Imminent Decentralized Computing Revolution
blogs.wsj.com
blogs.wsj.com
Nice overview, but something that bugs me whenever someone talks about Bitcoin (particularly some VCs): they talk about the blockchain independently, as a "technology," without recognizing that, in fact, it's as much a triumph of technology as it is a triumph of clever incentives. Small semantic point in this article, but more broadly applicable when folks say "they don't love Bitcoin, but they love the 'blockchain technology.'"
Spoiler alert: the blockchain is both reputable and trustless because there are thousands of miners securing the network by hashing the night away. And why do they do it? Because of the unit of account they generate by participating.
Whenever I hear someone talk about how they love "the technology of the blockchain" without explicitly mentioning the incentives at play, I automatically become suspect of whether they actually understand what's going on. (I'm looking at you, venture capitalists) Less of a problem with this article, but I think it's worth highlighting all the same.
Also, isn't blockchain style computing obscenely inefficient? It would be great if someone well versed in these matters could weigh in.
If you were to run an organization/bank/Paypal clone that maintains a centralized ledger, it would be just as expensive (if not more), because you'd have to hire thousands of employees, compliance officers, lawyers etc. You probably see what I'm getting at.
There's always going to be a price for maintaining accurate accounting ledgers. The only questions is where those expenses are going to be pushed and whether it's going to be your payroll or your power bill.
People just think that way because so far mining is the only way it's being done with any level of security. There are a couple of algorithms based on academic research that are being implemented now, and the public hasn't seen it yet.
The event I was referencing turns out to have taken place in Denmark, not Holland, and was auctioning sugar beets, not tulips. They are using secure multiparty computation algorithms to auction sugar beet growing rights to farmers.
https://www.cs.purdue.edu/homes/aliaga/cs197-10/papers/boget...
The paper shows a practical use of secret sharing and secure multiparty computation to discover the market price of a commodity. Not sure how secure it is, but it's interesting.
It's not the same thing as securing consensus of a blockchain ledger. In the paper they have over a thousand participants and 3 (or some small number of) servers. In the blockchain problem, each participant runs his own server. The fault tolerance model is different, and also the blockchain requires continuous consensus.
Proof-of-work is expensive and will ultimately be replaced with better consensus schemes that require no mining, namely those that only rely on cryptographic signatures, a good balance of incentives and penalties, and an underlying byzantine consensus algorithm.
Disclaimer: I'm one of a handful of people working on new consensus algorithms.
Are there any (serious question)?
Up to now, I've pretty much scoffed at this stance. The real breakthrough here is Bitcoin, or so I've always thought.
However, I've recently come across a project that made me realize that the blockchain technology may indeed bring about other revolutionary changes in addition to Bitcoin itself. One in particular is assembly.com and Coin, the blockchain-driven ownership records that formalize projects hatched under the assembly.com organizational framework. I'm most familiar with Assembly, but I wouldn't be surprised if other similar projects are actively underway.
If successful, assembly.com's approach to organizations has the potential to change the way we form companies, for the better. If you're interested in big societal changes brewing around us, I'd suggest going to take a look at how assembly.com works with the technologies that enable this type of organization (git and Github, in particular, in addition to the blockchain). If you've ever read the books of Visa-founder Dee Hock and his ideas about chaordic organizations, you'll immediately recognize this phenomenon. It's pretty fascinating, and as soon as I finish my current contract I intend to start working on an Assembly project.
But as for the power driving the blockchain technology (and thus enabling all of this), I agree completely that this is indeed Bitcoin and its underlying incentives. Pretty amazing stuff...
I think startups in general are a good analogy for this. Sure, it's more fruitful in terms of percentage of equity to start your own startup than to join one, but it's immensely difficult to get that company you own 100% of to become something valuable.
For a product to succeed on Assembly, it's often really important for a strong visionary to play an active day-to-day role. That initial ownership makes the whole thing more fruitful for them, but only in the cases where they do lots of heavy lifting to make something succeed.
If you throw a random idea up on Assembly and someone else in the community shows up and ships that idea and makes it into a successful business, then you'd have a fortunate outcome without much work, but that's not an expected outcome.
edit: as eric_bullington pointed out (and I should have), it's important to note that the ownership you have over a product you start is only ownership of that product. If that product never ships, the ownership doesn't translate to the success of other products on Assembly.
So, you can get 100% ownership of a thing that is just an idea with a few clicks, or you can jump in on a product like Coderwall that makes $20,000/month, and any work you do will earn you a portion of that revenue for as long as Coderwall is around. Both of these avenues are fun. :)
It's a high-tech, blockchain version of a partnership. What I think is so innovative is the way this has the potential to combine kickstarters with traditional business partnerships, with the blockchain to formalize the relationship.
It's been pretty fascinating watching new projects start up there and gain steam.
Bitcoin mining is now less distributed than steel manufacturing. Home mining is now insignificant.
It also has a problem of transparency in this respect because you cannot know whether two miners are actually just one miner manning two machines.
1. Mesh Networks: FireChat isn't a mesh network. There is a long thread on /r/Futurology about it, with lots of good posts describing what it is and the problems with implementing real mesh networks (https://www.reddit.com/r/Futurology/comments/2hvtoh/over_100...).
The article espouses all the benefits of mesh networks and glosses over the gritty facts of implementation that can't necessarily be solved by simply throwing more software at the problem.
2. The BlockChain: Yes, this is a massively distributed system. But the fact is it relies on the existence of a fast, reliable internet that so happens to be immensely centralised.
A rant I often like to have which makes me unpopular amongst the proponents of so called p2p systems is that the bootstrap of all of these so called decentralised systems is completely centralised. Bittorrent has trackers, Bitcoin uses DNS seeds to find peers, Tor uses directory servers and most others simply hardcode some URL list of peers into their source. If you can attack the bootstrap you can split these so called decentralised networks quite easily. Sybil attack is a common vector here.
The article also glosses over the fact that all the things which have driven bitcoin adoption and popularity (Blockchain.io, Mt Gox, BitStamp, BitPay, etc) are all centralised services which require the user to essentially give up any of the supposed benefits of decentralisation. Even in the case of mining, how many run their own bitcoind? 99% of miners run in a pool, another form of centralisation.
3. Autonomous agents: I can't even...
> But history has proven that the centralized model is flawed and inefficient.
History shows the exact opposite. For example, at one point every business above a certain size had its own electric generator, and developed their own electricity on-site. In other words, decentralized power. That was replaced by contracts with central generation providers because that centralized model was so much more efficient.
You can see the same thing happening today with computing power (AWS, Azure, Google), music (Spotify, Pandors), movies (Netlfix), and books (Kindle).
What you want is something in the middle. Email is a good example.
Your comment assumes that efficiency is the most important thing. Addressing power imbalances and improving local resiliency are also important.
In the 1960s and 70s for many people computers meant centralized power and oppression. Computers were huge and expensive and therefore centralized, and only big government and big business (i.e. The Man) could afford them. A lot of the appeal of desktop computers in the 1980s was that they were personal: you could run the computing tasks that you wanted, on a machine that you controlled.
Today what do most people use their computers for? Accessing the services of big service providers like Google, Apple, Facebook, Twitter, Amazon, etc.
I'm not suggesting we couldn't get by on decentralised systems, but rather that such concepts are largely the domain of the geeky. Centralisation is what provides the convenience and trust that drives utility and adoption.
For that matter, even Twitter adapted torrent technology to distribute binaries to servers at a datacenter.
How can you measure and transact in BTCUSD without going to BitStamp?
By going to BTC-e, OKCoin, Bitfinex, Coinbase, localbitcoins, or Craigslist?
I'm not suggesting we couldn't get by on decentralised systems, but rather that such concepts are largely the domain of the geeky.
Geeks didn't pass anti-trust law.
Billions have been spent to create the IP transport networks. Building and operating them at current capacities, or even at 1% of current capacity, is of out of reach for most people who would run peer nodes of a mesh network.
Any node that'd happen to be in the bottleneck connecting two large segments, or just on a route to a very desired host, would be hit by a huge demand to pass traffic. Top-of-the-line 300 mbit wi-fi will be easily choked by that, not giving the node nearly enough bandwidth for its own requests. Huge fiber cables and microwave lines exist for a reason, and are not going to disappear.
The shape of the traffic they carry might change, though. E.g. with the advent of bittorrent, much of the traffic worldwide has become peer-to-peer / decentralized. This did not affect major carriers too much, except that they had to add bandwidth.
I guess it would be useful to gauge the size of local vs international traffic - Facebook does not need to send my data to SF unless I am in contact with someone in SF. Video uploads and downloads are similar. Most of my daily traffic almost certainly has a strong correlation to my physical location.
If it just so happened that you are one of the few users with a mesh-network connection to a peer across the street, much or all of the traffic between your street and the next street will pass through you. Since it's a mesh network, you are expected to have a low-power, low-throughput node, which might be overwhelmed.
Do not conflate mesh networks (of nodes in physical proximity) with peer-to-peer networks (of nodes in logical proximity via all of the Internet).
There is no particular reason a mesh network needs to be low power and low throughout - I think one of the most potent mesh network opportunities is (slowly?) replacing in home routers with (firmware?) to communicate amongst a neighbourhood. With that a small and growing group of people will be able to exchange local data and video and voice.
I perhaps naively think that physical proximity will become a major deciding point in bandwidth between two nodes (which makes sense) and this will encourage an explosion of useful local perhaps democratic applications - think of it as garden fence protocol. Sharing not just pirated films but kids calling their neighbours to just chat, parents arranging bar eques etc
It's likely that anyone who wanted to operate a major server would be tied in to the network by one or two 1+gbps links, and probably colocating with other servers, allowing them to share the overhead of this, and have several connections operating to the network in parallel, lowering the stress on any one route.
Similarly, a lot of traffic is aggressively cached now to mitigate the need to hit the server with an identical request. There's no reason to think that certain kinds of content wouldn't be cached at various points in the mesh, making the total traffic to any particular server lower still.
A realistic scenario for a datacenter is that 50-100 people split the cost of having 10 1+gbps links to the network, hitting it at multiple points well spread in the network. The inside of the data center would do some routing to handle the load balancing. I expect the total cost of that would be about $35k to set up, but splitting it between that many people, each person is paying (on average, for 50 to 100 people) between $350 and $700 - well within a reasonable amount for a coop to set up a datacenter. Further, $35k is a small enough price tag that it would be acceptable for a business (or a couple businesses) to share such a datacenter themselves, if there were a particular incentive for them to operate as part of a community mesh, rather than with the current internet.
[1] http://www.amazon.com/Ubiquiti-AF24-Airfiber-Wireless-Comple...
[2] http://www.amazon.com/Ubiquiti-AF-5-airFiber-5GHz-Wireless/d...
There are a couple of considerations for me:
3rd party (particularly commercial entities) vs personal ownership of user data and identity seems like a nail which is quickly coming into focus a) with the ways that people are wanting to use social media and b) in light of recent privacy and monitoring revelations.
The kind of use cases above in SWAT, combined with p2p network effects seem to offer a glimmer of far more powerful functionality than any closed platform, however large, could ever keep up with (just as we see open source eat proprietary competitors through sheer scale and subsequent pace of innovation).
My instinct is that the federated technologies suffered from being ahead of their time and will get picked up again at some point.
With unreasonably recurring failures, there is little incentive for mainstream usage, as current networks are extremely reliable.
With excessive latency, there is little usability, as data may become outdated before being consumed, thus prohibiting real time applications (ie: one of the big reasons that we use computers).
With little bandwidth, little data can be sent, thus limiting the overall usefulness, especially in a day and age where data just keeps getting bigger.
But if you have all three, you can effectively have an internet that is extremely difficult to censor or alter, that is capable of connecting multitudes of users, that can handle most uses. And that, my friends, that will be the day.
"Most services today are centralized — sharing photos via social networks"
Still, decentralized systems (in the context of the article) are great, but they present one little challenge: What incentives do people get in order to host a participating node/agent? In the case of bitcoin, the miners get a reward ($) each time they solve a new block, but unless all these decentralized systems leverage an existing blockchain/cryptocurrency system and have a system of rewards of some sort, they'll have a hard time springing into existence.
Even email is centralised these days. Look how many providers host over 90% of people's primary email address.
If the case of a social network, you can't just switch and still remain connected with everyone else, because the provider controls the flow of your interactions. However, you can very easily switch your email provider, or even host your own, and as long as your friends still have the same emails, you're still "connected". I wouldn't call email centralized, but I agree is not decentralized 100 percent.
More generally, I think there's an incentive problem here: it is just too convenient to let Big Data host your email. And if you're crazy enough to have a mail server at home, it is just too convenient for everyone else to spam-filter you as a netbot (because you're using a residential IP address).
decentralized technologies would promise to benefit all participants; but i guess the problem here is legal protection; a registrar based on blockchain is a contract that would require legal recognition; how easy would it be to achieve such a change?
http://en.wikipedia.org/wiki/Legality_of_Bitcoin_by_country
this article has lots of words even when the currency is legal.
This is not true. Mesh networks are vulnerable to electronic warfare. Governments can use measures usually reserved for electronic warfare[1] to disrupt all signals except the ones the government needs. This is a standard capability of most modern militaries. Even the United States Marines, probably the least sophisticated among American branches of the military when it comes to intelligence and technology, have radio battalions that can do this.
Mesh networks are awesome, they have a life of their own and an be split apart and stitched back together endlessly.
Blockchains, on the other-hand, are big and bulky, require an extraordinary amount of co-ordination between the maintainers and the miners and the clients to ensure everyone is running the right software. Many are so big that more people simply defer to a company to a handle their interactions with the chain - and as these companies grow they simple forgo the blockchain altogether in many instances and simply move bits about in their own database.
Also, can anyone offer an insight into the benefits of decentralized voting and storage? I can already decentralize who I rely on for storage by trusting different parties different amounts. There seems to be no benefit for everyone to trust the same blockchain for storage - and plenty of issues (hard forks etc.)
As for voting, the existence of a vote requires the existence of a group that wishes to vote (and a co-ordinated campaign.). Many countries have decentralized counts (e.g. each region votes separately) - and may separate organizations and individuals can become monitors to ensure the integrity of the ballot box and count. What then is the advantage of moving the actual vote to the blockchain?
Clearly, for any sort of data where reliable access is important this won't be viable. It seems to me that it is only useful to do distributed storage among servers, rather than laptop or desktop computers, since servers have high availability requirements already and aren't likely to be taken off the network.
Second, you probably need to trust the servers you are storing your data on. Sure, you have some guise of anonymity, but ultimately you're probably transporting over TCP/IP and so you can look at the IP sender field to figure out who's data you're storing. Obviously this is a huge problem if some malicious person decides they want to corrupt or delete all files stored by certain entities.
Both of these problems (availability and integrity) seem largely intractable when dealing with distributed storage unless you make the limitations I suggested above (trusted computers and only using highly-available servers). Definitely an interesting thing to work on, but probably not viable for consumers (though I could see it working at an enterprise level).
My understanding is that this can be mitigated by giving nodes a financial incentive to regularly be online and able to provide proof they still possess the data.
Vitalik Buterin of ethereum worte an excellent blog post addressing all of these concerns. The conclusion is that you can in theory get 99.7% (99.99%) availability by using 3x (4x) redundancy from regular home users (defined as being reachable 50% of the time).
https://blog.ethereum.org/2014/08/16/secret-sharing-erasure-...
Now, one problem with his math is this: the assumption is that any node that goes down will eventually come back up. However, this isn't the case. Say I get a new laptop and throw my old one out or otherwise wipe its drive. Well that's no longer part of the distributed network anymore. Of course, you can add to your file storage protocol by shifting files around as nodes go down, but that adds a lot of complexity.
Ultimately, the problems that are trying to be solved here are very difficult and I think a satisfactory solution via distributed computing is unlikely unless you are willing to make severe trade-offs (such as the ones I suggested, use trusted servers).
Now, making trade-offs doesn't make it useless. It just means the system isn't useful in all cases. Realistically, the data you have on your laptop probably isn't so important that it needs to be replicated across the globe, encrypted, and stored in such a way that the government can't access it. And if it is? Well, it's probably much simpler to just encrypt it on your end, send it to AWS, and get on with whatever you were doing that's so important.
It's true - they do look like they will usher in a new world. It's pretty damn hard to justify the architecture of sending my wife's photos to SF just so they can appear on our neighbours Facebook feed.
But the technology feels too immature. I just wish I could tell if I would have felt the same way about perl cgi scripts in 94 - but I was too young then to know the difference.
This isn't as simple as saying Google, Facebook etc. have a vested interest in promoting a centralized view, but that it's basically impossible to make money competing against them with a decentralized system.
There's a definite trend towards late 90s techno-utopianism appearing again.
I haven't looked into 3rd party apps, but I imagine they could use the same protocols as Firechat for file sharing. But I think on Android it would only work through Bluetooth anyway. Google needs to mandate all OEMs to use Wi-Fi Direct otherwise.
I think I used it once to connect a pair of PSPs to play a two player game....
I think the use-cases for true Ad-Hoc mode are actually pretty limited and in most cases one system (android or iOS these days) can probably run in AP mode anyway.
Computer science researchers have been working on and dreaming about these ideas for decades. It's great that some of them have found real-world applications, but that doesn't mean a revolution is near. It will be just as hard as it used to be.
Could bring a lot of value around file sharing, streaming and such. Plug-in-less.
The demo page get to pitch in a couple of seconds.
Have a look at it.
He's sticking with his "Guy with the Red Tie" branding.
So how does a blockchain divorce work? :)