Decentralization for the web
lwn.net
lwn.net
This is the best summary of What's Wrong With The Internet that I've read for some time.
You can see other echos of NAT in today's slow adoption of IPv6... Since lots of software thinks it can safely run on a LAN without you caring about it, lots of systems (even Linux environments) start up with that in mind (even if only subtly, by listening on all interfaces instead of just localhost), and that fosters the fear of dropping that NAT firewall, which is the obstacle to decentralized (and truly competitive) services.
Edit: A recent comment showing the mindset I'm talking about above: https://news.ycombinator.com/item?id=9983056
Sealed by the decision of ISPs to provide a piddly 1m up even on lines with 50m down, disallow inbound connections to residential modems, etc.
Way back then, when your online connection was flaky, expensive and ~5-50kbps, the first online-only apps where greeted with "but.. but.. what if offline?"
Next thing I know: The iPhone mandates internet access and steam rolls everything into online-first/only mode for the average user. Business models based on that make sense, the industry follows. Privacy concerns are swept away. Control is taken away from the user to the service. Data governance/ownership is on its head. Synchronization is hard, marketing is easier. Decentralization is written off as anarchic geek fantasies. Interoperability does not fit business needs. Open protocols become data islands.
Are we actually about to come back to "but.. but.. what if offline?". I hope so. And how about reversing some of the problems we introduced earlier while we are at it? Privacy, decentralized services, interoperability.
Why exactly this happens is an interesting question. I don't know the answer, but I have a vague suspicion that it happens where there is a fundamental, ultimately unresolvable conflict, and the "answers" just oscillate around the central issue.
Certainly hashes will make things a lot better, but a lot of people I know (including some of the people referenced in the article) are heralding them as the second coming of the web.
I disagree because they present more problems, ones that are harder for humans and easier for machines. First off, in order to find the data you practically speaking have to already have the data. Second, if you don't have the data, then you have to know the hash, but how do you know the hash without having had the data OR trusting some authority for the hash? Now you are back to centralization.
Third, and most important from my perspective, a hash often times will point to outdated content because by the time you find it it might have changed (this of course is a good thing but can be problematic). I know they addressed this a little bit in the article but it is worth reiterating. The point is that it is hard to do synchronization on hashes and I would argue that sync is the more important problem and that is what I am trying to address in my open source database.
My wife and I visit a lot of the same sites, even see a lot of the same dynamic content (same facebook friends, etc). Surely there's some community effect that would make this a huge bandwidth win.
A similar mistake is security updates over bittorrent: a real-time list of vulnerable hosts.
You could still keep those bits off of the WAN, though. In a "privacy mode" the gateway could coordinate and throw in an average delay.
You're also spot-on that dynamic content is the hardest part of hash-based addressing, but the problem is a bit more nuanced IMO. On the one hand, all content, once created, is static. If a thermometer reads 32.8 at 15:40 on 30 July 2015, that data point is fixed in time, immutable. However, we humans think of things far more conceptually, and we build cognitive connections between things. All content is static, but all concepts are dynamic. So the question is then, how do you reconcile those two?
The best answer I've personally come across is bindings -- exactly analogous to binding names to variables in Python. Some hash bindings would be static, some bindings dynamic. That also allows you to construct more complicated objects like buffers natively. That's how I'm doing it in the project I'm working on (https://github.com/Muterra/doc-muse), though the bindings are a relatively new addition that I haven't had time to test yet (or update the documentation, for that matter).
There is also the data hosting cost. It seams more fair and logical that data producers pays for the hosting of their data. Otherwise there is a risk of abuse.
Honestly, I think there is a need to have solid p2p libraries, so to be able to use a DHT, NAT punchthrough or upnp more easily. P2P is immensely hard and has a lot of implications as much as it has usages. Isn't there a p2p filesystem already ? I'm sure you could distribute public data in a decentralized manner.
I don't know if libtorrent does it well, but since there are security issues implementing software that sends data packets directly, it would be greatly welcomed.
Also I don't believe diaspora went far enough in the decentralization method. There should be no server at all.
So far this thing: https://github.com/LukeB42/Uroko/tree/development implements a collaborative page editor. All it needs is a gevent-based implementation of Kademlia.
The difficult part is that you as a node in this overlay network may be requesting a document that only one other node has, and you have to trust that node. The other part is that different nodes will have cached the URL you're requesting at different times. How do you prevent lying nodes being taken seriously?
As for sharing edits to documents, that's probably best done as a premeditated thing with people you know, which is where merkle trees'll be useful for verification.
But having said that, I think that the reason a lot of stuff becomes centralized is because SOCIAL is not decentralized today. Bitcoin decentralized money but user accounts, profiles, connections etc are still done in a centralized way. That's why GitHuv and is centralized even though git is not. Social and security - if there were solutions to these, many people would decentralize.
And by decentralized, I mean you still have a server hosting your stuff, but it would be your choice - it could be on a local network, and you wouldn't even need the internet. You could be in the middle of rural Africa and your village couls run a social network, which sometimes syncs with the outside world but 99% of the communication wouldnt require it, wouldn't require those drones fb launches.
I think our company Qbix has decentralized social, in that way. It's not decentralized like bitcoin or mental poker, but honestly I don't know why zero trust is such a big deal. Even bitcoin has most people host their wallet with others amd take risks.
As long as there is programming there are bugs. We can't prove correctness of all programs by writing purely functional code. Having more eyes on the same code is more likely to expose these bugs. The caveat is that everyone hopes someone else has checked the code. But I don't see how using closed-source application would solve this issue.
Security by obscurity can be better than exposing all your code to the world where any hacked can compromise the whole network, BEFORE the fix is patched.
And even with open source, would I trust a random small host to secure it better than google? Look at all the android vendors that don't even install the latest patches.
Then there is what Schneier calls the security feudalism problem: maintaining a secure system is very hard, so people prefer to submit to a corporate feu lord which can provide them with security, paying with their privacy as a form of feu duty.
I think one of the biggest problems is DNS.
Until there is an ubiquitous decentralized name resolution protocol there is not much of a point in running your own server on a host that changes IPs every now and then.
(Decentralised protocols run quickly into "Zooko's triangle": https://en.wikipedia.org/wiki/Zooko%27s_triangle )
DNS is quite centralized with its registrars and the DNS root.
Not to mention that it needs to be much easier to setup a server securely for the average person, and have a digital wallet to store their micropayments. (I love bitcoin but asking your average facebook user to setup a wallet securely is still too difficult)
This article correctly called out the advertising incentive of today's big companies as not being in the average users' best interest. I wish it had also talked more about what kind of incentives that are missing to get people to become a part of p2p networks. Great article though, I like how the author tied in the slowing rate of discoveries/progress into the problem.
And never will be. Information assessment costs integral to the payment decisionmaking process are too high.
Bundle. Don't disaggregate.
(But allow for exclusion of high-cost items on request.)
I don't want to be lazy; I did look for some sort of thesis or argument but I'm not going to read that long paper without even knowing what it's about.
If people want to have true freedom of speech, or better privacy (or just enough), I really think there should be no server at all.
I really want to see an unmoderated decentralized forum. Even 4chan is slightly moderated.
But yeah, Ted was right, as were Vannevar Bush and Doug Englebart before him, or Alan Kay after. Some more recent thingies:
http://idlewords.com/talks/internet_with_a_human_face.htm http://idlewords.com/talks/web_design_first_100_years.htm
http://worrydream.com/TheWebOfAlexandria/ http://worrydream.com/TheWebOfAlexandria/2.html
Probably it was written almost live and there wasn't a recording available when the article was published. However, they didn't even add the video link afterwards. Why?!