How the Dat Protocol Works
datprotocol.github.io
datprotocol.github.io
The dat homepage[1] is also nicely done. It would be interesting to see how much and how it's being used. Since there is a central tracker does anyone know if there are stats available?
Scroll to Overview and then there are links to each section in the diagram
Here is a blog post describing how it was made: https://blog.datproject.org/2019/01/21/how-dat-works/
> Duncan is a technical writer and illustrator interested in internet infrastructure, peer-to-peer protocols and net art.
[1] https://blog.datproject.org/2019/01/21/how-dat-works/
[2] https://github.com/datprotocol/how-dat-works/commits?author=...
The scuttlebutt doc linked elsewhere in here is even more impressive IMHO. Kuodos to the designer.
Then I sit down to actually start playing with it I struggle to think of anything to build that would actually take advantage of the tech aside from purely static sites. I would really love to see more complex examples folks have seen in the wild.
What I realized is that this technology ticks a lot of the boxes of what we right now think only the big cloud providers can do. By using a more decentralized protocol it is by design
- Actually Serverless
- Offline-first
- Real-time
- Auto-deploying
- Live-updating
- 100% uptime
I really think there is something there from both, a developer and user experience perspective. The problem is that a lot of it is still very experimental and far from the usability and maturity of e.g. a Firebase or Heroku.
Love the design, by the way.
The design is taken from the Chat guide for https://feathersjs.com. Feathers is a JS library that allows to architect APIs in a way that they are protocol independent. Which worked great in this case because I just had to swap out the existing REST/websocket Feathers adapter for a DAT/Beaker API Feathers adapter.
How can someone without the keys add their messages to the feed? Wouldn't only the initial creator have the keys necessary to add to the feed?
There is some interesting stuff people are playing with in Dat - e.g. their Twitter clone (1) has the actual posts distributed in what they are calling their WebDB. I.e. you as an individual publish posts at a dat:// address you control, and the central site just references that address when other users load the feed. Your posts are distributed across the other users so there is no SPOF.
There is the kernel of something really interesting there in this GDPR-world where people need to be more aware of who has their data, and what can be done to control that.
Interesting stuff.
1 - https://github.com/beakerbrowser/fritter/blob/master/README....
Also a bunch of zines and music albums are shared over DAT.
There is hypercore (https://github.com/mafintosh/hypercore), a distributed append only log library.
Rotonde, a DAT based social network (https://github.com/Rotonde)
Secure Scuttlebutt now supports dat attachments...
Cabal (https://github.com/cabal-club/cabal-cli), a chat program.
[0] https://wiki.xxiivv.com/#rotonde
Code is here: https://github.com/kickscondor/duxtape.
Actually as someone that deals heavily with static sites, if dweb was only used for static sites and perhaps pushed us back to a more document-oriented web I and a lot of others would be quite happy with that. I don't think it needs to be capable of feature-complete replacement of Twitter and Facebook to be successful, and requiring it to be that way as a precondition for success is IMHO a losing strategy for dweb. Start with document-oriented and slowly increment.
- dat://connect-four.hashbase.io - https://github.com/rjsteinert/four-in-a-row-game
Specifically it seems like a better version of bittorrent. It seems to be more resilient that bittorrent, better able to do updates than bittorrent (they apparently promise to have live updates by multiple writers, implying one could host for example a live database in a dat URL), better able to find optimal peers than bittorrent, better security around discovery, a flexible datamodel (e.g. the data being shared does not have to be files and folders, where as bittorrent would require wrapping that in a file, this can allow the transparency through the protocol layer)
"Think WAN, act LAN"
or is it
"Think Wan, net LAN"?
I don't know.
You'd just create a dat feed for it and share that address in the thread. You can do that directly from Beaker Browser easily.
If everyone on the thread was getting it, you might only have to upload slightly more than 1Gb data, as all the chunks could be shared between the other people.
Same as bittorrent, but even better because you can easily add more files to the feed later.
So it's wasteful and can't represent reality well. So it's quite limited to how far we can use it as a whole species.
It's less the Amazon of algorithms and more the Myspace. In 15 years it will still have some impact in the abstract sense of how it influenced technology, but blockchain itself is unlikely to still be there.
• https://docs.datproject.org/docs/faq#how-is-dat-different-th...
"Discovery keys are used for finding other peers who are interested in the same Dat as you.
If you know a Dat’s public key then you can calculate the discovery key easily, however if you only know a discovery key you cannot work backwards to find the corresponding public key. This prevents eavesdroppers learning of Dat URLs (and therefore being able to read their contents) by observing network traffic.
However eavesdroppers can confirm that peers are talking about a specific Dat and read all communications between those peers if they know its public key already. Eavesdroppers who do not know the public key can still get an idea of how many Dats are popular on the network, their approximate sizes, which IP addresses are interested in them and potentially the IP address of the creator by observing handshakes, traffic timing and volumes. Dat makes no attempt to hide IP addresses."
Is that correct? Would those snoopers be able to prove who is sharing and who is downloading?
https://www.datprotocol.com/deps/0002-hypercore/#the-linear-...
The post here is an alternate visual version. Best of both worlds!
Personally I think that the way they're using diagrams to illustrate some of these technical concepts is really helpful. I wonder if the format would translate to spoken text? I.e. in parallel with the figures, having little descriptive "imagination breaks" in the text which would provide opportunity for focusing on some key concept or relationship that might otherwise be illustrated using a visual figure. Maybe even using spatial language?
IMO there should be way more aurally oriented technical materials.
Recently I was listening to math lectures on YouTube while driving and realized that they were much easier to follow than I expected. Still, I'd love it if there was an aurally focused higher math practice. Sometimes I find it easier to focus when I'm listening rather than using my eyes.
>Academic Torrents [13] uses BitTorrent to share scientific datasets, and BitTorrent has many drawbacks that hinder direct use by scientists. BitTorrent is for sharing static files, that is, files that do not change over time. Dat, on the other hand, has the ability to update and sync files over the peer-to-peer network. BitTorrent is also inefficient at providing random access to data in larger datasets, which is crucial for those who want to get only a piece of a large dataset. BitTorrent comes close to the solution, but we have been able to build something that is more efficient and better designed for the data sharing use case.
Although you can build applications on top of syncthing's API/protocol, it's primarily a high level application, whereas Dat is a protocol meant for building applications.
It would make sense to build something like Syncthing on top of Dat.
Is there any way to stream video using OBS via dat?
I can't share a file without having to copy paste the url, and I can't look at two similar urls and immediately determine which is which. No thanks.
This seems like yet another product that seems good on paper, and seems great if you're a nerd, but this will never catch on with the average consumer.
People are still trying to wrap their heads around decentralized naming systems. There are lots of ideas, and nothing has stuck yet in a groundbreaking way.
But yeah, the naming problem is independent of what this protocol gives us.