Hacker News Reader

Comment by jimpick | Hacker News Reader

There's a lot of similarities, as they are both peer-to-peer and decentralized.

I've mostly done Dat. I want to do a bit more IPFS.

Dat feels a bit more like git for files - you can create a local file archive using the command line tools, and it's a separate step to sync it to the peer-to-peer network. There's a global discovery service for advertising archive keys, but it doesn't work at the level of single files. It's very lightweight.

IPFS supports many of the same operations, but you're mostly interacting with a local gateway server which is continuously connected to the network. I believe IPFS tries to content hash every single file so they are de-duplicated globally.

View on HN

Does anyone has build-in realtime features?

Yes, Dat's core data structure is an append-only log that is designed for extremely fast real-time replication. http://github.com/mafintosh/hypercore

If you use Dat through its command-line app, you get real-time updates to the files shared using it.

Hashes and append only logs are all not very good for realtime data because of the extra overhead that has to be calculated, but CRDTs are.

CRDTs naturally fit with P2P/decentralized topologies, and we've generalized them in https://github.com/amark/gun which is the most popular (8K+ stars) open source (MIT/Zlib/Apache2) realtime decentralized database.

It is running in production on P2P alternatives to Reddit and other apps, that have pushed over half a terabyte in a day.

It's not incorrect to say that hashes and signatures add some overhead, but the question is whether the overhead is significant enough to matter for the usecase. Probably not.

Dat is realtime. You are notified about updates as soon as they are distributed. In Beaker, the files-archive API has a `watch()` method to do this. If you're accessing a dat files-archive, you're participating in the syncing swarm and so you'll receive those updates automatically.

You'll want to use a UDP socket if you're streaming a high volume of data with low latency requirements, for instance for a multiplayer FPS. But Dat has been used to stream live video, so it's probably real-time enough for most Web use-cases.

Small aside: Comparing Dat to CRDTs is apples to oranges. It's like comparing Javascript to B-Trees; they're not quite the same kind of technology. In fact, the new version of Dat uses CRDTs in order to allow multiple users to write to a files-archive.

When we met just a month or so ago you didn't tell me you were adding CRDTs!!! This is very exciting news. Dominic had mentioned a specific type of CRDT he had added (but wasn't generalized).

Append only logs have overhead that would make it a poor choice for most realtime applications, like GPS tracking, yes FPS games, google Docs, website creators, and many more. Basically any use case where data mutates.

In 2010 I used and built all my own custom event source system, I was the hugest proponent of this / append only logs. It was so futuristic. But 4 years in I hit all sorts of scaling problems and had to redesign everything from scratch and that is when I found CRDTs. On all accounts they are superior, in a mathematical or logical manner, because they are a superset to DAGs, immutable/append only logs, and many other popular data structures. Not apples and oranges.

I built a little side multiuser wiki side project that’s actually using two levels of CRDTs... hyperdb (underneath hyperdrive) and then automerge on top of that. Sort of hard to explain the full design in a short entry, but you can play with it here:

https://dat-tiddlywiki.glitch.me

The UX definitely needs some improvement, but it’s just a personal project so far.

Reply on news.ycombinator.com