Hacker News Reader

Comment by ex3xu | Hacker News Reader

While I'm just hearing about this Edsu project and am not in any position to vouch for it re: future betrayal, I think the specific concern you raised is solved by Edsu by using local storage rather than a centralized repository.

A few clicks in, the first line of https://edclave.com/ is:

"Edsu makes life better for both developers and users of Online Open Source Software (OOSS). Instead of the developer having to hold on to your data, you do. It gives you more control, and gives them less to worry about."

View on HN

Local storage feels really quite volatile. It lives on a local disk inside your browser. May as well just use notepad and a file.

It seems the data stays in sync with other places you're signed in, so supposedly you could get redundancy by signing in on more devices/browsers. In that sense, it's better than a crappy text editor on your computer.

You're right about the sync part, but it's an old skool client-server sync, not a peer-to-peer one.

Hmmm, a p2p version that uses local storage on each device you have could have some interesting applications. As a client-server architecture it's a tad complex for a lot of users to set up, but with a good UI it could become a viable alternative storage method, as well as allowing for potential app cross-compatibility. You would have to make sure at devices had a chance to sync - perhaps have a web service there? - and I'm unsure how you would deal with conflict resolution, but the idea definitely seems to have a lot of potential.

I'll need to think about this.

It's possible and there has been research in this area, see http://archagon.net/blog/2018/03/24/data-laced-with-history/ for details about conflict resolution.

The main problem is garbage collection - to guarantee that you can sync across all devices for all time, your data structure must be append-only, so your document can only grow. A long living document will eventually grow very very large. You could let the user decide when to collect garbage/establish a new baseline doc; maybe like 5 years. But this means that if you edited the document on a laptop, didn't sync it with another client, and then closed the laptop for 5 years you would lose the ability for the changes you made on that laptop to be resolved automatically.

Just use a CRDT

That's what I'm suggesting, but I'm pointing out the fact that CRDTs are generally append only data structures and it's possible to run into space issues with them eventually for some users.

A lot of CRDTs are append-only.

But not all! Quite a few that are state-based, auto-compacting, space-saving, small & efficient!

More info in a discussion on this: https://twitter.com/marknadal/status/1008610024875122688

A CRDT which models edits of arbitrary text with essentially unbounded revision history (required for merging) almost certainly can only grow (monotonically?) in size. Maybe you can be smart about compression, but I don't think it can ever shrink. Someone smarter than me can probably formally prove this with an argument about entropy while considering a series of particularly pathological edits.

But you're right that I'm wrong when I say "[all] CRDTs are generally append only structures." I meant to say "text-revision CRDTs are generally append only data structures."

...I just described a subsection of git... although with better UX.

But let’s say for example your phone and your computer are clients, and your computer has latest version of your data. When you open your phone, if your computer is not on the internet, you won’t be able to get latest. So, would this be practically useful for a terribly trivial use case like this?

Have a server be just another peer, not privileged in any way, but always online and available.

Maybe do the routing through a distributed hash table like we use for finding torrent peers (perhaps even the exact "mainline" DHT torrents use), which would mean that even with the server being down you could still sync with your other online clients.

Look at the WebTorrent project!

You can run an Edsu server locally, but I wouldn't. It's meant to be run on an internet-accessible server - a $2.50/month VPS would do, or if you don't want to run your own there's at least one provider that will do it for you.

Notepad++ and Dropbox work quite well together.

Reply on news.ycombinator.com