Swarm.js and React – Real-time, offline-ready web apps | Hacker News Reader

Swarm.js and React – Real-time, offline-ready web apps

swarmjs.github.io

162 pointsdanabramov44 comments

Interesting. I still think that the Flux (specifically Reflux) is the way to go with React. I know this solves a different issue, but I've been itching to build a truly unidrectional data flow application with something like a Rails REST API and an Elixir socket server with a Redis queue in between.

All requests go to a REST API which only returns head responses, and shoves the actual data response onto a queue. The websocket server reads off the queue and publishes the response to connected clients.

On the client side it's all actions and dispatchers and react components. Keep meaning to write an in depth blog post about that when I get the time.

Anywhere I can keep track of you and your progress? I am looking at an almost identical setup (with Elixir) and need to get some inspiration.

AFAIU you're doing Twitter right - I mean your last tweet. Easy follow.

> All requests go to a REST API which only returns head responses, and shoves the actual data response onto a queue. The websocket server reads off the queue and publishes the response to connected clients.

This reads very similar to how IMAP is defined (it's not a request/response protocol).

And Websocket seems like an overcomplication of the system as you need a WS-compatible server and broker, it would probably be simpler to use SSE[0][1]. See http://matthiasnehlsen.com/blog/2013/06/23/angularjs-and-pla... for an example of SSE+occasional HTTP calls.

The big difference, of course, is that such a system is not really offlinable, the application is basically frozen as soon as the connection is lost.

[0] even if MSIE doesn't support them, they can be polyfilled even in IE7 and old android browsers

[1] plus SSE has native support for server-configured reconnection timeout, event-ids and custom event types, all that in an ASCII-based format

Nice. I hadn't played around or even heard of SSE.

I'm not _as_ worried about offlinable, to be honest. This post just tickled that part of my brain that had been thinking about React.

> Nice. I hadn't played around or even heard of SSE.

Yeah SSE is one of the most interesting APIs people have never heard about, everybody jumped on WS but for many use cases SSE is more than sufficient, much simpler to deploy and use and a much simpler upgrade path from polling.

In theory it could be at least partially offlinable, so long as you push a UUID or similar with each request you send to the server. You can then immediately push the expected response into the message queue, which would update your UI, and on receiving the real response update the relevant entry in your local cache of the data store.

I built a websocket server in Go that pulls messages from Redis and shoots them off to subscribed clients, similar to what you're mentioning. https://github.com/johnernaut/goatee

The Flux+Swarm TodoMVC was mentioned in the post. These are orthogonal things. https://github.com/andreypopp/todomvc-flux-swarm

They are different things. Have you used them?

It always annoys me when SPAs don't allow me to do basic things when I lose my connection.

Webstorage has excellent browser support but isn't utilised enough (http://caniuse.com/#search=webstorage)

Webstorage is a bit handicapped for SPAs. There is a 5MB size cap, and you can only store strings. It's nice as a cookie replacement, but the limitations hurt for anything more advanced. IndexedDB resolves these issues while also being fairly well supported.

IndexedDB has terrible support. Sadly LocalStorage is the only reliably supported option right now. I really hope in 6 months this comment is wrong.

--EDIT-- sorry @untog, I replied to the wrong comment and deleted before as you replied.

Use a library that uses IndexedDB and uses WebSQL as a fallback then.

The Web SQL Database specification is no longer being maintained and support may be dropped in future versions.

Also - these fallbacks are not exactly swappable. Its like swapping from a Database to a Filesystem as a fallback. Difficult to abstract into a single library/api.

And you still potentially leave out IE users if those are your options (depending on your IndexedDB needs).

IndexedDB to localStorage fallback might work - but again might not be an easy API abstraction.

I have had good luck with https://github.com/mozilla/localForage as an API abstraction

PouchDB, ydn-db, IndexedDBShim, and Lawnchair will do this as well.

Web SQL may be "deprecated," but it's unlikely to be dropped by Apple or Google, since so many mobile web apps depend on it. Even the mobile version of Gmail uses Web SQL.

This is what we do and it worked great until iOS8 came along. Read more about some problems here http://www.raymondcamden.com/2014/9/25/IndexedDB-on-iOS-8--B...

I wonder if asm.js + the sqlite3 driver + localstorage would suffice.. it would certainly be interesting.

> .. and you can only store strings

That's isn't much of a problem with JSON.strigify and JSON.strigify.

Except that eats into your already small storage quota, and burns CPU.

This is really interesting ! I'm currently working in my company heavily with offline html5 features. The assumptions are not really the same as this framework on my case since on my case the app will probably be 95% of the time in an offline mode.

I've created a small library which is making the html5 manifest much easier to work with so I can put http headers like commands inside it. The client is checking with an interval to update the app and there is a built-in loading bar to show the download. This supports also an additional client-side api to make things easier.

Here is a gist of how it looks like currently : https://gist.github.com/alex-min/1bbc304ad9b96bdf5c96 I think I could see if I can open-source it if that's interesting for some people.

But this library seems really nice, I will have a look since the synchronization of objects seems really well done. (from the demo page).

Great work. Eventually consistent state across sessions will be the norm rather than a novel feature in a few UIs.

Does anyone know how Swarm.js compares to ShareJS and other OT libraries?

The very general response is: Swarm.js is CRDT and thus more async-friendly. It may survive failures all along the chain (db to server, server to client, client to cache, etc). It may synchronize with multiple sources: WebSocket to the server, WebStorage to the cache, WebRTC to a peer -- all at the same time.

That flexibility is paid by (1) implementing Lamport timestamps and by (2) limited use of version vectors (on handshake). ShareJS versions are linear, for example, but those linear versions are specific to a replica, as far as I can tell. In OT, version 3 here and version 3 there are possibly different.

CRDT/CT is generally easier to reason about than OT, esp. considering various non-standard situations and implications. That is mostly because CRDT (this flavor) employs "partially ordered log of immutable operations" while OT operations are mutable. That is the formal difference.

It looks interesting but I wonder how it works if you have to filter or search for records when you have many records. I did not see any query language only iterating through all records and filtering in memory.

Firebase is actually implementing an offline mode soon that isn't LWW.

How does this deal with offline storage limits in various browsers?

TodoMVC hardly needs that, but otherwise we are working on an IndexedDB storage implementation. Our Storage interface is asynchronous anyway. So, no problem. Cache eviction is also applicable here.

FWIW, PouchDB [1] advertises itself as an in-browser implementation of CouchDB, but it also serves as a robust layer on top of IndexedDB & WebSQL that tries to smooth out a lot of the browser differences.

It looks like their clocks/IDs are a combination of local timestamp and user ID? Is there technically anything in place to prevent duplicates?

There is a detailed post on timestamps at http://swarmjs.github.io/articles/lamport/

Lamport timestamps are "guaranteed" to be unique.

How comprehensive are CRDTs?

There are CRDT implementations for:

* plain text

* rich text

* maps,sets

* by-column Last-Write-Wins objects

* vectors

* counters

...and many other useful things.

Do you have any links for the rich text implementations? That's one I've not seen yet…

There are some articles on plain text CRDTs on my academic homepage [1]. Causal Trees are the same family as WOOT, Logoot and TreeDoc.

Also, we did a RichText implementation for one-$10bn-company-that-forbids-to-mention-its-name-too-often. I did not publish that yet.

[1]: http://www.pds.ewi.tudelft.nl/~victor/

Love to see it when you've published! I've thought briefly what a rich-text implementation of causal trees might look like…but it hasn't bubbled to the top of my priority list yet.

Also, has someone compared the CRDT and OT (like Racer) approaches?

Website is a complete mess. Hardly anything on it works.

Are you running HTTPS Everywhere? That seems to break the blog for me (on Firefox), and disabling it fixes it up.

Thanks, we'll check that.

Which website? The blog on which this was posted? It looks ok to me, as does the todo example. It wouldn't hurt to be a little more contructive!

Reply on news.ycombinator.com