Hacker News Reader

Comment by mbrock | Hacker News Reader

The most useful pattern I know of for offline web apps is the command queue.

Basically, the rendered state of the client is the acknowledged server state plus the client-side command queue.

User actions don't make a server request and then update the UI. Instead they directly append to the local command queue, which updates the UI state, and right away the client begins communicating with the server to make the local change real.

While the client's command queue is nonempty, the UI shows a spinner or equivalent. If commands cannot be realized because of network failure, the UI remains functional but with a clear warning that changes are waiting to be synchronized.

(The API for connectivity status are useful for making sure that the command queue resumes syncing when the user's internet comes back.)

View on HN

You still have to handle cases where the server state has been updated (possibly via another medium, event, ...) and when the user's internet comes back it's not a matter of pushing clients' commands anymore. Instead you have to merge changes (either backend or front end side) before fetching the new state. For example, I try to purchase an item on my desktop, but can't because I lost connectivity. So, I proceed to buy it on my mobile. When the net comes back, did I mean to buy this item two times or just one ? In this case, it's easy to find a workable solution (cancel the second order command), but you get the idea: It's not trivial.

CQRS and ES, are wonderful tools in such situations. Past that point, your web app is far away from the little CRUD mashup it was at the beginning.

Yeah, heavy transactions like purchases need special care, but that's true in general. At least you'd want a confirmation from the user.

In our case, a simple command queue solved the vast majority of use cases, and the corner cases were not disasters but at worst somewhat confusing.

Conflicts are actually not all that difficult to solve. We had to solve this exact problem with a time entry solution that needed to be mostly available offline.

Taking most of our inspiration from Git, it was simple. Define the atomic unit of conflict, and the definition of a conflict that cannot be automatically resolved, and simply help the end user understand and resolve the conflict.

95% of cases were fairly easy to merge without interaction, once we had defined "conflict." And the remaining 5% just required a little extra user experience design to help surface the appropriate resolution.

The problem I have with this approach is that you don't encode the intention with each change. You just encode the data itself. This makes it a bit of a hack, unfortunately. You can end up in conflict-resolution cases which you cannot always predict beforehand.

Yes, from this viewpoint, Git (and every similar version control system) is a hack. But it works well in practice because humans are running it, not machines.

I've once built a system (an outliner app) that actually encodes intent, and uses a command queue ("transaction queue").

Intent could be for example to insert this new item as a first child of that other one, or to move these items to a position right before some other item, or to set font size on this item to 4.

The data structure was a tree of item objects linked via next/prev/children/parent. Automatic conflict resolution works wonderfully in this case.

The intention was "I want to change this guy's hours from 5 to 8." I don't know what else you might mean by "encode the intention."

Unless you're referring to something I'm utterly overlooking, I don't see how you'd get around the invalidated assumptions that came from being offline and having outdated information. Intent doesn't matter if the intentions were invalidated.

When assumptions change, you really have a management problem on your hands. There's more information (sometimes very complicated information, like "the back office changed his hours to at least 8 because of union contracts, so now he has 11") that must be communicated and decided upon, sometimes by more than one person (e.g. foreman and superintendent) to figure out the appropriate resolution.

In version control systems, this is the same, and unavoidable for asynchronous workflows. It's basically optimistic locking, really. You make changes hoping that nobody else has altered the data, then check to see if any of your assumptions (i.e. nobody else is making changes) have held. If they haven't held, then you need to recheck your assumptions and resolve the conflict; there's no way around that.

The intention was "I want to change this guy's hours from 5 to 8." I don't know what else you might mean by "encode the intention."

Suppose you change foo from 9 to 10. Was that because you now wanted foo to be 10 specifically, or because you wanted to increment foo and it happened to be 9 before so it becomes 10 now?

In isolation, these have the same effect. However, one is absolute and the other is relative, so if two of you happen to make that same change at the same time, your intentions matter very much to how your respective changes should be combined.

For example, if your code knows that both changes were intended to set new absolute values, it can automatically determine that the combined effect should also be 10. Similarly, if your code knows that both changes were intended to increment foo, it can automatically combine those effects to get 11. But if all it knows is that two people changed foo in concurrent updates that now need to be merged, that probably results in a conflict that requires manual resolution by a user.

Right, but the assumption of original state is still invalid. An increment from 9 to 10 might be invalid if, say, the guy has already worked 40 hours and by union contract cannot work any more hours. If it had been corrected to 8 and my intent was either "set to 10" or "increment by one," the values of 9 or 10 are both invalid. So, unless we want to attempt to account for all possible rules (and vastly overspend on the project) the best choice when assumptions are invalidated, regardless of intent, is to surface the conflict.

This is very much just a distributed database where we've chosen availability in the presence of network partitions rather than consistency. The end result is that conflicts will inevitably happen, and aside from a hugely complex set of rules, the cheapest resolution is still human intervention.

Intention has no 'from' clause. You express the intent as "I want this guy's hours to be 8."

It doesn't matter if the UI shows 5, and another mechanism has changed the value to 9 in the background, your intent to set the value to 8 is unaffected.

If, however, you assume the "from" clause and do a +3 instead of =8, you get a new invalid state 12.

Encoding intent implies declarative statements as apposed to impaitive statements.

JSON-Patch [1] can be helpful here. You can maintain a list of changes that need to be applied, such that they only touch the parts of the document that need to change (other changes to other parts can be interleaved). If necessary, you can include tests to assert that some value in the document is what you expect it should be, and the patch can be rejected if that test fails.

[1]: http://jsonpatch.com/

That's if what's to be changed is a document (which is a big if).

Things get ugly quickly for any more complicated scenario.

Yeah, that's true. JSON-patch won't help with all situations. It is still an under-utilized tool, IMO.

I think it's strange that it doesn't efficiently handle patches of strings.

By the way, I'm still looking for a 3-way merge algorithm for any datatype in Javascript.

CQRS = Command Query Responsibility Segmentation – http://martinfowler.com/bliki/CQRS.html

ES seems to be Event Sourcing – http://martinfowler.com/eaaDev/EventSourcing.html

Having recently built an offline first app (mobile app I'll admit), we didn't use a command queue, and I regret it deeply now. I advise everyone who's reading this to go for a command queue. Saves a lot of time debugging data sync.

Our biggest challenge was to sync data with relationships, especially data with circular relationships. We couldn't come up with a generic way to sync circular relationships, so we ended up building a very special purpose buffer on both client and server side.

is there anything more you can say about your buffer solution? Curious if it is something similar to vector clock syncing that you guys came up with.

What was the challenge with circular relationships? I work on an offline-first system and all inter-object relationships are managed by GUIDs, so cyclical references don't present any problems - objects are sent and received as one big list keyed by guids, not as a tree.

"objects are sent and received as one big list keyed by guids, not as a tree"

That's an interesting pattern I've not heard of before - any links you could recommend for more details on it? It sounds really useful for moderate sized data sets.

I remember coming across it fairly heavily in react/redux stuff, mostly because it makes dealing with redux's stores a lot simpler if you avoid nested structures like that.

There's a library called normalizr[1], whose purpose is to take nested API responses and turn them into flat structures. Have an article[2] about it.

[1]: https://github.com/paularmstrong/normalizr

[2]: https://tonyhb.gitbooks.io/redux-without-profanity/content/n...

didn't Netflix do a bunch of research into json-graph? They seemed to have figured this problem out.

Did you consider using an off-the-shelf solution like Realm?

In my experience it made it trivial to handle the synchronization of arbitrarily complex object graphs (including those with circular dependencies).

I'm a big fan of command queues, but if there is a chance of a server-side failure that can't be realized immediately on the client (e.g. an edit conflict with a different user editing a same piece of information), then I find that surfacing those errors in a meaningful way can be difficult and frustrating for the user.

By the time the client has synced with the server, the client could be doing something completely unrelated in an entirely different part of the app. Explaining to them that "the thing you were editing two hours ago has an issue, go here to resolve it" can be tricky.

Yeah, it can get tricky, but it's a tradeoff between working nicely in the majority of cases and being accurate in the corner cases.

If you make sure to show the user a clear warning that they're working offline, then they might be more understanding if their changes are rejected two hours later.

Treating changes more like emails/outgoing order forms might be useful here.

You could popup a /failure/ message and leave a little indicator icon that takes them to the queue and lets them see the items that failed. If retry is an option they can see it, if retry is not an option they can be told why.

What about in cases where the commands may fail because of the actions of other users? For example, say you had a virtual market in a game and someone did a "buy from John" command while offline. they then proceed to use this purchase to beat future levels. However, once the user reconnects online, it turns out that John already sold to somebody else, and thus their purchase and everything after it is invalidated.

In your command queue pattern, what sort of ways do you handle this? Have certain "chokepoints" where the user must be online to proceed? What if you have an app where that sort of chokepoint seems to occur too frequently to make the queue useful?

Edit: I think this is similar to Fiahil's sibling comment, also relevant points made there. Thanks for the quality thoughts everyone!

The real thing is, it is important to synchronize with the server at that point, otherwise something could be invalid.

It may not be possible to make every app work while offline. You'll probably want to potentially disable certain features while offline, such as store purchases (or, at least, queue them up, but don't show them as having been successfully purchased)

That case would probably need server synchronization to confirm, but other cases can be handled with operational transformations.

https://en.wikipedia.org/wiki/Operational_transformation

That wasn't a big problem in our particular application, so in case of failure we would just report a potentially unsatisfactory error message along the lines of "Your changes could not be saved. Please try again." (with a list of the discarded commands' descriptions).

In a situation where this kind of thing was more important, I would think about how to let the user decide how to reconciliate their changes. It could be that a choice of discarding or retrying would suffice, or something more complex.

The "chokepoint" notion is also useful, and in fact our app did have a distinction between potentially offline actions and necessarily synchronous actions, but our synchronous actions were mostly queries like searches.

Unfortunately, the "command queue" abstraction does not work very well with access control. Imagine that one day the requirements w.r.t. security change. What do you do when some parts of the state may not be viewed by all users? And what if that logic depends on the state itself?

Sorry, I don't understand. Could you give an example?

I see management and security on different planes requiring parallel priority paths. Activity on these channels can be intrinsically disruptive and may require clearing queues elsewhere.

Know of any good examples/tutorials of this?

Unfortunately not, but if you think of it as using the command pattern in a queue-ish way to get a similar behavior as the command queue of "The Sims", you have the basic architecture, and from there it's just a matter of coding.

The thing about React-like frameworks that makes it very nice is that you can keep the command queue in a separate place and have a root rendering function like this:

1. Set the view state to a copy of the actual state

2. Update the view state according to each queued command in turn

3. Render the view state including a status bubble showing that some changes are not saved yet

And separately from the rendering, you have a worker that tries (and retries) to perform the queued commands. When a command is successfully performed, it's removed from the queue and its effect on the state is saved in the actual state.

Since I can't find anything on Google when searching for "react command queue" it would be cool to write a blog post with a simple example, but I don't know when I'd have time, so I encourage anyone who's implemented a similar thing to go ahead.

This is similar to what I wrote about here: https://blog.superhuman.com/architecting-a-web-app-to-just-w...

That feedback that the commands are queued or can't be processed is key. I encountered an issue on an app I manage at work where the UI showed that a certain state changing action had taken place before it was resolved on the sever. Needless to say it caused a variety of issues as users thought the system had processed a significant action(hiring a candidate) when they hadn't.

This makes so much sense. Would have saved me many headaches when developing an offline app a few years ago.

Yes, This. I struggled with understanding how I'd deal with a complex offline multi user collaborative web app until I starting looking at in terms of a Kafka queue. Gives you everything, including undo/history.

Any suggestions on front end frameworks?

I implemented such a solution in a React app and I really appreciated the ability to apply pending state changes without mutating the state itself.

I don't know of any open source libraries for the command queue itself. If your state changing commands go through some kind of layer that you control then this stuff is easier. When I implemented it, I first refactored all the commands to go through the same code path, which I could then modify to implement the queue.

I suspect that using Promise.all() helps greatly with this.

The way I've done it is to just have an array of commands and a simple queue thing that keeps retrying its head element until success, triggered by connectivity change or manual user retry.

(This was also useful when our backend had random issues causing 500s sometimes.)

A command has both an AJAX request and a state updating function. It's really easy with a React-like framework because you can just apply the command queue's state changes as part of the main view render, without actually modifying the main state.

Yeah, look into using Promise.all. You're probably looking at a good few hundred lines of code reduction. What you're describing as a "command" (request / state update) is just a promise with a map function applied. Your "simple queue thing" can just be an array that you fire Promise.all at.

Can you serialize a promise to localStorage?

How do you deal with network failure in the middle of the promise array?

If a promise in the array is rejected, Promise.all (which returns a promise) rejects with (supposedly) the first rejected promise, but that's extremely hard to predict if you have two promises which could reject.

Serializing a promise to localStorage; I get what you're trying to do, you want to have a worker pick up exactly where it left off when you left the app. This is where a service worker would help you. I suppose you could write some kind of durable mailbox a la Akka, only running in the browser.

Yeah, these kinds of issues were why I decided to manually manage an array of serializable commands instead of using Promise.all.

Also showing the state of the queue (Sims-style, or just a number of pending changes) is easier when you control the queue yourself.

Do you mean like CQRS/ES?

It makes a lot of sense in an event sourcing context because the code used for applying events can be the same in the optimistic case and in the real updates.

In our case, we weren't using an event sourcing architecture, but this client side pattern can be used anyway.

Reply on news.ycombinator.com