Say hello to offline first | Hacker News Reader

Say hello to offline first

blog.hood.ie

64 pointssteveklabnik48 comments

I hope this becomes a popular maxim - fed up of offline not working for productivity tools (not necessary for other types of apps, but Trello is a shining example of this omission making productivity apps near-useless).

This could be seen as an extension of "YAGNI" (You Aren't Gonna Need It). Does your app need to hit the server constantly? Can it not cache data and only do network operations on an as needed basis?

This is not really caching. Caching is about not hitting the network to get data you have already gotten before. Offline is about having an application that works correctly in the absence of network, which means preemptively getting all or a subset of your data.

But I agree that you could see the server as YAGNI, especially if all your app is doing is data consumption and your data is small enough to be downloaded to the device in its entirety.

I have written about caching and offline here: https://winch.io/blog/article/caching-prefetching-and-the-of... (warning: promotional content for a service that helps you do that for native mobile apps at the end).

Apology not needed for promotion; I'm biased, but I think far too much emphasis is put on services that encourage needless reliance on constant connectivity and third parties' servers. Offering an alternative is welcome.

As for local storage, even on smaller capacity devices, there's a whole lot that can be kept - why, for instance, does a todo list, addressbook, appointment manager, workout tracker, etc, etc, need to connect to a remote server for it's very basest of functions? Sure, backup, synchronization, sharing, etc need connectivity, but those aren't (IMHO, shouldn't be) the main focus of these apps.

I'd like to echo this, specifically focusing on AJAX. It's not enough to be offline-friendly if your requests don't themselves retry or work well when internet is spotty. A problem I've often encountered in online-only scenarios is when AJAX calls fail. How do you ultimately retry them? At a certain point, you need the same infrastructure you'd be building for an offline app to cache and queue requests, as well as present sensible non-success responses. So, just as mobile first solved problems for both tablets and focusing a design, offline first can solve problems with error messages, caching to speed up online responses and AJAX retries.

or you go away of AJAX calls entirely, by storing all data offline and having a separate process that synchronizes all changes, like remotestorage.io does it, or hood.ie

That's what I implied by AJAX retry. That you need some method that sensibly queues the request, until it succeeds or is no longer needed. That process could easily be called synchronization, sure. Depends on the app, really. The request might "expire" or be no longer necessary, so it wouldn't show up at "sync" time. Perhaps the request could be automatically retried through other devices that are connected -- imagining for the moment, an app that could use bluetooth or local wifi to communicate an operation across to other running instances even when offline.

"This is quite literally a new frontier, largely unexplored and full of interesting problems and unimagined edge cases. In most other realms of app design we are spoilt with UX/UI patterns we can readily employ, but offline-first is terra incognita, there is practically nothing to go on in terms of patterns and metaphors."

Alright, let's step back a minute here.

We have over a quarter-century of offline-ONLY desktop applications to draw on. There are also a fair number of intermittently-online desktop applications that have been in existence for years. I realize it's not fashionable to consider these as applicable, but the only thing new here is the platform (browser). UI patterns abound - though unsurprisingly a quick google search for UI design patterns doesn't show anything except the latest web ui patterns.

These desktop applications still power the everyday work of most computer users[1], to a large extent more than most transient web-based apps. Why would we start any discussion on the assumption that they're not relevant?

[1] and before you tell me that you do just fine on your chromebook using only connected services: you are NOT most desktop users.

@GrinningFool totally miscommunication, sorry if we cam off wrong. We don’t mean to dismiss the offline metaphors we know from desktop apps, in fact we desperately need to transform them over to predominant app design, especially on mobile.

Thanks for the clarification.

We have over a quarter-century of offline-ONLY desktop applications to draw on.

and

though unsurprisingly a quick google search for UI design patterns doesn't show anything except the latest web ui patterns.

. . . seem to contradict each other. Sure, offline (or at least intermittently connected) apps have been around a long time, but as you so rightly (unsurprisingly) found, that kind of thinking (patterns) has been surpassed by more trendy ideas.

Part of it can be blamed on the SDK and API designers; they implicitly tell us "your app will be online all the time". Part of it (as you pointed out), is due to general ignorance of history. I think a large part of this is that large swathes of new programmers (only in it for the money?) have jumped on the apps bandwagon on phones (online by default), offline be damned. That, as the authors argue, should be fixed.

Generally speaking, when offline apps went online, they would "sync". This process is not straightforward, quick nor modern. Web apps don't "sync", they just "work". How can you design a native app that behaves like a web app? That's a less explored question. I'm not saying it hasn't been looked at yet, of course, and phones are some of the first to do it. Microsoft could take the lead here, with their approach of "only showing icons for connectivity when the connectivity changes" and of hiding buttons when they aren't going to work, so you can presume that anything you see on screen is going to work even when offline. Windows Phone and Windows 8 guidelines have a few more examples like this. On the other side of the fence, Apple has done similarly with their apps. iTunes Radio will continue caching the next song and part of the song after it, so even when I go down to the subway and lose signal, I still get to hear another 5 min of music. That said, the rewind and replay features of TuneIn were still superior and it'd be nicer still if iTunes radio transitioned into playing back either cached tracks or my own library instead of stopping dead. (Wouldn't it be nice if while listening, between tracks or when it stops, Siri stepped in and announced why the track can't play and suggests alternatives?)

Part of this new medium is to come up with techniques of explaining that apps work offline -- people's expectations, still, are that if they visit a website, they need to be online. That alone is a huge hurdle.

In addition, some of the best services, e.g. browser-based push notifications from Apple (rather than those of native Chrome) will not work without Internet since they use the same push protocols as native push notifications on iOS (iirc). That said, even on iOS, I think apps can now add notifications without internet, so that should be changing, just as apps can sign up to do background pre-caching of data.

There are some HTML-only problems too: How do you version web requests from remote hosts? How about third-party JS? There are so many considerations in offline scenarios that most web developers never need consider. I really hope offline first takes off as the next "mobile first" or "responsive design". If nothing else, it will make for more robust online-only apps.

The paradigm is not particularly new, even if the implementation needs to be. You're essentially describing IBM (Lotus) Notes sans cruft (proprietary rich text structures, UI, etc., that predate HTML/WWW). Space- and/or time-limited replication when getting the whole data set is unnecessary, local design/code caching and so forth are solved problems, so to speak. (They've been solved at least once before, and so an be solved again.) Say what you will about the Notes client (as a developer, I had to fight it every step of the way), but the underlying ideas are still valid and, IMHO, still necessary.

To be explicit, 20 years ago, there were people on airplanes reading, composing, responding to email and reading creating updating, deleting documents in multiple applications. They were creating and following links to docs and email in applications. All offline. When they reconnected via modem in the hotel room, it all synched with the the servers. Code and design changes also replicated so the apps as well as the data were always up to date and in synch. Replication could filter only those records that the users needed. Even long-running background processes could be run locally. That was Lotus Notes 20 years ago. CouchDB and its relatives can do this because Damien modeled it after the Notes Database, but with open protocols and standards (yay!).

Back then, even online was slow and expensive so many people worked in offline mode even when connected. Many still do this today, even on fast local networks. Perhaps there is yet more to be learned from the ancestors.

I agree with both of these replies. Lotus Notes was an interesting beast to learn about, back in the day. I suppose it's always been a problem to solve, though: We're not suggesting that before the WWW, there were no online-only services either. But graphical web browsers changed people's expectations. An offline only app that runs in a web browser... has about the same impact, for both the technology architecture and expectations. I look forward to this future and what new frameworks/"cloud" evolve, and I'll have to investigate more in this personally. :)

"We have over a quarter-century of offline-ONLY desktop applications to draw on"

I agree. You can't even span a table cell in Google Docs. Wordperfect 5.0 could do this over 20 years ago. This is not progress.

So many "cloud" or browser-based apps have limited functionality compared to their desktop equivalents. The makers of these apps tout the simplicity of these apps as a selling point. But simplicity is not necessarily the absence of features (in fact, simply removing features could be argued as the lazy way of simplifying your app).

What's more, online apps give companies unprecedented opportunities to track online behaviour. This is one reason why I would never use Google's ChromeOS. Even something as simple as printing to your desktop printer requires signing into Google's cloud printing service. It's clearly in the interests of Google (and other online app companies) to sell the idea of online apps and services as the way forward. I'm certainly not convinced that it is the way forward.

It's funny how GrinningFool seems to be associating "online" apps with cloud-based services (Google docs, dropbox, etc.) and thus equating offline with their pre-cloud iterations.

I think the OP's intent applies to every app that uses data (pretty much most apps except self-contained games). Think of something like Foursquare (remember them?) that is geo-based, yet totally dependent on communicating to their server. I never used the app, but how did it handle not having a data connection? Better yet, look at the iPhone's built in Mail app. It creates an outbox for any email I try to write and send while offline (so it can handle being offline), but at the same time, it gives me a no-connection pop-up warning for almost every action I take, which is understandable, but also not very usable.

Also, the old offline-only apps are NOT the same (in terms of UI and behavioer) because the new offline-aware apps have to live in both worlds. They must do work when connected and "cache" work (both up- and down-stream) when not connected--and they must do this seamlessly even when the connection can come and go without notice or predictable duration. I think it's this dual nature that the OP is talking about and I agree it is a new paradigm that most apps don't handle gracefully.

I don't think I was making any association between online & cloud that wasn't inferred from the article itself. I could be mistaken though.

Desktop mail clients have been queueing mail for delivery since their inception (as have MTAs but that's out of scope here). A desktop mail client also has to live in both worlds - while you have a connection you expect to be able to get near-real-time notification of new messages; yet while you're offline you expect to be able to view most of your message content as well as compose and queue messages. You more-or-less expect to be unaware that you're offline.

Firefox supports an offline mode where all content is served from cache (I'm not sure if Chrome does, haven't used it recently). Switching into and out of it isn't very graceful or automatic, but it is there and fairly well done.

Which leads back to my point. I agree it should be done gracefully, and will concede that the kinds of data have changed. However, at heart it has been done previously, and more-or-less gracefully.

In an age of always-connected phones, web apps, and a world confined to the browser, those lessons seem to have been forgotten - I just wouldn't want to see this go the route of re-learning them from first principles.

Living in the mountains of Northern California with crappy cell coverage, I hope thinking "offline first" really takes off. Through my day I pretty much only use apps that can "hold it" until we get to another wi-fi or cell coverage oasis.

Besides my personal coverage challenges, any mobile app should just be smart enough to handle loss of connection gracefully as a matter of course. It's one of the error conditions (along with low battery) that is guaranteed to happen.

I will definitely be watching hoodie for my own projects. Most of the US (by area) is in the same boat, I suspect.

> We can’t keep building apps with the desktop mindset of permanent, fast connectivity...

That's just silly. A lot of apps are rightly useless without normal connectivity, like checking the weather [edit: for current conditions], or sending e-mails, or Facebook, or shopping on Amazon. There's no reason for these to work offline.

I mean, an app is pretty much online-only, or mostly offline (like most games, for example).

I don't see a big middle ground between the two where online apps need to be made to work better offline. And the few which do (which involve syncing data), seem to work fine.

So... I don't get it. What is "Offline first" trying to accomplish in the real world?

> A lot of apps are rightly useless without normal connectivity, like checking the weather, or sending e-mails,

Except that thinking leads to really shitty behaviour from apps. Like weather apps that refuse to show you the data cached from the last time they checked, or e-mail apps (I'm looking at you here iOS GMail) that are feeble-to-useless at letting me read offline. Facebook: I should be able to check my events calendar offline, or look again at favourite photos.

The set of apps that consume online data is much, much larger than the set of apps that are usable if and only if they have absolutely live information.

By not thinking offline-first, too many apps behave like they're in the second set.

Responding to you and everyone else using weather as an example:

NO, I don't EVER want cached weather data! In fact, it drives me nuts that when I look up the weather on my iPhone to see what the current temperature is outside, I have no idea if it's accurate or 20 degrees wrong, because it just assumes that cached data from 1am is good enough, until it happens to get a connection again. For people who live in places where the temperature can vary 40 degrees in a day, this is really important.

[Edit: why is this being downvoted? Is it not clear that I'm talking about current weather conditions, as opposed to 3-days-out? And that having my phone present stale data as current data can be harmful? Many people use their weather app for current conditions, not forecasts.]

Your weather app generally is able to show around a week of data in advance, so if I check weather today and am offline tomorrow, it should be able to show me a reasonable approximation.

Also you generally want to see what the weather is in the future, and its always a best guess, if you want an accurate way to know precisely if it is raining right now, look out the window.

And you asked for an explanation of why you are being downvoted, you are obviously trying very hard to make an example of a counter point for something that doesnt make sense '12 hours ago we thought the weather would be X' is unarguably better that 'this application is offline', but you are attempting to argue it. Even if you did find a single counterpoint it doesnt add anything to the conversation, not absolutely everything is going to have an offline use case, but its clear that we could do better providing offline capability (and to be clear, I didnt downvote)

Surely all that takes is a "last updated x hours ago" label and you have the best if both worlds?

Just because the current designs have flaws for your use cases doesn't mean we can never design around them.

You're being downvoted because based on a single, arguably marginal, use case ("I only ever want to know the exact temperature right this very second and everything else is useless to me") you're a claiming that there's no need for anybody to consider the much more important and larger use case of "I need my phone not to instantly become a dumb brick when it loses reception"

I saw elsewhere that you think mobile apps are already doing the right thing here. The reaction to you - and the point of the OP - is that they're not. Proper offline support is often a poor second in application design, if it's considered at all. We should change that.

We are trying to get people to talk about offline-first app design for when it makes sense.

Weather is a great example, being offline could communicated as an error, leaving the user with an abysmal experience. Or some older content could be shown with the note that it is likely that the accuracy is off, but if the user just needs a general idea of the weather, that suffices and is not at all an “error” in the networking sense.

We have no common UX/UI language for these things and we’d like to change that. ;)

We got by with weather information that was hours old on the past, shouldn't be a problem now.

Good luck. Anecdote: after finding a small scorpion on the bathroom wall in the Honduran jungle, I had a huge compulsion to tear everything apart to make sure there's no nest. The only thing that calmed me down was access to offline, text-only Wikipedia on my Nokia n800. It was 2 a.m., slept like a baby when I found out this species was harmless. To this day, I never travel without it. (Edit: Wikipedia, not the n800 ;)

> or sending e-mails

Better mail applications allow you to "send" the mail with no connectivity and then simply hold onto it until there's a network connection. At which point the mail is actually sent out. But for the user it's transparent. He or she hits send and (at some point), the mail goes out. I wish more phone apps did this (looking at you, Twitter).

This actually was my everyday experience back in the nineties with Pine on Debian 2.x

I liked that.

If my mobile email client didn't have good offline functionality I'd be looking for another one. Reading and queuing emails to send should generally not require a connection.

Weather is another strange example for you to pick. Yes, the more recent a forecast, the more useful it is. But if I checked the weather this morning, and then want to reference that again in the afternoon (to make a plan for the weekend, for example), that should work without a subsequent connection.

And really, there's no reason I shouldn't be able to check Facebook updates and comment when it's convenient for me. Inherently asynchronous activities like this are well suited to offline scenarios.

Shopping is a better example, but you're 1 for 4 here, in my opinion.

I disagree.

Many parts of the world have reasonably accurate weather forecasts for 24-48 hours out -- you'd only need to have five minutes of connectivity at the beginning of the morning; the rest of the day the app should work just fine. And if you're stuck without connectivity for longer, it can show the right part of the 7-day forecast in many locales (albeit with lower certainty). If you need to know the weather _right now_, many phones ship with a barometer sensor, or you can just look out the window.

Sending e-mails was a mostly offline process in most of the 90s; you'd only connect to your dial-up connection to actually send and receive messages to your ISP's SMTP/POP servers.

Facebook already displays old news feed items when you're offline, and with a bit of work, it could decide to cache comments and likes offline when you interact with news feed items. Similarly, it could place photo uploads and status updates in a queue and sync them when the connection does come online.

Amazon predicts items you might want to see based on prior shopping history. While you wouldn't be able to do a search or see real-time pricing/availability information, it could happily show you pictures and product description in a carousel while offline.

Being able to read, organise and draft reply to emails, being able to see my facebooks profile / contact information, in every app where you need to be online, being able to view history (or precache future data) when offline would always be useful.

I am most certainly not the only person that has had an important page shown on my phone etc and accidently clicked some link and suddenly lost all that information.

Most applications have a very large middle ground between offline and online capability, I think the reason its hard to see is that as developers we are so terrible at doing it well right now

Mobile apps that are really online-only could do a lot better job though. The biggest examples would be to queue and store data to be posted and also to handle larger caches of downloaded info.

If ¥ou look at Instagram, it does not store much in the way of old photos in your feed, your old likes, or even older photos of your own profile. It's probably done in to part to simplify the experience, so you know the comments/like-count on photos are up-to-date, and it also keeps the app's footprint smaller. The percentage of time you are not connected and want to use Instagram is small, but it would be nice as disk space increases, to have more functionality available.

Apple's shared photostream is a good example of a similar use case (photos) where data is permanently cached and available offline ( to a degree) and you can still interact and 'post' to it where it'll wait and perform the action when network connectivity returns.

> "A lot of apps are rightly useless without normal connectivity, like checking the weather, or sending e-mails, or Facebook, or shopping on Amazon."

They aren't fully functional without fast connectivity, but they aren't useless. It is shocking how many ways apps break when offline.

- If I'm offline I should still be able to view the last weather report I got. Many apps simply refuse to work and throw up an unhelpful "you're offline" screen instead.

- If I'm offline I should still be able to type out a reply to emails. In fact, I should be able to queue it for sending, and my device should intelligently take care of it at the next connection opportunity.

- Similarly, if I'm offline I should still be able to view emails that I have seen recently.

There are a lot of ways people can interact with apps while experiencing no/poor connectivity. I believe the essay is simply saying that some basic affordances should be built into your apps - that features which don't strictly require a network connection shouldn't fail in ugly ways when the network is down. It is atrocious how many mobile apps have only one reaction to a loss of connectivity: completely shut off all access to everything.

But... phone apps already do all those things, so I don't get what you're complaining about. My point is, apps already seem to be doing the right things. So what's some kind of new manifesto needed for? What is supposed to be changed?

Phone apps might, most web apps do not. Hence this post.

I just downloaded the first 4 results for 'weather' on my iphone. 2 refused to show any data offline, one showed cached data without saying it was, and one showed a needlessly verbose error about being offline, then showed cached data with an indication of how stale it was.

So no, not all apps are already doing the right thing.

> like checking the weather, or sending e-mails, or Facebook, or shopping on Amazon.

* You don't need to check the weather every time. It's enough if you show me the data you fetched this morning or maybe just 2 hours ago (maybe so the time of the last successful request somewhere)

* Just let me write (and read...) my emails, change the label of the send button and put them in a queue when i click it. Actually send the mails when i get online again. (Fun fact: I am already doing that on my laptop, i write a lot of mail offline and msmtp queues them until i got a connection again. Useful on train rides and flights with spotty or no connection.

* I do not use facebook, but i hear that a lot of people also use it as an addressbook, for example. At least some of that data (names, telephone numbers, addresses,...) should be available offline if possible. Depends of course...

I think this offline-first approach is very interesting, and definitely something that is overlooked in UI and app design. The OP is correct about telco and wifi signals being far from total coverage. There are lots of "holes" and in some places, it's more a metaphor of moving between islands that have connectivity.

There is also another case of artificial data "scarcity" and that is the case no-data plans. To save money, I have a low cost ($10 and up) minutes-only phone plan. With apps that work with the offline-first mentality, being offline could be almost invisible on a 1-hour commute between the wifi at home and the office (no, I'm not one of those people who needs to read and write tweets in real-time).

Sometimes it is also a hardware issue. The first time I tried to use my phone as a true GPS with MotionX (this was 4 years ago on an iPhone 3G), I discovered it's not possible. The app does download maps and cache them so you can go hiking out of cell range, but the iPhone does not support GPS-only operation. Either I leave the radios on and it depletes its battery looking for a signal, or I put it in airplane mode and GPS is disabled. I don't believe a GPS-only configuration is possible even today.

Another point is that offline-first does not only apply to smart-phone apps, although that is probably the biggest problem space. But there are a lot of people who commute with laptops, and who could maybe live without a 4G dongle if their apps supported it.

I have also thought a lot about the sailboat cruising community (see the last sailboat picture in the OP). There are many recreational boaters out wandering from port to port, using sparse wifi and sailmail (very limited email services via marine HF SSB radio). A lot of them blog and send text and email to their family, friends, and followers, but have to struggle to time their online windows. Just imagine a laptop app that lets people write emails and blog posts whenever they want (in the middle of an ocean), and maybe it uploads the text to their blog via sailmail immediately (but slowly), and then adds the pics automatically when they get to port and connect to wifi (or have a 4G dongle within range of a cell on the coast).

So not only are there online and offline states, but also limited bandwidth states such as sailmail (it allows short text emails in both directions, but not images).

I don't believe a GPS-only configuration is possible even today.

Are you referring to iPhone? I'm fairly certain that's a software limitation, as I know of non-iPhone smartphones that can easily enable GPS alone (no cell, no BT, no WiFi, etc).

Yes, I'm referring to the GPS on the iPhone. Of course it's ultimately a software limitation, but from the consumer/non-jailbroken point of view, it's baked into the product. No app can override it.

Been working with SymmetricDS (open-source Java database syncer over HTTP). Works on Java desktop, Android, Java web servers. An IOS version in the works.

Anyone else using it?

Would be nice to see a pure AJAX / IndexedDB plugin.

Since a local database would write directly to a central database, you really want row-level security and good triggers, so a user can only screw up their own stuff.

Related: CouchDB for mobile.

With its master-master replication and explicit conflict handling, it handles being offline extremely well.

Did this post get nerfed? It had 45 points in one hour but it seems to be stuck at the top of page 2.

It was at the top of page one at one point in time; along with another interesting article (that was killed) about a new operating system that looked promising, I'd say someone with an agenda has been flagging perfectly relevant articles today. Perhaps it was branksy. Or mobile app developers who don't like the idea of their users disconnecting because then they can't track their every move. Or web app developers who can't conceive of an app that doesn't involve a web server.

Doing my college homework, if I had assumed I was offline and then only went online if I determined I was, then I might have spent half the time debugging my homework.

Explain, please? You appear to be comparing a state of mind (assumption of being offline), to a way to design apps (don't assume constant connectivity), with an unclear result (what do you mean by "spent half the time"? would it have taken you half as long as it originally did if you assumed you were offline? or you would have spent half your time debugging instead of looking up answers?).

Reply on news.ycombinator.com