Private client-side-only PWAs are hard, but now Apple made them impossible
andregarzia.com
andregarzia.com
> As mentioned, the seven-day cap on script-writable storage is gated on "after seven days of Safari use without user interaction on the site." That is the case in Safari. Web applications added to the home screen are not part of Safari and thus have their own counter of days of use. Their days of use will match actual use of the web application which resets the timer. We do not expect the first-party in such a web application to have its website data deleted. If your web application does experience website data deletion, please let us know since we would consider it a serious bug. It is not the intention of Intelligent Tracking Prevention to delete website data for first parties in web applications.
[1] https://webkit.org/blog/10218/full-third-party-cookie-blocki...
> have their own counter of days of use. Their days of use will match actual use of the web application which resets the timer.
This makes it sound very much like homescreen apps will have their data wiped after 7 days of non-use.
> We do not expect the first-party in such a web application to have its website data deleted.
And this does not. It's a very confusing word salad.
Edit:
Also what about desktop?
But said timer... does nothing? Why does it exist?
What exactly does that mean? So you use the app for seven (perhaps non-consecutive) days, and now all third parties that haven't been, uh, interacted with, get their data wiped - but not the the first party, because that has been interacted with, by virtue of the PWA being launched in the first place?
I guess that solves the problem?
But it's no surprise that Apple would want to impose an "install" step on the web to prevent it from looking more attractive than the App Store.
It's kind of a nightmare due to both Google and Apple messing things up.
PWAs could be an amazing platform but both companies are really messing it up.
Apple is trying to kill them by giving plausible explanations as to why they can't have PWAs. Security this, blah blah blah. There's no reason they can't have PWAs work well in Safari other than they want you to port your app to the App Store and get locked into their native APIs.
Google's problem is, well, they're Google. Meaning things are somewhat incoherent, docs are all over the place, they start new initiatives then abandon them half way, etc.
Consumers are another problem. They have no understanding of PWAs and they go to the app store, don't find us, and then complain we don't have an app..
The plan now is to use Google TWAs and port our PWA to Android.
We're going to do the same thing to Apple after we do the Android release BUT I think there's a 50% chance that apple will just flat out block us.
I think we might have a chance of getting around it if we use mobile gestures properly, use platform specific APIs like the camera, audio, and GPS that aren't on web and try to really integrate into the platform properly.
For example, they have an API to detect dark mode now. IF that's on we're just going to magically enable our dark mode in our app.
- If I press the settings gear, the text on the settings page is about twice as wide as the screen, requiring horizontal scrolling.
- On the front page, if I open the color picker, it's partially offscreen.
- On all pages, if I do a scroll gesture in the wrong direction, it scrolls the entire UI rather than just the scrollable part. Admittedly, iOS has long made this hard to avoid without hacky JavaScript, but it's been doable, and it's much easier now [1].
- The hamburger button on the left opens a modal view that covers all of the screen but a small margin on the right, making it unreasonably hard to exit.
- If I try to create a tag or folder, the name prompt appears under the other modal view and is improperly sized.
- Oh, and the UI looks thoroughly non-native, e.g. Google-style floating action button, UI not covering the status bar, bottom tab buttons too short, etc. The animations are also haphazard.
My point is not just to nitpick. It's just that while I sympathize with the idea of PWAs in principle, almost every single time I see someone talk about theirs, the PWA in question has immediately obvious glaring UI defects that have nothing to do with browser limitations, and leave it far below the standard of a good native app, or even a bad one. I honestly don't know why this is, but experiencing it over and over makes it hard for me to care about PWAs.
[1] https://benfrain.com/preventing-body-scroll-for-modals-in-io...
Very interested in hearing about pain points you've had building out PWAs, especially if there's features you were keen on that haven't been released. Easiest way to reach me is on Twitter: https://twitter.com/b1tr0t
Fully agree with you that docs are all over the place. We've started to consolidate docs under web.dev, and the PWA section launched recently (https://web.dev/progressive-web-apps). Consolidating and adding docs is an active area of investment, and our goal is to create a well lit path for developers to succeed with PWAs.
Making it clear why a TWA is in the app store is hard in itself. Trying to explain why it's better for consumers over a native app + mobile site is even harder.
See these reviews for yourself here: https://play.google.com/store/apps/details?id=uk.co.openrent
You blame Apple, Google and your consumers, instead of just making native apps. Why?
Really? You're blaming your customers for not being sufficiently tech savvy and not wanting what you're providing?
Personally, I am happy with Apple's decision here.
Is it possible they also want you to port your app to the App Store to prevent an explosion of garbage and malware that could happen if PWAs really took off?
Now we store the account token in iOS keyring and that works.
ref: https://stackoverflow.com/questions/32927070/complete-data-l...
It is not clear that a user coming to your website before the 7 days, even offline, is exempt of it.
I hope they come up with some good options as this news settles. It's hard to see this as anything but even just a accidental push ('well you should always have written an app for the app store') to force folks to write a native app / participate in the app store.
Edit: getting downvoted without any reasoning provided, so I assume I'm incorrect, there are more/less ways of storing data in the future for Safari users?
- Standard HTTP Cookies - Flash Local Shared Objects - Silverlight Isolated Storage - CSS History Knocking - Storing cookies in HTTP ETags (Backend server required) - Storing cookies in Web cache (Backend server required) - HTTP Strict Transport Security (HSTS) Pinning (works in Incognito mode) - window.name caching - Internet Explorer userData storage - HTML5 Session Storage - HTML5 Local Storage - HTML5 Global Storage - HTML5 Database Storage via SQLite - HTML5 Canvas - Cookie values stored in RGB data of auto-generated, force-cached PNG images (Backend server required) - HTML5 IndexedDB - Java JNLP PersistenceService - Java exploit CVE-2013-0422 - Attempts to escape the applet sandbox and write cookie data directly to the user's hard drive.
https://github.com/samyk/evercookie
In short, everything and more can be used for tracking, and that has really killed the party for the many people who have created responsible, useful applications of these browser APIs.
Mobile apps suffer these kinds of problems far less, partly because it's understood that actually mobile users don't install apps then get upset about "tracking", in fact, the vast majority of apps will want you to sign in to some sort of account and those that don't will be using ad networks to fund themselves, that users understand and accept this and that throwing up permissions screens doesn't achieve much because users will typically grant the permissions. Privacy on mobile platforms is more about stopping activity the average user would recognise as illegitimate spying - turning on cameras and microphones to feed conversations to angry ex-girlfriends, that sort of thing.
If the web's architecture had some sort of coherent view on how the tension between users, content providers and advertisers should work, then we wouldn't see this steady endless churn of app-breaking API changes. Everyone would know the rules of the road and there'd be way less tension as a result. Mobile platforms aren't quite there because they were designed with security architectures that were then pressed into service as ad-hoc privacy architectures, but they're still far more coherent on the topic than the web.
I think your comment really hits the nail on the head, IMHO the frustration shouldn’t be directed toward Apple but more toward the groups who have pushed the tracking practice so far to necessitate such draconian measures.
Therefore, since native apps are more of a platform differentiator than web apps, moving forward we can expect Apple to start systemically hindering web apps, especially on ones that are good on iPadOS, in order to boost native apps.
(I’m not saying this necessarily the start of this, but I am saying I'm not surprised. This is exactly the type change, targeting the exact type of app I’d expect to be targeted.)
[0]: https://blog.robenkleene.com/2020/03/20/ipadoss-new-mouse-su...
As a web developer, I've never believed Apple has hindered web development on their platform, purposefully or not. They just don't spend their resources adding in WebBluetooth or whatever new API-of-the-day Google has decided to come up with.
As I see it, their focus is on the user, which is why they've been slow to adopt APIs that are privacy concerns, or drain battery, or have other negative implications.
They have been doing this for quite some time now. Always ostensibly to protect users but always also conveniently putting webapps at a permanent disadvantage to native apps.
For my part I'm not interested in being a user of a platform so hostile to the web that it disallows any third party browsers.
Browsers need to be severely limited due to them running arbitrary code from the web. Doesn't matter if it's an offline web app. If you want more access, make a native app (with or without web technologies).
But the argument that this will protect privacy in the first place seems really weak.
Before this change in Apple's policy, an app could store my config data on my PC.
After this change, they'd need to have me log in and send the config data to their servers.
That seems like I've lost privacy, not gained it.
> deleting all local storage (including Indexed DB, etc.) after 7 days
which I can see how it might help privacy (since you could be tracked via local storage too) but also how it might break any potential web app that might need data to last more than 7 days.
> If you want more access, make a native app
But then, everybody will complain about yet another Electron app, right? Not to mention that you have to fork over $99 and go through the signing / notarization hoops that change from one week to the other.
I think in the name of privacy and security only Apple and some select few corporations will be allowed to make software in the future. macOS / iOS and Windows 10 are evolutionary dead ends in many ways.
which is somewhat ironic, because the goal of a web app is to break free of the walled garden and become OS-independant.
Browsers usually ask for an additional permission in this case which would be a good approach. Your post sounds like "browsers need to be severely limited, so if you want to watch video, just launch VLC". It does not work this way.
The real 'write once, run everywhere' are webapps, a webapp doesn't care if you are using Apple, Windows, Linux, BSD, whatever, if you have a compatible browser you use the app.
Sure there is Electron (or React Native), to me it doesn't make sense, what is the point that every application needs to ship basically a browser? And still Electron apps need to be compiled and packaged for every platform, while with webapps you enter the URL in the browser and you are done with it.
Doesn't adding APIs to browsers not only to use the local storage but also to access the filesystem of your device (of course asking the permission to the user) make more sense?
Of course what really Apple fears is loosing the control of the apps that gets used on their device, now they control the App Store that is the only way to get apps on their devices (beside jailbreak), with webapps is different, since you can access them directly from the browser.
And the thing that is absurd is that the first iPhone didn't have the App Store since Apple decided that the only way to get third party apps was trough the browser, now they are aiming for the opposite thing.
How's installing a native app better for a random user privacy or security wise, exactly?
Native apps have the same problems too and such "severe" limiting of apps in web browsers still doesn't solve it. The only more or less privacy preserving model I can think of for native apps today is open source repositories with app distribution not controlled by app developers, like f-droid or repositories in various linux distros.
Otherwise this has absolutely nothing to do with privacy or tracking.
As a customer, I'm tired of devices functionality being limited coz "security risks". Functionality that is arguably superior to native apps apart from the security risk.
...and give apple their cut. Why not add permissions to webapps? Like location, or push notification... oh that's another feature that happens to be missing only in safari.
Just accepting these moves from apple as "in the interest of users" is naïve. Apple has a huge vesting in their appstore, and every webapp is a potential appstore-app that is some lost revenue.
I mean, maybe apple is right, and the web should go back to a readonly document-like format, like in the old days. Articles and links. Apps for everything else. But let's not kid ourselves that they do it purely in the user's interest.
A big chunk of the web these days uses JWT and localStorage for auth.
Can PWAs not switch to using IndexedDB which seems like it’s more purpose-built for this use case?
No snark intended. I’m legitimately curious what the situation is and where any blockers are.
- Indexed DB
- LocalStorage
- Media keys
- SessionStorage
- Service Worker registrations (I guess this means service worker caches)
[0]: https://webkit.org/blog/10218/full-third-party-cookie-blocki...
IndexedDB is also subject to the 7 day limit. Leaving no persistent storage for web apps at all.
Actually, I'd be even happier if any form of offline storage required explicit user permission anyway.
https://dexie.org/docs/StorageManager describes the StorageManager API which lets you prompt the user to allow your IndexedDB data to be stored more reliably. My first thought after reading this article was wondering if this would allow an exception to the 7 day rule... but then I remembered that Safari is the only "modern" browser which does not support the StorageManager API
lol, sucks for users of my client side JS video game!
Even offline storage that is only used locally? Say a game with savegames that has doesn't use online connection to play it.
Another example: a password manager.
Without being able to use localStorage as a long term store, I'll have to register for an account, have to deal with them handling my data, etc. Losing the functionality of localStorage as a long term store has disadvantages.
I feel the comparison of Apple with data companies such as Google, Facebook is by itself at fault. Apple like any computer company of 70's was not into data, just because Internet itself didn't exist at that point like it does now. 'Apple didn't choose to be in data' is projected as altruistic, instead of just a marketing ploy(they didn't choose, because it wasn't available).
Apple doesn't receive even the fraction of scrutiny Google, Facebook receive (which they should). e.g. iCloud hack, Apple's response to iOS vulnerabilities targeted by state actors, Newer Safari being incompatible with privacy extensions such as uBO etc.
Personally I feel good that Apple is not into data, just because I feel if they are into data; they might be more evil than Google or Facebook aided by their walled garden.
I simply see no technological solution to this problem, it'll always be a cat-and-mouse game, until governments catch up and makes it illegal.
I'm eager to hear if someone here does have any solution to this problem though.
It's compiled using LLVM, which also contains thousands of lines of open source code by Apple.
Of course you might argue that these examples don't prove your sweeping statement false, but please read https://en.wikipedia.org/wiki/No_true_Scotsman before arguing.
Yes.
> or as in this case was counter-intuitive for privacy
I fail to see how this is counter-intuitive for privacy.
> iCloud hack
Targeted spearphising?
> Apple's response to iOS vulnerabilities targeted by state actors
https://news.ycombinator.com/item?id=20897368
> Newer Safari being incompatible with privacy extensions such as uBO etc.
It's not incompetence. When you request a route, your iPhone breaks up the request into separate, unrelated segments so Apple doesn't even know your total route. They've done work to avoid tracking you.
Call it a "marketing ploy" or "altruism" or whatever, but the fact is that Google wants to know where you go, and Apple doesn't.
I was developing a feed reader that was supposed to be a client-side-only PWA but that's tricky.
Apple definitely makes it difficult to use them effectively. For example you need to use Safari on iOS in order to download the PWA - it won't work if you're on chrome or another third party browser.
Twitter, Instagram, Starbucks, Pinterest and more have PWAs as well.
> ITP 2.3 caps the lifetime of all script-writeable website data after a navigation with link decoration from a classified domain.
i.e. the 7 day timeout for local storage only kicks in if you've been redirected from a domain that ITP has classified as one that tracks users. So, for example, web apps that users navigate to directly will be unaffected.
[1]: https://webkit.org/blog/9521/intelligent-tracking-prevention...
why would you look at the old blogpost for the new behavior?
It's all web pages, regardless of classification or redirects. The new webkit blog post is quite clear:
> Now ITP has aligned the remaining script-writable storage forms with the existing client-side cookie restriction, deleting all of a website’s script-writable storage after seven days of Safari use without user interaction on the site
https://webkit.org/blog/10218/full-third-party-cookie-blocki...
Or straight from the ITP lead's twitter:
> Fifth, all script-writeable storage is now aligned with the 7-day expiry Safari already has for client-side cookies.
https://twitter.com/johnwilander/status/1242516001939324928
(with follow up replies on what resets the seven day clock)
I see the same "oversimplifying" in webkit's 2020-03-24 blog post linked from the original post. See "7-Day Cap on All Script-Writeable Storage" in https://webkit.org/blog/10218/full-third-party-cookie-blocki...
- Somehow goodsite.com's user ends up on evil.com
- evil.com redirects to goodsite.com?clickID=1234
- goodsite.com's storage gets flagged
So my guess is you are fine most of the time, except if you allow other sites to embed your content in their page. In that case, you should:
- provide the embed on a separate subdomain
- remove features requiring identification if the content is view embedded: attempting to use them redirect to the real site.
Otherwise ITP will mark your domain as tracking and wipe you after 7 days if your user don't interact directly with the site.
I have a hard time deciding if it's a good thing or not.
I guess it has the potential to be mostly a good thing, provided that:
- I understood it correctly, which I'm not sure, as their wording is not clear
- It's implemented correctly. Once the deal is done, it's in the wild years, fix or not.
- It's implemented in good faith. Apple wants to promote the app store and has shown to neuter web apps in the past.
I still have a strange bad feeling about this.
I don't think that's true.
I asked the head of Webkit dev on Twitter and he said:
> This time limit affects first-party storage
Could someone please change the title of this post? It's rather inaccurate and spreading FUD... legitimate offline web applications are not going to randomly lose their storage abilities in Safari. Tons of people read this (admittedly hard to follow) blog post quickly and then took a nose-dive into their own hot takes.
Hoping Webkit pushes another of these posts later to clear things up.
My app is an inventory control system used by businesses that build electronics (https://partsbox.com/). Deleting client-side data after 7 days is ridiculous. You can't assume that people will always log in every week, in small businesses or design/manufacturing companies there are times when 2-3 weeks can pass without building new hardware or touching inventory.
Some web apps already saw the danger of having an easily purge-able storage on the client side and simply implemented an export function for their tools. I admire those tools more than the ones who overuse local storage for everything.
One such tool is draw.io, a flowchart maker. You use the app, persist everything in local storage and when you are done, you export your project into a file, all happening on the client side. When you need to edit, you import the file on launch. It's portable, it's protected from browser bugs/decisions and imho pretty user (privacy) friendly.
If you put data in IDB, it will stay there for 7 days and then if it gets deleted the delta sync would just download it again.
Why is a PWA better from a privacy or security perspective than a native app?
I seriously doubt it. Apple has been undermining web dev for years.
With this change, you're essentially "logged out" after 7 days of inactivity.
This is pretty a bad user experience. I honestly am not sure how to mitigate this. MacOS Safari might not be a massive market, but iOS Safari is.
Any thoughts about how we should address this change?
I could be misinterpreting your comment but are you saying your keys are simply destroyed upon this “log out”? Then I’m not really sure why your platform was considered working in the first place, if it’s tied to a specific browser of a specific device and won’t survive a clearing of storage which any user can do at any time for a variety of reasons?
WebKit blog post from yesterday: https://news.ycombinator.com/item?id=22677605
Do you mean that you deploy a 'native' app that's really just a wrapper around a web view that would also be just Safari? Same policy applies, but now, you have the option, in native code, to siphon off data and put it into Real Storage.
(proceeds to not answer the question)
Found the answer: Progressive Web Apps[1]
1: https://medium.com/@amberleyjohanna/seriously-though-what-is...
It is a “Progressive Web App”. Sorry for the jargon usage without explanation. Basically it is a marketing term used to place some new web APIs and best practices into an umbrella of a “near native UX on a Web App”. What it usually means is that your application is:
* Served from a secure context (a requirement for the other APIs anyway).
* Has an application manifest (this contains metadata about your web app and is used by browsers and OSs to add icons, names, themes, etc)
* Has a service worker (which enables your application to potentially work offline beyond what other cache solutions did in the past)
So with these in place, browsers can offer a “Install this site and an app” feature which allows the site to open in its own window, with its own icon and name on the launchers and home screens.
Local storage should be treated as cache.. it may get refreshed.
What Apple did was fine. A backend isn't only for storage either.
Users clear their caches. They swap browsers. They swap machines. They use their phone instead of their desktop. They use private mode, or sand boxing. They re-install their OS. They buy a new machine.
Don’t be lazy. Using local storage without a backup is not acceptable.
And what kind of ‘progressive’ web app expects all the features in every client? Have we forgotten what progressive means?
Don’t be entitled. You are not more important than your users.
Then, if the user clears cache or changes computers, they lose the stuff they were following and have to wait for new items, but it's not the end of the world. They might even expect it if you name/describe the app a certain way.
E.g. if you download an app called "Podcast Downloader" that says it just downloads any new podcasts from feeds you follow for your later offline consumption on your current device - you might not expect a podcast on your phone to magically jump to your desktop without a re-download from the original site.
Seems like it could be a valid trade off if it lets a front end only web dev publish apps he couldn't publish otherwise because he can't/won't do backend. Storing user media on the backend is not cheap. The company I'm at has spent months of developer time moving over from Google to Amazon, for example, just for infra cost improvements that come from serving terrabytes of data off one instead of the other.
Apple didn't kill offline web apps. You can always add an interaction to your app which exports the stored data into a file which then can be saved by the user. It can be done entirely on the client side as well. If anything died here, it is the implicit consent by the user for allowing unnoticed storage space consumption. Implementing an export function will automatically make your app portable, which is always appreciated I believe.
Most data on local storage is some kind of structured tree, table or blob. All can be exported with only little effort.
HTML5 games -> Prompt user with a dialog to download saves/assets after they play the game for a while.
Productivity apps -> Detect "ctrl/cmd + s" to prompt a save dialog. Add save buttons somewhere visible.
Map like apps -> Do nothing. If the user is not visiting the map for 7 days, they don't need the map data persisted either. If necessary, allow explicit save with UI buttons for people who travel often.
Apps/sites which use local storage for auth related artifacts -> Notify users if they click "Remember Me" and explain them the caveats. Allow for encrypted save if users ask for it.
Kiosks -> Use Electron or a similar tech.
I am open to counter arguments. I don't have any idea about how mobile browsers behave for the scenarios stated above.
Edit: I use draw.io since last year and the experience there is as refreshing as it can be in this SPA jungle. I use it as a good example to learn from for my own web app projects.
Name a modern game that required you to manually manage game state files, let alone didn’t have autosave. It’s a feature users expect, and they’re going to have a bad time. I don’t want to play a quick game on my phone and have to remember to save and where I am keeping my save files.
I’d argue a far better options would be just to treat local storage as a permission like camera or microphones.
But... why? Drag the user through some dialogue to save a file locally / manage / be responsible for that and then deal with that whole deal? That seems like very... old / unnecessary.
The fact that applications store some random things locally to me is neither surprising nor a hassle. Browsers already cache files and etc. Unless I don't know something... LocalStorage and other non cookie options seem just fine / safe.
I get the concerns about cookies and such but this seems a step beyond what is needed into the realm of unnecessary / a hassle for the user.
Maybe I'm missing some bad patterns / dark patterns using LocalStorage and etc but it seems to throw them out with the bathwater.
apple should fix their safari bugs first before starting with this nonsense.
That's the problem, it won't work there. Apples support for PWA's is frustrating to say the least.
It's fair that you might need consent from the user before storing and keeping large amounts of data, but by removing the option you are forcing a bunch of developers to make a native app instead of a webapp which I find quite infuriating.
> If anything died here, it is the implicit consent by the user for allowing unnoticed storage space consumption
What about explicit consent? It also dies. That's just inventing problems.
> Apple didn't kill offline web apps.
Yes, they did. For an app to work offline, you need to be able to at least cache the app itself. If that gets wiped after seven days, you can't call your app "offline capable".
> If anything died here, it is the implicit consent by the user for allowing unnoticed storage space consumption.
What about the "implicit consent" that bandwidth is being consumed?
> You can always add an interaction to your app which exports the stored data into a file which then can be saved by the user.
That would be awful. Imagine being prompted to import your data every time you launch it.
Maybe that sort of works with document-centric apps that have no persistent settings, but even then it wouldn't be possible to integrate properly into the file system in the way users would expect (file assocations).
> HTML5 games -> Prompt user with a dialog to download saves/assets after they play the game for a while.
More like constantly reminding the user that their valuable progress gets wiped after seven days, should they make the poor choice to run the app offline.
> Productivity apps -> Detect "ctrl/cmd + s" to prompt a save dialog. Add save buttons somewhere visible.
Same as above, except the data might be even more valuable.
> Apps/sites which use local storage for auth related artifacts -> Notify users if they click "Remember Me" and explain them the caveats.
"I'm sorry, we made a decision to write an app with technology that, in hindsight, we shouldn't have used. Therefore, your user experience will now be more annoying. Thanks for sticking with us while we're rewriting the app!"
But that is not the deal, the deal is that they fear that more and more developers are moving to webapps instead of developing native apps that need to pass trough the App Store and thus be approved by Apple, and they don't like that.
It's annoying how far Apple is behind Mozilla and Google when it comes to progressive web app functionality, but I don't think their action is as user-hostile as is being raised here.
Pretty much any PWA that was using ipfs as anything but a caching/distribution layer is no longer viable. This is a huge blow to decentralization technology.
Sure, you can make a standalone app, but that is going to cripple already difficult adoption.
This sucks :(
Rather than wiping local storage/indexed DB data after 7 days, could you not just make it an opt in thing, like the camera or mic? For example, ask users "Allow myapp.com to store app related data on your computer?". If they allow it, then give access to local storage APIs, otherwise don't. That way users can still have fully local PWAs if they wish.
As an ardent PWA developer, this change annoys me immensely.
> Heck, they could even go further and ban apps from corporations like Facebook, Inc., and Alphabet, Inc., that have violating your privacy as the core tenet of their business model.
If Apple were to ban the Gmail app (and obviously block web access via iOS too because that would be a loophole otherwise), I would throw away my iPhone, swear off business with Apple, and search dearly for a way to sue them.
I don’t love the walled garden iOS represents, I merely live with it in exchange for great hardware and UX. If the bargain changes to be more restrictive, I would turn against it in a heartbeat.
Thinking about that, is no surprise Apple is striking out early to make web apps useless. If they wait too long, they will become entrenched, and people will feel like they have lost something if access is restricted. Apple really wants to jealously protect its control, and more importantly ability to take 30% tax of every transaction that they can perceive.
I have a copy of my “DAT Shopping List” demo I last opened about 6 months ago saved to my iPhone home screen... I opened it, and the data was still there. I’ll be really sad when I open it again after iOS autoupdates and the data will be nuked.
"After seven days of Safari use without the user interacting with a webpage on website.example, all of website.example’s non-cookie website data is deleted." (https://webkit.org/blog/9521/intelligent-tracking-prevention...)
Granted, this could turn out really well if the industry adopts another standard which requires user permission, overcomes this limitation, overcomes the existing limitation of LocalStorage on iOS getting automatically cleared when a device is low on storage, and overcomes the problem of sites being able to use up a lot of storage on users' devices without their knowledge.
I'd be very welcoming of such a standard. These could be good future replacements if the industry can adopt them:
https://chromestatus.com/feature/6428344899862528
https://wicg.github.io/kv-storage/
These technologies can be leveraged to improve usability. Unfortunately, advertisers and 3rd party trackers make it so we can't have nice things.
They threw LocalStorage and etc out with the bathwater that are cookies.
> "...But deleting all local storage (including Indexed DB, etc.) after 7 days..."
From the Apple announcement:
> Now ITP [Intelligent Tracking Prevention] has aligned the remaining script-writable storage forms with the existing client-side cookie restriction, deleting all of a website’s script-writable storage after seven days of Safari use without user interaction on the site. ...
https://webkit.org/blog/10218/full-third-party-cookie-blocki...
On the other hand, this direction keeps web devs honest: local storage, service worker, cookies and other script-writable areas are meant to be temporary.
Apple want to move you closer into their walled garden with the prospect of enhanced privacy.
The idea that iPhone apps are more private than web app because Apple must approve your apps is troublesome.
Other domains I'd actually prefer to be indefinite. I've got a notepad thing that uses local storage and doesn't store its data on the server. There's no excuse for deleting its data since its user data. Apple therefore has no permission to delete that data. Do I have a non-cloud workaround for that?
Good question. The definition of a "progressive web app" is vague. What they seem to mean is a web page which, once you visit it, is cached locally, and thereafter runs locally. The web page accesses various servers, not necessarily ones from the same domain as the web page. Persistent state, if any, is stored locally. The page gets its own icon on the home screen somehow, so it sort of looks like an "app".
Apparently "progressive web apps" are supposed to have a browser service worker so they can get notifications pushed to them from somewhere, although it's not clear why that's essential. That would seem to depend on whether the function performed requires being notified of something happening elsewhere.
Apple apparently dislikes this because they don't get to force people to use their store, with their big cut of the revenue.
Is that about right?
Does this only apply to pages read through Apple's browser, or does it impact Firefox, too?
This is part of the motivation. The other is advertisers using persistent local storage to track users [1].
[1] https://clearcode.cc/blog/alternatives-to-cookie-tracking/
Progressive Web Apps are strictly defined:
1. The app has an app manifest describing metadata about the web app, enabling it to be treated like an app (e.g. it can be installed)
2. The app has a service worker, enabling it to work offline like a native app.
3. It's served over HTTPS.
Those are the 3 technical requirements of a PWA.
There's also the philosophical direction of Progressive Web Apps: they're progressive, meaning they offer the app's essential experience no matter the device, but enhance progressively based on the device they're running on. That is, more capable devices let the app offer more functionality without blocking out users on lower-end devices.
This applies to WebKit, but if that decision sticks Mozilla might follow. Who knows... I hope not. Also be aware that Firefox on iOS is WebKit.
Sorry, but no way.
To violate privacy (aka enable tracking) a sub-iFrame could be set up that uses "local storage" with a parent page security policy that allows communication across the iFrame boundary. Sorry, yes, I am being a bit vague.
Who cleans up ~/Library/Safari/Databases? I personally see crud in this directory from 2011 that has been migrated from older systems.
Almost not relevant now, but Flash also had a "local storage" system that was shared across all Flash Apps. It also allowed (before sandboxing) local apps to proxy and communicate (via shared memory) with any standalone Flash App on the system through any page that used the Flash plugin -- i.e any running web browser, violating all attempts to have web compartmentalization rules.
Yes [1].
[1] https://clearcode.cc/blog/alternatives-to-cookie-tracking/
We then link them to a $30-$50 Android device that they can buy on Amazon and use as a second device to use our services "if they are interested in a more powerful web experience". We provide a basic version to all users, but put a shamewall for advanced features. Best use of our time and resources.
It is time to push back, stop making Apple's problems your problems. Educate people without ranting and offer them solutions, developers have the bad habit of trying to cover up this kind of non-sense and taking the blame while really Apple are the ones who should be ashamed. If people love your product/service getting a $30 phone to be power users and make their life easier and their experience richer will not be a big deal for them. It's all about educating them the right way.
There is a lot to love about Apple products outside of a few safari restrictions. They're not perfect but better than a lot of alternatives.
That’s a genuine question by the way. I’ve been frustrated by Apple’s reluctance in the past but since they implemented Service Workers things have gotten better. I still really wish they had Web Push but I do understand at least conceptually why they’d be hesitant.
So you're suggesting shifting the development costs of you building a native / cross platform app directly to your customers? Does this work?
Of course, if everyone did the same, people would start to realise the problem might be with Apple, but the chances of all (or most, or even many) big web services deciding to alienate such a large portion of their (potential) customers seem slim.
My point is that Offline Web Apps (i.e. PWA) that are installed on user's desktop should have a bit more permissions than websites but people in charge(google, apple etc) seems to think otherwise.
https://discourse.wicg.io/t/proposal-full-network-access-in-...
This doesn't "destroy" the PWA ecosystem. Just makes a user's intention explicit when they save a PWA to their home screen, rather than continuing to use it within the browser.
From the WebKit Blog (https://webkit.org/blog/10218/full-third-party-cookie-blocki...) "Web applications added to the home screen are not part of Safari and thus have their own counter of days of use."
A little LocalStorage isn't going to hurt you.
Cookies I get, but I don't know of any dark patterns with localstorage / the benefits are pretty great.
(it does on android)
I get that controlling the walled garden is apple's mobile strategy now, but this is costing developers so much blood sweat & tears.
Both xcode and android studio are heavy + horrible compared to web, and the fact that you have to use both tools to release at scale makes them worse. Shopify wrote a dev post a few months ago saying 'we're react native as much as possible now' and claiming it makes life easier, but react native is worse than PWA because you still have to build for mobile 2x and deal w/ app store nonsense.
If PWAs supported push on ios, with or without cookie expiration, they'd be the preferred launch strategy for most non-game apps.
I’d be happy if Spotify gave me an API key and essentially went away except for a monthly bill.
But software has to be a product the masses get first to get made in our world.
I’m glad some folks are having their itch scratched but free streams are more than enough and I can wrap them for consumption as I choose.
Once again building your life around importing someone else’s priorities turns into an exercise of despair from not learning how reality doesn’t stand still no matter how hard you hope it will this time.
A local storage is the only way webapps can store any data in your computer (other than asking you to manually load / save some configuration file). Not all webapps can afford cloud storage for all user.
If Apple provided an alternative this would be ok. An alternative such as the native file access API (still a WIP). Or a prompt so that the user can allow long-term storage. Or supporting the web app manifest so that users confirm they want to "install" a web app, granting it greater permissions.
But they've offered no alternatives here, that I can see. They've determined that client-side web apps are simply not important.
For webapps that keep a session token stored locally, this will be inevitably wiped, so users will have to re-login after that time. I can already hear the complaints coming. Should devs now build a back end just to keep the token, and connect there with a cookie?
I worry that 7 days is too short of a period even then, but I do agree indefinite local storage does not make sense in most cases.
Perhaps using something like this: https://developer.mozilla.org/en-US/docs/Web/API/Credential_...
Anyone know of other Web APIs that could be used?
Forgive me, I’m a long time Android user, but do a lot of people choose to use safari as their main iOS browser, or are the usage numbers inflated because of the vendor lock in?
A browser plugin might be one way to achieve something like this. Personally, I really don’t care about the data my feed reader has, so I wouldn’t mind even public data storage backends, like gist. Or steganographically encoding my list of feeds and uploading it to porn sites :)
I can understand Apple's decision to do this, as there's a lot that can be improved about offline storage on the web:
* asking for user permission (i've seen demos try to exhaust the users' storage, and trackers can use this to invade privacy) * async writes and reads
However, making a change like this with no suitable alternative leaves PWA developers stuck in a hard place. I'm not sure what can be done in the short term here.
There's a few web specs that address these issues. I'd love to see them come further along, and maybe improve things for developers and users in the long run. If anyone knows, is there anything that members of the community can do to support these efforts?
https://chromestatus.com/feature/6428344899862528
https://wicg.github.io/kv-storage/
I noticed a text editor I bought from the Mac App Store, iA Writer, includes silent spyware that transmits your activity back to the developer without notice or consent (thank you, Little Snitch). Apparently, I "consented" to this in the Mac App Store ToS (right).
When I left a negative review on the app, their response was "we aren't doing anything not permitted by Apple in the App Store".
I don't use App Store apps any longer, and I take most of what Apple says about privacy with a huge grain of salt.
PS: OSX phones home to Apple in about a dozen different ways even with iCloud entirely disabled and all reporting/telemetry/feedback options turned off during the OOBE/setup. Try doing booting a fresh install of macOS with Little Snitch, but disable the Apple/OS exemption in Little Snitch's rules. I was astounded. Dozens of things.
I wonder if there's any major, widespread GUI OS in a default configuration that does not transmit to your ISP and third parties (including government snoops) when you open a local text file to write. I block all of these requests; most do not.
I am reminded of Winston Smith's paper journal.
If this is done for privacy's sake, that is.
No matter what browser vendors do, it will never be enough for "privacy" activists.
I can easily go to the settings area and delete my entire browser cache (Remove All Website Data), in fact if you are running low of space it even tells you to do it.
Why are people assuming things stored on a browser are a good place to store things. Nothing stored on a browser should be assumed to be forever.
All apps that you download from App Store can live offline, where they're usable without Internet or trusting some faraway web server.
You can't make a web app that can do that, and to some people it smells like Apple trying to force developers to release through App Store.
But if you have to wait every time your last interaction is more than 7 days ago, the whole experience will change. And supporting a reliable offline experience will be very hard to build.
There is no hard limit on how long things will be stored. Data in localStorage might still be stored for weeks/months/years, as before.
The only limit is on how long things will be stored if the user does not interact with the site/PWA.
If you are a website, not a natively-installed app, that I haven't "used" in a first-party sense for 7 days or more, I don't think your data belongs on my device.
Storage space can be limited, and any app I haven't used in 7 days should be happy to re-fetch my data from a server or convince me to install their native app.
To act like this is some nefarious plan by Apple to get people to build native apps instead of PWAs is absurd. If a PWA was written properly in the first place, this change will have basically 0 impact on it.
The more we use these, the more likely the APIs are to be fully implemented (and hopefully have features added to them).
I think they just want to push people towards native apps so they have full control. Apple always wants control.
First they prevented plugins like uBlock, made it very expensive to create your own plugins, and now they're messing up LocalStorage.
It editorializes away the point of the post, which is that, according to the author, "Apple just killed offline web apps while purporting to protect your privacy [by forcing WebKit to delete all local storage after 7 days]."
If this encourages more apps to go the native route, we've done more harm than good. Apps can gather a lot more data than websites, such as the dreaded contact list access.
There is no change without applying pressure at Apple. If this is important, we must speak about it, all of us. And yes, I understand that some people feel that this is not important for them, that is OK, we have different values and understandings, but if you have an opinion about this, please go out and post to your blog, dev.to, medium, whatever, but post.
edit: 17 minutes after posting this comment critical of moderation, I am unable to submit a new story. Coincidence?
I was confused as to why the page in question had changed, but I realized it was moved.
Why Apple is not facing antitrust charges for not allowing competing browsers on their platform? Microsoft didn't SHIP competing browsers, but allowed them to run just fine on windows, and was fined nonetheless, but Apple somehow gets away with not even allowing competing browsers at all!
I'm not from the US, so maybe I'm missing something about these antitrust lawsuits. Can someone please explain?
1. Apple is not a monopoly player in the app market.
2. Microsoft's antitrust fine was for forcing OEMs to not include any competing browsers (Netscape) on threat of losing special pricing.
If tracking companies cannot use cookies they can use JS and local storage instead. Then they can keep tracking people for long periods.
So, in the escalating war Apple alters local storage so that non-use for more than 7 days doesn't keep data along. It becomes less valuable for use with tracking.
The trade-off is that offline web apps become less capable and some use cases go away (e.g., completely offline).
Which trade-off is better for whom and in general? I've not thought to know. But, the trade-off is worth pondering. Whether we agree with Apple or not.
(edit Turns out that's not quite right, see diggan's reply.)
From the article:
> You’d almost think they had an App Store to promote or something.
There's certainly a tension here. I'm still not sure why more vendors don't make iOS PWAs to get around the App Store payment rules.
Perhaps related: Very roughly a year ago, something changed in iOS that broke the 2048 PWA. Its swipe-detection no longer works. A pity.
Because users won't use them. For users that don't have a technical background: if it isn't in the app store then it essentially isn't an app. For techie users: lots of us don't want web apps because of the power, memory, and bandwidth usage is often higher than a well written native app. The fact that there's a gatekeeper who has some control over what shows up in the app store is usually a feature and not a bug.
If there were big parts of the app ecosystem that didn't have native apps, then eventually users would find web apps. But that isn't the case. Think of anything and search for it in the app store and there's an app for it (including 2048).
Similarly, an offline capable web app is not necessarily a PWA, as PWA carries a lot of features to it besides being offline capable.
One reason is because Apple have incentive to break PWAs and they will do it. It's not a wise business decision to act against big player.
Seriously, you should browse the web for a bit and see just how many "client side PWAs" you've used/installed, vs how many tracking identifiers have been installed.
Why? Why isn't the case that the code which runs Electron, and library code JIT-ted by Electron can't be reused by other processes on the same system?
Of note, John’s replies also mention this policy does not apply to WKWebView or UIWebView, because they lack ITP.
https://twitter.com/johnwilander/status/1242882202301427712?...
Gaia is one example: https://github.com/blockstack/gaia (I've worked on Gaia so I'm biased but there are other such decentralized options as well.)
PWA is about the web.
I have already stopped building native apps because the App Store process is so painful.
I remember when Edge/IE was crap, I put up a couple of banners that Firefox/Chrome/Safari are officially supported browsers and people did move away from Edge. Had <1% of traffic from there.
First of all, the various kinds of browser local storage have always been volatile. It has always been a bad idea to treat it as permanent storage. Maybe it's a little more obvious now? Not exactly a bad thing.
> the PWAs I was building here might just be dead for iOS users
If so, it was already dead for your users, whether you realized it or not. I guess you were going to implicitly promise something you could not deliver: that your PWA would keep track of the feeds the user was subscribed to (and perhaps also keep track of what had been read, and other user state). But you were going to screw your users, because a PWA without external persistent storage could not do that reliably. It's really luck for your users that this caught your attention and has you rethinking your app.
A partial list of things completely external to your app (not including this change) that could cause your users to lose things important to them that you stored in various local storage...
* user switches browser
* user has multiple devices
* user upgrades phone (or tablet, or workstation, or laptop)
* phone (or other device) goes in for repair or upgrade
* major change to browser (like Edge moving to chromium)
* some OS updates
* user clears browser data (as innumerable troubleshooting processes suggest)
For the past couple of years I worked on an education app where users are 90% of the time offline. Users can remain offline for weeks. There the is no reliable internet in most of the schools in Mexico.
I don't work on that company anymore but this is going to be a massive headache.
It does seem that Apple intends to cripple web technologies in order to move developers to their native platform but this will likely do more damage to privacy than anything. All of the alternatives to local storage for simple mobile apps typically involve moving data to a third parties like Firebase, AWS, etc.
Simple apps that didn't need a server and could just keep data or user-preferences locally would now need to either create their own data service or pay for a BaaS which means moving your data out of your control.
This behavior leads to companies like Under Armour to house data they shouldn't have and puts everyone (150M people) at risk.[0]
[0] https://www.wired.com/story/under-armour-myfitnesspal-hack-p...
Most apple apps are privacy hogs which don't have any way to turn off tracking. In apps, Apple created a prison which noone can question and everyone will allow them to do all abuse. Look at Apple News.
LocalStorage is not a substitute for an actual database, it’s a cache. The problem with the author’s technique is that privacy minded users clear their browsers from time to time, so they would be inadvertently clearing data they actually wanted to keep because who uses LocalStorage as a persistent data store? Sure it could be used like that as an “off label” use, but generally it’s used to cache what is persistently stored elsewhere or used as a means to avoid multiple network calls in the process of doing something (such as saving calculations, the results of which would be eventually persisted.) Local Storage should be used as if it were a session store rather than something persistent.
If so, assuming the application is used more than once every 7 days, this seems like less of an issue.
"Privacy is a fundamental human right."
Apple will do whatever it takes to protect its closed ecosystem, and if that means killing PWAs built with open web technologies they'll provide any dubious excuse to justify it (security, privacy, blahblah). They did the same back in 2010, killing a perfectly valid app platform that was picking up momentum, but they didn't control. A platform that was 5-10 years ahead of the "open web".
Looks like this time they won't use HTML5 as piss-poor excuse.
- They have a News app
- They haven't rejected apps from Google and Facebook
Can you imagine what would happen if Apple rejected apps from Google and Facebook? Can you even fathom the outcry?
Apple News uses differential privacy and doesn't track user history, but yes, they do provide personalized News I guess? They must not care about privacy at all then!
If you're upset about a seven-day limit on local storage, okay. I get it. It sucks. But to claim Apple's reasons for this are invalid because they allow Facebook apps to exist, that's... weird.
Sounds good to me, I don't want websites turning my browser into a p2p node :)
I'm 100% on team privacy but this isn't the right way to do it.
https://battlepenguin.com/tech/rss-the-original-federated-so...
Interesting he ran into the CORS situation with PWAs. It makes sense. It feels like even PWAs aren't that far off from Electron. Sure you're not launching another browser and can share a browser engine, but you hit other limitations.
I'd rather have a real, lightweight, stand alone app most of the times honestly. I wish people would write more stuff in Qt5. You can bundle Python+PyQt5 together for a reasonable licensing fee. A great example is the Resolve color/video editor is written in C++/Qt5.
Your offline app should ALWAYS sync to the server whenever possible. The only bad thing I can see here is that if you can't upload the data in time and the user then doesn't use your app for 7 days, he will lose what he last worked on, but such is life and why you should rather use real apps. Offline apps needs to work differently, they need to get permanent storage just for that app but only if the user explicitly choses to install it like that. Not every random page should get permanent storage on your device. This is the right move, Apple might just lack an alternative for apps you actually chose to "install permanently" ;).
Rational readers click back and move on. You end up just preaching to the choir.
This particular complaint is paradoxical because Apple birthed web apps, and has done more than anyone to make them a reality. Unfortunately they remain a very rare beast -- extraordinarily rare -- and are dwarfed by the privacy concerns of people using iOS just to browse. So the team dealt with that. Seems a fairly obvious pros and cons analysis.
Maybe they'll add an exception for installed to desktop webapps.
EDIT: Clarification from a Webkit dev https://twitter.com/alexcroox/status/1242559843354972161
The new change to ITP is here: https://github.com/WebKit/webkit/commit/4db42c1571d821572ea9...
The cookie filtering logic specifically is located here: https://github.com/WebKit/webkit/search?q=filtercookies
The file:/// handler is implemented here: https://github.com/WebKit/webkit/blob/master/Source/WebCore/...
(I don't have anything to do with Apple or WebKit.)
The company is just too big and not working in unison.
The Apple Safari Team is killing/hurting offline apps. The author asks why they don't take the same approach in Apple News - as if it is the same team that is in charge. Different team with different priorities and likely not talking to each-other.
I think the larger point is valid - but it better to understand that this isn't some cohesive cross-company strategy at play. Its size-able teams working on their own priorities within a larger roadmap (presumably).