There is no simple solution for local storage | Hacker News Reader

There is no simple solution for local storage

hacks.mozilla.org

73 pointsnthitz20 comments

> In order to appear snappy, web browsers load the data into memory on the first request – which could mean a lot of memory use if lots of tabs do it

> localStorage is persistent. If you don’t use a service or never visit a web site again, the data is still loaded when you start the browser

That can't be right can it? I can see the data always being on disk until explicitly cleared, but if a browser is loading it all for all sites from disk on launch I'd call that a significant memory and I/O management bug in the browser.

Even stranger is their conclusion that key/value is certainly slower than the full-fledged SQL database. I just can't be. Somebody doesn't understand something basic, and if it's I, please make me see.

This came up in a prior localstorage discussion as well, with a participant linking to -

https://blog.mozilla.com/tglek/2012/02/22/psa-dom-local-stor...

Big Mozilla fan, but they need to stop criticizing APIs for implementation details (whether WebSQL and now localstorage).

I also do with people stopped saying localStorage is 'too slow', its certainly not a fully fledged database, but its almost perfect as a cookie replacement, where you werent storing 5MB of data in cookies anyway.

I spent some of this weekend playing with IndexedDB, its very hard to get into at the moment, mostly due to the different implementations across mozilla / chrome, but there is some complexity that doesnt look easily solved by a wrapper, just understanding what is meant by onupgradeneeded / setversion seems needlessly complex.

Other problems include "Indexed DB is not available in private browsing mode", how are people supposed to build offline webapps when the applications can easily be prevented from accessing their own data?

IndexedDB is nice, but its quite opinionated and complex, I think browser vendors needs to think about what lower level constructs can be given to developers in order to create their own databases, and as much as I dislike the thought of SQL client side, I think it was probably a mistake to shoot websql so early on, sqlite is the standard of embedded databases.

> I spent some of this weekend playing with IndexedDB, its very hard to get into at the moment, mostly due to the different implementations across mozilla / chrome, but there is some complexity that doesnt look easily solved by a wrapper, just understanding what is meant by onupgradeneeded / setversion seems needlessly complex.

This is definitely an annoyance but it is pretty easy to overcome:

  const DB_VERSION = 1;

  req.onupgradeneeded = function(e)...

  req.onsuccess = function(e) {
    var db = e.target.result;
    if(db.setVersion && Number(db.version) !== DB_VERSION) {
     var dbReq = db.setVersion(DB_VERSION);
     dbReq.onsuccess = function(e) {
       req.onupgradeneeded(e);
       req.onsuccess(e);
     };

     return;
    }

    // You can use db now.
  };

> Other problems include "Indexed DB is not available in private browsing mode", how are people supposed to build offline webapps when the applications can easily be prevented from accessing their own data?

Good point, are there any queues to the developer that the browser is running in private mode? If so I think we just need to inform the user.

> IndexedDB is nice, but its quite opinionated and complex, I think browser vendors needs to think about what lower level constructs can be given to developers in order to create their own databases, and as much as I dislike the thought of SQL client side, I think it was probably a mistake to shoot websql so early on, sqlite is the standard of embedded databases.

There are going to be a lot of IndexedDB libraries once it becomes supported (in the modern spec) by all browsers. There will be libraries that give you a simple key/value store like localStorage does today (I'm working on one of these), there will be libraries that let you write SQL, and probably every other paradigm that people enjoy.

I think you are confusing private browsing with offline mode. Private browser is when the user specifically doesn't want the application to have any access to previously stored data, and doesn't want the application to store anything either.

I wasnt confusing private browsing / offline mode, and I do understand that private mode ensures that data is temporal, but disabling the feature, instead of just cleaning it up when the use leaves private mode, just means another edge case for developers to have to handle, or for them to just ignore private mode, which doesnt make sense for a lot of 'desktop' like apps in the browser

Are you saying that in private mode you can't use index db at all? If so, then yes I agree with you.

That is what the mozilla documentation suggests @ https://developer.mozilla.org/en/IndexedDB/Using_IndexedDB

"Obviously, browsers do not want to allow some advertising network or malicious website to pollute your computer, so browsers prompt the user the first time any given web app attempts to open an IndexedDB storage. The user can choose to allow or deny access. Also, IndexedDB is completely disabled in the privacy modes of browsers (Private Browsing mode for Firefox and Incognito mode for Chrome). The whole point of private browsing is to leave no footprints, so attempting to open a database fails while in this mode."

Testing it in chrome it looks like it is working fine, I should know better than to trust the docs :)

This article is only about localStorage as a cache for performance optimization, and fails to signal this clearly enough. If, instead, you want to use it to store things locally (for example, persist a blob of data across page reloads), I assume it could be a fine solution.

There also seems to be something not quite pinned down about the argument that localStorage hits the disk on every access, and is therefore slow, especially since another argument is that if you "never visit a web site again, the data is still loaded when you start the browser." I'm a performance-conscious web developer, but is it really on me to carefully keep the browser from hitting disk? It's possible but it sounds suspect.

I was skeptical about the same argument. Both indexing and virus scanning are non-blocking processes that shouldn't impact page load time. And I'm not sure what he means when he says "for an end user on the web they could mean a few seconds of waiting during which the web site stalls"? Filesystem caching isn't something specific to developers' boxes...

On Windows, lots of virus scanners hook into the vfs and (at least a few years ago) sometimes do terrifying things like check every access. This can really, really slow down i/o.

That doesn't sounds like a good enough reason to redesign a web API, especially if it means making it several orders of magnitude more complex. Stuff like that is clearly a problem, but it's not a browser problem.

By the articles own admission, the problem of the storage being persistent can just be solved by doing sessionStorage instead of localStorage.

I was just analyzing websites recently to figure out how they were not sending any data back to their servers while still having AJAX-like responses. We are doing AJAX and the amount of data we are sending back causes 2-3 second delays. Similar sites have accomplished it almost instantaneously.

What I found was that the sites I checked used localStorage, or at least from what I could tell from the minimized javascript. It seems like most of the established players have concluded that there is no better alternative and that they will just need to hack together something with localStorage: which brings me to my request:

Perhaps someone can elaborate not only on the pitfalls of localStorage/sessionStorage, but the best ways of working around them?

Yes there is!

Http://github.com/alexmng/sticky

Sticky is a key/value store abstraction for localstorage and indexedDB.

Yeah, that's too bad. It probably makes sense to change the spec to async; it's (kind of) easier to have the browsers change than the many web apps. You could have the document onready wait for load to finish, I imagine.

That said, if we are going to shift to another API, we should do it now and loudly. I'd prefer a database solution; people who are troubled by the complexity will use a good wrapper.

localStorage may be slow for read/write, but it's very easy to mitigate the problem by doing memory caching and delayed writting. Also storing file in localStorage is a bad idea. There are better way of caching file (offline storage, expire headers, etag, etc.)

localStorage has a lot of space for improvement, but I don't think they should stop to advocate it. It just feel like going backward.

It's pretty ironic getting an invalid security certificate for hacks.mozilla.org (yeah, I'm using HTTPS everywhere). Somehow, it's using the cert for a different subdomain.

I'm getting the right cert.

It appears to be working now. I got a notification that the cert had changed since my last visit, so they probably fixed it.

Reply on news.ycombinator.com