LocalForage: Offline Storage, Improved | Hacker News Reader

LocalForage: Offline Storage, Improved

hacks.mozilla.org

113 pointsrnyman20 comments

Not sure how this compares to the myriad other similar libraries that have been around for a while, like Lawnchair? That would have made a good section in this blog post.

But the real problem with offline storage is that Apple refuses to support IndexedDB, when every other browser vendor implemented it years ago (even IE!). My question is, why? I can think of three potential answers:

1. They just don't care about the web. (But in that case, why raise such a stink about not letting other web browsers run on iOS?)

2. They are incompetent. (But in that case, how have they made so much other great software? And IndexedDB isn't even very complicated. Even if it was, they could just copy Mozilla's or Google's open source implementation.)

3. It's a conspiracy to cripple HTML5 apps and encourage people to write native apps. (I don't like conspiracy theories, but it's hard to come up with a better explanation than this...)

Or: it's low on the roadmap and it's not something that markets well, since the non-dev user doesn't care about IndexedDB support.

Occam's Razor applies here.

I guess that's possible, but then why did Google, Microsoft, and Mozilla all decide it was important enough to implement years ago? It's not like this is some controversial political thing like DRM, or some huge technical challenge that would require a lot of resources - it's just a simple database API. Even if they ignored the multiple existing open source implementations, they could build their own very easily. And they've had years to do it.

Incidentally, I've been trying to make non-dev users care. I wrote a semi-popular HTML5 game that runs everywhere except iOS because it uses IndexedDB. When people complain that it doesn't run on iOS, I implore them to direct their complaints at Apple for not supporting IndexedDB. AFAIK nobody has even gotten a response.

I agree that the absence of IndexedDB is a little weird, but I don't know if it's "crippling," just annoying. Safari has supported Web SQL for a long time. While it was (rightly) abandoned by W3C because it was attempting to standardize SQLite as an implementation, there is a polyfill (https://github.com/axemclion/IndexedDBShim) that implements IndexedDB semantics on top of Web SQL. It looks like localForage also supports Safari using Web SQL.

The polyfills (there is another by Facebook in addition to the one you linked) are slow and buggy (look at all the open issues, or try using it in a non-trivial project yourself). Web SQL in Safari is also slow (check any performance benchmark) and buggy (see the huge bugs introduced in the most recent version of Safari). So the truth is probably closer to "crippling" than just "annoying". Who is going to go through the trouble of working around all the polyfill and browser bugs, only to be rewarded with horrible performance? Not many people except those with simple needs. Making something complex? Better put it in the App Store.

Libraries like localForage are great if you're doing very simple stuff. But if you need to use slightly more advanced DB features (indexes, transactions, etc), they don't work. IndexedDB, while not a panacea, does generally work for that stuff... except for Safari :)

It not marketing well isnt the simplest answer, it markets as well as every other html5 feature that they have implemented.

Given they already have a working implementation just waiting to be merged, and webSQL is both dead and their implementation horribly broken, I also cant come to any other conclusion that Apple have little interest in their devices being used for advancecd web content

re: lawnchair

Would have been nice seeing as it solves the same problems in a very similar way. I would rather have one good implementation than two buggy ones.

Mozilla was involved with the standardization of the LocalStorage and IndexedDB APIs. If this is the API that users really want — and I'd argue that it is — I wish they would've realized it sooner and lobbied to standardize a simple, asynchronous, variable type key value store.

One major goal for IndexedDB was to be low-level enough that people could build the APIs they want on top of it. This type of design is popular these days due to some older standards like appcache and localstorage which have nice, simple, usable APIs but are not flexible enough to handle many real world situations.

(Now, one could argue that IndexedDB is still not low-level enough... but that's a separate problem.)

> One major goal for IndexedDB was to be low-level enough that people could build the APIs they want on top of it.

IndexedDB is neither low or high level, it's just worthless. I'd like to be able to use HTML APIs WITHOUT a framework. At least WebSQL was usefull. One could build an IndexedDB on the top of it if one wanted,or use it directly, but you cant build WebSQL on top of IndexedDB.

Thank you Mozilla.

It's a bit much to call IndexedDB "worthless". You can asynchronously store/access large amounts of indexed JS objects. That's a huge win, despite the fact that I agree the IndexedDB API is not ideal.

you cant build WebSQL on top of IndexedDB.

I don't see why you couldn't build Web SQL on top of IndexedDB. Nobody's done it and it would be a ton of work, but it should be possible.

I'd like to be able to use HTML APIs WITHOUT a framework.

You can use the IndexedDB API directly. It's not that hard. It's just overkill for something like localForage that is only doing very simple data storage and retrieval.

Thank you Mozilla.

It wasn't just Mozilla. Microsoft deserves much of the credit/blame, since they were also in the anti-Web SQL and pro-IndexedDB camp. And Google quickly came to agree with them, even though they had implemented Web SQL previously.

It's a nice API, but I don't think it's the API everybody wants.

Simple key-value store is quite useful in most cases, but it's not enough if you want to build really advanced applications, e.g. when you have a dataset that doesn't fit all in memory at once, but you want to filter/sort/search it (imagine GMail that stores e-mails client-side).

IndexedDB primitives are powerful enough to be an engine for a SQL database (and I presume was the idea given it's been created as the WebSQL killer).

I agree that it's not the only API people would need, but as a high-level API it's loads better than the localStorage API.

Nice! Good to see others tackling this sort of problem.

I'm actually working on a similar project, but it instead uses the file system abstraction for storing and retrieving data. It's called BrowserFS [0], and it emulates the Node file system API.

I haven't written an IndexedDB or WebSQL backend yet, because I've been focusing on the architecture and other problems recently. But I'd appreciate any comments people have on it. :)

[0] https://github.com/jvilk/BrowserFS

Interesting, this could even help to integrate same codebase using the chrome.storage API on Chrome Apps:

http://developer.chrome.com/extensions/storage.html

Still, it carries the normal limitation of key-value only storage in the sense that you have to keep track of your keys yourself. No queries are possible.

It would be great to have a low-level binding to LevelDB in the browser for developers who want to implement their own indexing and transactions. I have been working with WebSQL and IndexedDB ever since they launched and performance is often an order of magnitude slower due to design decisions taken by browser vendors.

Very true. I have been doing some wacky custom DB stuff using Python and LevelDB (for very strange indexing requirements, and using the Basho variant of LevelDB) and it's superbly fast, predictable and easy to use, at least until you get to about 100m k/v pairs!

The horrible binary manipulation capabilities in JavaScript is a serious headache for any kind of serious networking or storage interfacing.

To chime in with a "me too": I ported an app from Python/SQLite to JS/IndexedDB, and I also noticed about an order of magnitude decrease in performance. I think it's mainly because you have so much less control over how/when you read/write from the hard drive.

While localStorage is blocking it offers A LOT better performance than IndexedDB, at least based on my experience.

There's also some benchmarks here: http://jsperf.com/indexeddb-vs-localstorage/6

That's because most localStorage implementations are in-memory. Chrome and FF (as far as I know) have to synchronously load all your localStorage data from disk at page load and then flush changes to disk later. This increases your page load times dramatically if the user has a spinning rust hard drive. (This is part of why localStorage has a ~2.5mb cap in Chrome and a ~5mb cap in Firefox.)

So, in practice you're benchmarking actual I/O vs in-memory data structure updates.

Reply on news.ycombinator.com