What’s new in IndexedDB 2.0? | Hacker News Reader

What’s new in IndexedDB 2.0?

hacks.mozilla.org

56 pointsdwaxe30 comments

I still feel that the api is a lot more complicated than it has any right to be. WebSQL for example was nifty little thing that worked just fine as far as me as a dev is concerned. I realize there must have been considerable issues with it from implementation POV but IDB seems so much worse (to me at least).

It wasn't so much of an implementation issue as it was politics around the spec that killed it: As WebSQL basically happened by just exposing a SQLite runtime to JS, there was never a clear spec of what WebSQL should support (no - "it just supports all that SQLite supports" is not good enough).

And even if somebody went in and specified all of SQLite's features, then there would still be the issue that any accepted web standard needs two independent implementations, meaning that one of the browsers would have had to clean-room reimplement SQLite.

And finally, there would be the question of what to do as SQLite involved. What would browsers do as SQLite gains features or changes behaviour? Keep the old version of SQLite around for JS? Freeze the whole browser on the old version of SQLite?

I agree though that WebSQL was much more convenient and useful than IndexedDB.

IIRC, the idea was making IndexedDB low-level, giving way to the library authors to create something easier to use.

I'm not sure if that ever happened.

Dexie.js [1] is doing that and is one of the most undervalued projects I work with.

[1] http://dexie.org/

PouchDB[1] sits on top of IndexDB (or WebSQL and other methods) and exposes a CouchDB-like interface.

[1] https://pouchdb.com/

But that's yet another dependency to load that is not written in C++/C, not good enough.

It's client side JS, why would it be written in C++/C?

Neither WebSQL or IndexedDB are written in Javascript themselves.

I get the need to standarize properly WebSQL, but the two different implementations "rule" is often ignored in the implementation of crypto, compression, image decoding, etc. where different browsers often share code with a common ancestor (or just identical).

If SQLite gets new features you do the same thing you do with other specs. If it makes sense to expose them, you update the spec.

But there are multiple competing implementations of GIF, JPEG, and thousands of other image file formats, and the browsers vary in exactly what code they use. Browsers actually DO use multiple TLS/SSL stacks (NSS, BoringSSL, OpenSSL at least), and multiple implementations of tons of protocol components (like compression) are around as they need to test things like performance. Multiple implementations of e.g. WebRTC exist. You could replace any of these components with a completely different implementation with relative ease, because other implementations do, in fact, exist.

Furthermore, outside of the TLS case, most of those are still within the ballpark of reason -- feasible to reimplement even in a small group. That's definitely something to be considered -- it's not just a case of if it's possible, but whether at all it's even realistically feasible for most people to reimplement.

Replacing WebSQL is on a completely different level of difficulty compared to these. Not only is it massively more complex to implement, there really is only one implementation, so it's pretty much impossible to reimplement exactly to spec, with the same kind of performance/usability characteristics -- without just doing a clean-room analysis of SQLite.

> If SQLite gets new features you do the same thing you do with other specs. If it makes sense to expose them, you update the spec.

It's never that easy. What happens if SQLite changes the way a feature behaves, even for good reason? The developers aren't beholden to browser devs, so this is within their right. What happens if they don't change something and a bug is then "enshrined" as part of the implementation -- or what if someone wants to make a change like, I dunno, actually checking the types of values you insert into the DB?

I do think WebSQL was promising and it does suck it was axed. But I think it was axed for a fairly sound reason, even if humans aren't always perfect robots who make exactly perfect decisions in similar instances (e.g. a similar difficulty argument could be made against TLS, although we're already waaay past that point).

> What happens if SQLite changes the way a feature behaves, even for good reason? The developers aren't beholden to browser devs, so this is within their right.

Actually, we are beholden to application developers of all types. It's part of the SQLite social contract. SQLite does not change in incompatible ways. New features are added, performance is enhanced, but we work hard to not break any of the millions and millions of diverse applications that use SQLite.

I can kinda see the point as far as difficulty in standardizing for a spec, but on the other hand, IndexedDB is amazingly feature-poor compared to SQLite. I'm not generally a big fan of NoSQL, but I'll forgive the lack of JOINs and related sub-querying functionality given the nature of it.

I just started messing with it a few days ago, and so far, you can only query on indexed fields, and only one at a time, and the only allowed conditions are <, <=, >, >=, and =. If you want any kind of compound conditions, you're going to have to do it manually. There's no documentation for how to sort, and it only supports sorting on a single indexed key at a time. Seriously, I had to dig up and read the Chrome source code to figure out how to specify the sort order of a cursor (Hint: It's a string, like 'next', 'prev', not an IDBCursorDirection as the MDN docs mention but don't describe). Don't even bother asking about aggregates, projections, etc.

When I read the linked document, I couldn't help but think - honestly, they're calling being able to change table and index names a headline 2.0 feature? How about at least documenting sorting in queries? Much less supporting querying on multiple fields, on non-indexed fields, using AND and OR, etc. This feels like a 0.2 product to me.

SQL has standards, why don't we use those? SQL-99 is a perfectly reasonable basis for WebSQL.

If you're interested in a websql-like API, take a look at lovefield: https://github.com/google/lovefield -- it's a very websql-like api and looks pretty good.

I've found this wrapper library for IndexedDB very pleasant to use: http://dexie.org/

  > I still feel that the api is a lot more complicated
  > than it has any right to be. WebSQL for example was nifty

From the Web SQL Database W3C Working Group Note, 18 November 2010 (https://www.w3.org/TR/webdatabase/):

  "The specification reached an impasse:
   all interested implementors have used the same SQL backend (Sqlite),
   but we need multiple independent implementations
   to proceed along a standardisation path."

I would imagine it's no small project to reimplement an SQL interpreter from scratch.

Mozilla has said that IndexedDB is an API at a lower level than SQL:

  "IndexedDB . . . allows for a wide number of use cases.
  . . . wrapped by JavaScript libraries;
  for instance, there’s plenty of room for a CouchDB-style API on top . . .
  it would also be very possible to build a SQL-based API on top . . . "

--- https://hacks.mozilla.org/2010/06/beyond-html5-database-apis...

Although you can use the IndexedDB directly, it sounds like even they admit that you probably would want to use a higher-level library much of the time --- and that one of those libraries could be SQL, if anyone wants to write one.

In that article they also give some of the reasons they don't think the W3 should try to weld SQL directly onto the standard. I think they make some thoughtful points. Although I love SQL, I have to admit that it's one of those things that sounds great from 10,000 feet but the devil is in the details.

   select * from table

How hard can that be? But it's when you dive into all the nitty gritty and differences among database brands, and how the spec itself is ultimately ambiguous on certain things, plus the astronomical amount of work it would be, that I think their solution is sound.

The announcement made me check caniuse for IndexedDB support, and it turns out that Safari 10 finally has non-broken IndexedDB:

http://caniuse.com/#feat=indexeddb

https://developer.apple.com/library/content/releasenotes/Gen...

https://github.com/localForage/localForage/issues/604#issuec...

https://gist.github.com/nolanlawson/08eb857c6b17a30c1b26

Despite what some claim, Safari 10's IndexedDB is still quite broken, except possibly for some trivial cases (like if you only use one object store, or you don't use indexes): https://www.reddit.com/r/javascript/comments/545ylm/psa_inde...

Good to know, thanks for sharing!

If any of you are serious about using IndexedDB, please consider:

https://github.com/google/lovefield

Explanation here: https://www.youtube.com/watch?v=S1AUIq8GA1k

I'm currently working on a Angular2 cache management, similar to Redis, wrapper for IDB and these features will help quite a bit with a few of the lower performance operations in my API.

What happens if you access the database simultaneously from different browser tabs? Also, is there some eventing mechanism to push messages through the database between tabs?

What happens if you access the database simultaneously from different browser tabs?

It works fine. Transactions prevent conflicts. And you can even do schema upgrades because there is built-in versioning, and it provides events to let you handle when a version change occurs in another tab.

Also, is there some eventing mechanism to push messages through the database between tabs?

No, you have to do that yourself.

> Also, is there some eventing mechanism to push messages through the database between tabs?

http://caniuse.com/#feat=broadcastchannel

No mentions of promises? Any reason to rev the spec without promises?

IIRC the main reason indexeddb does not use promises is because promises are required to reject/resolve asynchronously [1]. Indexeddb requires that you manipulate the object stores in a synchronous way from it's event handlers, so it can terminate the transaction after running the event handlers synchronously. It can't do this with promises that will run their resolve handler at best in the next microtask.

It's been a while since I looked into this, but I remember this being my conclusion.

1: https://promisesaplus.com/#point-34

I understand the transactional requirements but AFAIK most IDB operations are already async. Open for example:

https://developer.mozilla.org/en-US/docs/Web/API/IDBFactory/...

"The method returns an IDBOpenDBRequest object immediately, and performs the open operation asynchronously."

That's actually not true in Chrome https://bugs.chromium.org/p/chromium/issues/detail?id=457409 although it's not clear if that is a bug in Chrome or in other browsers. It would be nice to have more clarity on that, or an explicit promise API like https://github.com/inexorabletash/indexeddb-promises because until then it's really shitty to use - you have to constantly worry about cross-browser compatibility.

Not all operations are asynchronous.

Here's the discussion on promises: https://github.com/inexorabletash/indexeddb-promises

The 2.0 spec is just a bunch of minor changes. Promises will be a big change - although I would argue it's way more important than all this other stuff combined.

Reply on news.ycombinator.com