Hacker News Reader

Comment by tobiemh | Hacker News Reader

Hi typingmonkey. So SurrealDB abstracts away the document aspect of IndexedDB so it can be used as a key-value store. We then use that key-value store (in a similar way to RocksDB) to store the data in SurrealDB. You can see our abstraction layer here: https://github.com/surrealdb/indxdb . It's actually pretty performant in our tests so far, and we'll be releasing this soon as a WebAssembly library, enabling SurrealDB to run in the browser locally.

I know rxdb (am a stargazer) so I'm really interested to hear suggestions or thoughts around this area!

View on HN

Thank you for these links, I will inspect that.

As far as I know, accessing IndexedDB from WebAssembly only works by passing the data through the JavaScript layer (correct me if I am wrong). So from how I think WebAssembly works, it is likely not any faster then directly doing that in JavaScript. I mean, the storage layer does not have to do any heavy computations, so tunneling the data would be more expensive then what you get from plain js.

When you store the data into an IndexedDB based key-value store, how does the querying work? Do you still use IndexedDBs secondary indexes when a query only matches a range of documents, or does the K-V store has any method to support querying a subset of the data without fetching the whole database state?

Yes you're right. Accessing IndexedDB from WebAssembly needs to go through JavaScript. The only benefit from our perspective is that building it in Rust enables us to use the same logic and code that powers our entire database platform. From a performance aspect, you're right, it still needs to go through JavaScript.

So we don't use any features from IndexedDB at all really. We literally treat IndexedDB as a key-value store interface like RocksDB or something like that. So all of the indexing is done within SurrealDB, and is stored in IndexedDB as a set of binary keys->values (Uint8Array if I remember correctly).

When we want to query a whole table, or a subset of a table, SurrealDB determines what individual keys to load or what key ranges need to be loaded to perform the query.

Would love to chat further about this - it's a really interesting area of discussion for me!

Ok now I understand.

You can come to the RxDB discord for a chat if you want. I am really interessted in using WebAssembly for browser side storage.

I look forward to the day where someone reimplements IndexedDB via Webassembly, with the exact same API, so that we have a faster IndexedDB which uses the old one as storage layer :)

Awesome I'll see you on the RxDB Discord!

Reply on news.ycombinator.com