Hacker News Reader

Comment by samwillis | Hacker News Reader

This is a good writeup.

It's worth noting that this loads the entire db into memory, and only saves to the filesystem when you tell it to (not on each transaction) so you could loose changes on a crash.

There is a brilliant project to add true transactional flushing and a block like storage backend to WASM Sqlite.js called "Absurd SQL", worth checking out. It's currently built on top of IndexedDB but they are working with the WG designing the FileSystemAccess APIs to ensure it has suitable block level support and locking for this type of tool.

https://github.com/jlongster/absurd-sql

https://jlongster.com/future-sql-web

View on HN

Glad you liked it! And thanks for mentioning absurd-sql, I did not know the project.

You are absolutely right on memory and crashes. It is a very inefficient way of storing data in a browser from a memory perspective (and data loss risks). I did mention it in my conclusions :-)

If FileSystemAccessAPIs added block level support and locking for at least certain files that would be a game changer for things like this. In the meanwhile IndexedDB is surely a better option.

I analyzed [1] that problem and tried to reproduce data loss with LokiJS, which also has everything in memory and only partially writes to disc. It is very hard to reproduce a loss. You can call the saveDatabase() when the window 'unload' event fires and also after each write when the database is idle again. This works quite reliable.

The only way to lose data is when I shutdown power directly after a write.

It is also to mention that having everything in memory will not support multi-tab usage.

[1] https://rxdb.info/slow-indexeddb.html

Combining these two approaches would let you get rid of the IndexedDB abomination.

I believe FileSystemAccessAPI, as implemented in browsers, does not yet support block level access and locking. Both required to make this work, the developer behind Absurd SQL is working with the team designing the API to ensure it will have support. IndexedDB is the only way of doing this on browsers currently.

https://github.com/WICG/file-system-access/issues/323

Unless I'm mistaken, FileSystemAccessAPI is only implemented in Chrome. Both Safari and Firefox are not implementing until four different file proposals (all from Chrome) can be reduced to one.

I thought Firefox flat out didn't want to implement FileSystem APIs on grounds of "security" and protecting their users?

Yes, many are refusing to implement it due to security concerns. A notable one is Brave, which is based on Chromium, and yet refuses to enable it (https://github.com/brave/brave-browser/issues/11407).

I'm skeptical that reducing access to the FS actually protects users. Those that would be fooled by scams based on FileSystemAccess APIs would very likely be fooled also with other less intricate tactics. So I doubt that the overall security of users is in practice affected.

At least, browsers that refuse to implement the FileSystemAccess APIs could implement them, but leave them disabled by default, and require some non-trivial action to enable them. So users with a very basic understanding of how things work in the browser would not be able to enable them.

> So I doubt that the overall security of users is in practice affected.

The problem is the sheer number of APIs that Chrome ships and wants other browsers to ship, and what browsers already ship that require access via prompts: camera, location, notifications, file access, bluetooth, usb, motion sensors, serial ports, midi devices, clipboard...

Just prompting user to allow stuff is no longer enough, and adding more prompts leads to worse security.

It's sort of a double prompt scenario since the FileSystemAccessAPI requires users to actually select the file/directory for the web app to use. You can save the file handles into IndexedDB in which case the user will only be prompted if they refresh the page to authorize the web app to continue using that file/directory.

From my use of it, the only real vulnerability I see is that Chrome still considers anything from file:// to be the same origin. This means as a developer you absolutely should not be saving file handles to IndexedDB if you are loading your app via file:// instead of https://. This is a pretty niche use case, but I do think static html "apps" are an underappreciated form for distributing software and this new API makes such apps a lot more powerful.

I wrote a plugin for TiddlyWiki that lets it operate really smoothly using this API https://github.com/slaymaker1907/TW5-browser-nativesaver that demonstrates why this API is worth the trouble.

I'm not sure any API that requires "secure context" can be run from a file://, for those reasons.

Nope, I use the file system access API every day from file:// to take notes with Tiddlywiki.

I wouldn't be surprised if some (or most) browsers don't implement the parts of the FileSystem API that are designed to access the users home directory (with permission). But the parts that are for a "sandboxed virtual drive" will be, that is what's needed for web apps and PWAs for saving their own data in a local block level storage (rather that IndexedDB or LocalStorage). That part of the api is very much designed for enabling things like WASM Sqlite.

https://developer.mozilla.org/en-US/docs/Web/API/FileSystem

  "This interface will not grant you access to the users filesystem. Instead you will have a "virtual drive" within the browser sandbox."

If you need to use IndexedDB and you do not want to have to handle the absurdities of IndexedDB, give Dexie a try (https://dexie.org/). It really makes using IndexedDB a breeze.

Reply on news.ycombinator.com