RemoteStorage.js 1.0.0 released
community.remotestorage.io
community.remotestorage.io
This is vastly different than some web app, ie a single page app, storing data on a cloud database somewhere (DBaaS for example). Because, the web app owner still owns and controls that cloud database. I'm very interested to see where this goes.
Edit: reading the docs, I have a lot of questions. This appears to be wrapper for browser local storage, that can then sync to a third party provider (local first). But browser storage is basically a key value store. Also reading the docs, they talk about using data modules defined with jsSchema.
That's all fine, but consider the use case for a note taking app. I store all the users data as json objects, that are evenutally persisted to the storage provider. So to query the users data on a new device, to get last weeks notes for example, I have to pull down all the user note data. So basically with this model you are pushing around large sets of json from device, with no way to query or get at a range of objects for example.
This is less appealing than it first seemed. Am I missing something?
However, having used RS apps for years now, I think it's worth mentioning that most people considerably over-estimate the amount and size of the personal data they use daily, and in the case of notes, I never had performance issues with Litewrite pulling all my notes whenever/wherever I connect my storage: https://litewrite.net/
That's not just because this is actually pretty fast with multiplexing on HTTP/2, but also because you usually don't connect and disconnect all the time on all your devices. So you will usually have your existing data cached and available on app startup (including when you're offline), and only sync remote changes when you get online. This is also what similar proprietary apps like e.g. Google Keep do.
We (draw.io) 100% believe in this model, it's absurd that the data should live with the SaaS application provider. It's user lock-in, plain and simple.
What I'm getting at here is that remote storage isn't service agnostic really, is it? Unless the data structures are really standardised/simplistic.
Nonetheless user ownership is worth enough in of itself.
Likewise, this might make sense for open source PWAs. They can now be developed without needing to provide a server, and can simply be hosted on e.g. GitHub Pages or S3. People can now serve different instances of it and people can switch between those.
Also we can hope that the next step is standardized ways to store some data (e.g. a common key and data structure for calendar events) allowing multiple apps to read the same data =D
We'd be happy about any and all feedback and ideas about this specific topic. Our community forums are a good place for that: https://community.remotestorage.io
I'm pretty skeptical that this is really practical for most people. I think it's a lot like 'programming for everyone' tools. Most people are basically doomed to fail in some kind of way because both maintaining data and programming are just inherently difficult.
Hell, I'm pretty confident in my programming abilities, but most possible interesting projects require a significant amount of time, if nothing else. Similarly, even tho I'm pretty comfortable with servers, command line shells, etc., managing a working and reliable personal backup system – that's actually resilient to infrequent but inevitable catastrophes – is not a trivial amount of work. And getting the damn thing setup in the first place has been a very significant amount of work!
I'm not sure most people are making a wrong decision with respect to all of the relevant tradeoffs. Most data is probably not that valuable and so the effort to actually secure it and protect it is wasteful.
Edit: the library in question also supports using Dropbox and Google Drive, so your users don't even have to know about RS in order to sync your PWA's data to their own account with those.
I wrote this before I realized you're right:
> Yeah, I get it. But the real benefit, that outweighs the cost in time and attention, and maybe money too, is that you can control your data. Not having your own copy, and your own backups, or not having the technical ability to access relevant info in your data, as I'd expect the vast majority of people to not have, doesn't seem like it means that much effectively.
Thinking about email was what opened my eyes. Even tho most people don't understand email technically, it's a standard feature, effectively, that pretty much anyone can have a new email provider import their emails from their old provider. That's a big benefit and just that benefit alone is worth implementing something like RS for a lot of webapps, if not nearly all of them.
At leas that is what I recall. Still think it would be great to integrate remoteStorage properly with Nextcloud! :)
Are there any other data protocols aimed at helping users "own" their data out there?
Tent is built around posts. Each Tent server stores a single user’s posts and sends copies to the user’s subscribers.
Another side idea that comes with this is something like IPFS or even torrent, where bits of the file storage are stored on individual nodes but incapable of meaningful data unless put together with other bits of data from other nodes.
I admit I'm not even sure how bittorrent make sures the other "bits" are valid, would love to know if there is some p2p-storage.js you could drop on your webpage
If you need access to the users data then you will need to run your own server anyways, so you deal with the trust problem as any other web application does.
>p2p-storage.js... Maybe webtorrent.min.js is what you're looking for.
@Firefox devs: please add WebSQL support. Now that you lost the IndexedDB devs who voted against WebSQL, it's rather time. Mind you, NoSQL and NewSQL nowadays coexist next to each other. Safari and Chrome based browsers (90% global market share) support WebSQL, except Firefox.
Right now it's defined as "What SQLite does", which is not a way to implement a standard.
Not to mention that even if they did create a full "specification" for the SQL that it uses, it seems like way too much to force onto a browser to develop for not all that much benefit.
Safari and Chrome both support it, but it's not going to get any major updates and only exists in the browser today because it was added at one point.
The cut-down SQL would include just the basics: CREATE TABLE, CREATE INDEX, INSERT, UPDATE, DELETE, SELECT. No triggers or views. No support for partial indexes or indexes on expressions, common table expressions, or other such features. There would be a minimal set of built-in SQL functions, but with support for application-defined SQL functions written in javascript.
A competing implementation would not need to understand the SQLite file format nor the complete SQLite language nor would it need to implement the SQLite API. All it would need to do is support the basic SQL subset defined by the spec.
If you get the browser vendors on-board with this idea, and I'll write the spec.
But if someone out there has experience working with whatwg or W3C on these kinds of things and wants someone to help out, I'd love to be a part of it.
But like you said, the hardest part is getting the different browser engines to promise support (or even take it seriously). I remember reading on HN a post from a woman who was involved in trying to get SVG in the browser standardized (sadly I can't seem to find it any more!), and just reading about the struggles she went through to even talk to someone involved who, while receptive, basically told her that it wasn't going to get done.
It seems like there would need to be an "advocate" who is part of the teams on EdgeHTML and Gecko that would be willing to fight for it before it would even begin, otherwise, like you said, it would all basically be wasted effort.
The JavaScript standardization system is quite nice, I wish that browsers could get together and come up with something similar for Web APIs...
BUT... what about making an exception to the rule? SQLite is sort of a unique beast -- public domain license, used almost everywhere (most smart phones have it embedded, Chrome and Safari have it), and rock solid? Also, it's not like it's a 100 million LOC app.
A real spec is "loose", it allows wiggle room in many areas, it allows for multiple ways to implement things, it allows implementations to act differently, and in most cases that comes out as a win for the user. Saying the spec should do what a single implementation does is about as close to saying "you aren't allowed to improve this" as you can get.
I really believe that a monoculture is a bad thing. Luckily on most platforms SQLite (while the best in most cases) doesn't have a monopoly on what it does, and while it's a "defacto" standard in many cases, there are alternatives at every step of the way. Saying that a platform must implement SQLite takes that defacto standard and codifies it. It's forcing a monoculture. It doesn't just deincentivise improvement, it almost explicitly forbids it.
For example, javascript changes can't be put into a standard until they have (IIRC) 3 major browsers which implement them.
Pinning a standard to a specific implementation (and a specific version) is the opposite of every other web standard. Plus it means that any alternate implementations must reimplement SQLite themselves if they don't want to or can't use SQLite themselves.
If you're referring to SQLite itself being implemented multiple times, I don't understand why that's relevant to Web SQL. You wrote that Web SQL shouldn't be supported, i.e. implemented, until there's a spec. It seems like there's a 'spec' now, if not an official formal spec. But now you're claiming that Web SQL can't be a standard because there aren't multiple implementations. But it certainly seems like there are already multiple implementations:
- [Can I use... Support tables for HTML5, CSS3, etc](https://caniuse.com/#feat=sql-storage)
But still, your logic seems circular: Web SQL shouldn't be implemented for all the browsers because there's no spec and there's no spec because there aren't multiple implementations.
I imagine there's a distinction you're trying to convey that I'm missing. What is it?
It is a bit circular, but generally something gets proposed as a standard, multiple browsers will implement it, then it becomes a finalized standard.
In this case it was proposed, the standard says "your implementation needs to do what this other implementation does" (and no other information is given about the SQL system), 2 browsers (really 1 engine at the time, webkit) used SQLite directly, and it never went anywhere else.
I guess my point isn't that there's not a "spec", but that the spec that there is doesn't say anything about the SQL part of WebSQL, it just says what the interface to the SQL engine looks like, and then mandates that the SQL engine act, look, and work like the one in a specific version of SQLite.