Ask HN: What storage protocol for Bring-Your-Own-Storage applications | Hacker News Reader

Ask HN: What storage protocol for Bring-Your-Own-Storage applications

1 pointbeingflo5 comments

I'm currently working on a tiny task management web application for my own needs. To minimize the implementation work I came up with a concept that I think could suit a number of applications like this as well:

- No user signup (no backend at all)

- Local first (via localstorage)

- Optional bring-you-own-storage functionality for sync and backup

The last point would allow the user to provide, say, an S3 endpoint and api key to allow the application to synchronize its state to a remote data store. By supplying the same credentials to the application on another device, one would have cross-device sync effectively for free. To avoid exposing data to the storage provider I'm also implementing client-side encryption.

Now the only remaining question is what storage protocol to use. I've found S3 client libraries to be severely lacking for browser environments (the AWS SDK is a mess and seems to rely on nodejs shims to work, not to mention its gargantuan size).

What are other easy to use and widely available storage protocols? WebDAV? A git repo? Curious to hear what else is out there!

Is this strictly for techies that would possess AWS keys in the first place (see Dropbox HN launch post, just set up your own FTP server, etc) or eventually intended for normal people?

If you are implementing client side encryption, something like "log in with dropbox" oauth, GDrive, etc or similar to just serialize an encrypted snapshot into a known folder would be the most usable for non techies based on previous stuff I've worked on. Asking people to set up restricted IAM roles and keys to write to a single bucket for a todo list is honestly very high up there on the ridiculous list, as much as I would totally do that.

It's only intended for techies (and friends thereof I suppose). It's really a tiny project that I've set out to finish in 16 hours as a challenge. So far the progress is great but those S3 libraries are throwing a wrench in my schedule. Because of that tight time budget I think oauth would be out of scope as I've never done anything like it.

Getting into this I was under the impression that uploading a file to S3 was as simple as chucking the api key into the Autorization header of a post and off we go. That's the type of ease of use I'm looking for. Turns out its not quite as trivial with a whole signature process etc.

I haven't had an issue with it, you're basically just calling a function.

https://github.com/awsdocs/aws-doc-sdk-examples/blob/main/ja...

Apparently the sdk doesn't play nice with vite. And it's way to heavy for my use case anyway. After looking further I found this library that is perfect for my needs https://github.com/mhart/aws4fetch.

If it's the AWS SDK that bothers you can't you just use the S3 REST API (https://docs.aws.amazon.com/AmazonS3/latest/API/Welcome.html) directly ?

Reply on news.ycombinator.com