Show HN: Etebase – An open source and end-to-end encrypted Firebase alternative
etebase.com
etebase.com
More users care about the privacy and security of their data every day, and encrypted applications are becoming mainstream. However, talking with developers, many get encryption wrong, and they don't even realise it. As good cryptography and bad cryptography look identical to the untrained eye.
The idea behind Etebase is to make an easy-to-use API to enable developers to build encrypted apps, and it's based on our learnings from building EteSync[2] over the years. There's still a lot more to get done, but Etebase is already used in GNOME, KDE, Tasks.org, EteSync, and more. With libraries available for Rust, JavaScript/TypeScript, Java, Python and C/C++.
I hope Etebase will enable a new age of privacy-first and end-to-end encrypted applications, and that our data will no longer be used to track, analyze and manipulate us. Your data, yours only. Let's end-to-end encrypt everything!
I'd love to hear about your experience building encrypted applications. Got any questions? Suggestions? Please let me know!
As for your specific question: we currently don't solve that, but we plan on doing it in the near future. Our approach will be to maintain a client-side index that will be synced across devices. Etebase already utilities to maintain consistency, so this can be done safely.
This question, and ones like it, are exactly why we created Etebase. Building encrypted applications is fundamentally different and comes with a lot of challenges. We plan on continue building all of the tools needed to develop using this different paradigm.
One of the people I talked to had been involved with a startup called Adrenaline Mobility which did something very similar, but ultimately ran out of runway and was acquired by Twitter. He told an interesting story about how they (over-)built a highly scalable service but had a really hard time selling it. Those who lack the training to know the difference between good security and hand waving didn't see the value, and those who did have the training and skills didn't trust a third party to do it right anyway.
Nonetheless, a lot has changed in the past five years, and end-to-end encryption is more widely appreciated now. I hope you have better luck with the commercial side of things; this looks like a very promising start.
I'd love to have a chat with you to learn from your experience if you are willing! My email is tom at etebase, please drop me a line (or let me know how to best reach you).
Thanks again!
Perhaps a straightforward E2EE toolbox could help companies implement their products rapidly so they remain complaint. Clearly this won't help fix issues like Microsoft 365 or GSuite (which need access to the plaintext for email and similar), but it might help some types of SaaS to thicken up their client application and prevent the backend having access to unencrypted data, thus making the transfer permitted.
I just want to echo the above comment around how most customers don't have the knowledge to understand the benefit of this, but hopefully this is changing as we see more strict enforcement of penalties for data breaches. Data is fast becoming a liability you don't want to have the ability to see, and systems like yours offer usable solutions for those who don't understand all the technology, but need a solution.
I wrote something similar, a couple of years ago[0], but it doesn't include encryption; it merely gives a place to add encryption. I didn't want to deal with the legalities of included encryption, and I think others can do far better than I (but it is quite secure, nonetheless[1]). It was really done, just to "retool" my architectural and engineering skills, as I was pivoting from being a manager, back to being a developer.
I am now using it as a backend for an app under development (currently closed-source). It has been working a treat.
But after the app has been released, I may encourage the folks to look at alternatives for the backend. I am not really into maintaining a server system. I like to stay on the app side of things.
[0] https://riftvalleysoftware.com/work/open-source-projects/#ba...
[1] https://riftvalleysoftware.com/BAOBAB/PDFs/Security.pdf (Downloads a PDF)
I think the encryption is the key differentiator here, at least for me. I don't want my data saved exposed on someone else's server, AKA the cloud.
Anyways, great work!
Do you have many FOSS users self-deploying it?
And since you don't have analytics, please know that there's at least one person that will use the light theme, please keep it ;)
I know that stuff's useless to work with but it's really helpful when learning a new existing DB.
More pertinent, though, how do I ensure my users aren't abusing my service? If I allow folks to upload avatars for folks in their address book, how can I stop someone from running up my bill by uploading their MP3 collection? As far as I can tell, if everything is encrypted, I just have some very active users. Obviously there's some amount of tradeoff here, but it would seem like a necessary counterbalance is strong quota controls, which I don't see in the docs.
As for migrations: that's a tough one, and we indeed don't have an easy-to-use mechanism yet. I mentioned it in another comment: the API at the moment is powerful and used in production, but it's still missing some goodies to make the developer experience even better.
As for including every migration in your application bundle: yeah, well until you consider an old version EOL. It's the same challenge Microsoft has with Office, and every app has with their local storage.
Nice bit of competition for https://userbase.com/ (though their focus is mostly "Serverless" and the Web).
As for userbase: I think our and their scopes are very different. They are more of an easy way to store data that happens to use some encryption, not a comprehensive end-to-end encryption toolkit. They don't do sharing, integrity management, data de-duplication, and lot of other things that are needed for more serious applications. In addition, they don't have libraries for anything other than JS.
As said in the other comment, our mission is to encrypt everything, and this is the next step towards that! It's actually something that we've been planning since forever, but it's so nice to finally have it out!
Is the item metadata encrypted as well, or just the content?
Do you have any documentation explaining the encryption process in detail?
We hope to get all of it done in the few days. Fingers crossed. :)
I don't think we are really direct competitors, we are targeting a different niche. It's just that you are the best example of a backend-as-a-service and thus the best way to explain what we do. :)
One thing that's not clear to me from first glance at the docs: does this support offline-first apps? So what about sync with local storage, search, CRUD of collections/items offline, conflict resolution, ...? Am I missing something, or is this something that the developer has to implement themselves?
All the rest of the things you mention: they are all in the works. What we have now is already enough to build great apps (as done with EteSync for example), though we want to build the entire ecosystem around it.
Question: The Collection/Item data structure seems simple but also quite powerful at the same time. However, for lots of applications I imagine it would be nice to have some common abstractions available in the API instead of having to roll my own. I'm thinking of, e.g., data structures like queues, stacks, dictionaries, trees, and also conflict resolution strategies. Are there any plans to add those to the API?
The API is already powerful, and powers real applications, but it's quite raw. We plan on gradually building all of the above and more. We just first needed a solid base to work from.
Yes, the protocol can support that, though it's not yet implemented.
Not sure what you meant with the rest of your comment.
- there's the notion of a local cache but is it synced / de-duped automatically or is it something I still have to do?
Let's say I have a ToDo mobile app. While online I enter the task "buy milk". Sometimes later I'm in airplane mode and I modify the task as "buy condensed milk". Do your library updates the changes automagically once online?
- how realistic is it to run your own server? Is it a couple hours of config and forget about it or it's a multi-days endeavor without certainty of success?
- in your examples you use the name Cyberdyne. If I use Etebase do I unknowingly contribute to Skynet? ;)
Running your own server: trivial, not even a couple of hours. Check out the repo for instructions, or just use the docker images (created by the community).
As for Skynet. We are already too late.
Although this does not yet have the same sort of real-time push syncing (or subscriptions) the way Firestore (or Firebase) both have, it looks like it has the right foundation to support that.
Also impressed that this has such wide client SDK support already. Again, I can tell a lot of work went into it. I'll be trying this out and following the project! Excited to see where it goes next.
As for the wide client SDK support: I think it's essential for adoption so made it a priority. Great to see it being appreciated!
A secure encryption key is derived from the user's password using a random salt an Argon2id, though the data is encrypted with a randomly generated key. Keys are generated for each "layer" of the account (so one for the account, one for collections, and etc), so each part can be re-encrypted (or not) separately if needed.
Password change: you can either re-encrypt the data if you want, or more likely, if the password hasn't be compromised, just re-encrypt the main encryption key that's used to encrypt the data.
Password lost: tough luck. We can't help you recover that, because we don't have access to your data. Though we have some ideas on how to maybe enable recovery (using key custodians, shamir secret sharing, or another method, haven't decided).
However, on the long term, we plan on considering passwords more of a backup feature, and instead move to a model where you just authorise your devices from running devices (though can always fallback to passwords).
That's a question that's looms over every new company and open-source project. I don't think Firebase will go down this route, it's a very different offering to what they already have, and any encrypted stuff will be incompatible with the rest of it. There's also Google's (rightfully earned) bad public image when it comes to privacy, which will limit adoption.
In addition, being open-source is very important for a solution revolving around data ownership and privacy, something that Google will just not do. Our plan is therefore to continue building the best open-source project we can and let the developers decide.
May I ask how do you make money? Just by hosting the service?
Is it production ready?
Make money: hosting the service, yeah. Though also from our user facing apps.
Production ready: yeah. We use it for EteSync (our user facing apps), and through that it's been integrated to GNOME, KDE, Tasks.org and more.
With that being said, Supabase is very cool, and I'm a big fan! If you don't care about end-to-end encryption, you should definitely use Supabase!
Developer tooling is definitely lacking at the moment for building encrypted applications and it's something we plan on addressing.
how can user share data with each other?
It's how we did it for Portabella (https://portabella.io) anyway but that's not based on Etebase/Etesync, just another e2ee tool
Looks very cool, I'd love to chat if you are willing, my email is tom @ etebase, could you please drop me a line?
See the relevant section of the docs: https://docs.etebase.com/guides/collection_sharing