Self-Hosting a Firefox Sync Server | Hacker News Reader

Self-Hosting a Firefox Sync Server

blog.diego.dev

322 pointsshantara84 comments

Note that this setup still relies on Mozilla infrastructure for several surrounding services for auth, email, account metadata, etc.

https://github.com/dan-r/syncstorage-rs-docker/blob/main/app...

It's a good start though and as long as you're OK migrating your account you can plug those in later. Fully self-hosting all the components is just as doable but a little bit more involved. Compared to when I last set it up they seem to have been improving on docs and devex.

https://github.com/mozilla/fxa

https://mozilla.github.io/ecosystem-platform/tutorials/devel...

https://mozilla-services.readthedocs.io/en/latest/howtos/run...

> this setup still relies on Mozilla infrastructure for several surrounding services for auth, email,

Why does email rely on mozilla infra?

Mozilla's email-related service is https://relay.firefox.com/ which obviously needs to rely on quite a bit of Mozilla infrastructure in its default incarnation, and is probably a fair bit harder to self-host than Firefox Sync.

Is Mozilla Relay (email) really needed for Sync?

Never used Sync, so maybe I don't understand something. But does not sound safe to put all eggs into this busket at once.

Relay is unrelated to that conversation.

That is also unrelated. Firefox Sync itself requires Mozilla Accounts to function at all, which requires email for verification and password reset. To properly self-host Firefox Sync you need those parts too, which includes an SMTP server (or substitute).

The other way around. Server-side email verification for accounts and password-reset is an integrated part of the flow and also needs to be hosted and wired up, unless you want to dig into the code and patch those parts out.

Maybe eventually the whole stack can make its way into a docker container with a shell script for config.

Maybe it's just me but having dedicated employee's to manage security is a prerequisite for me; atleast I could point my finger at someone if things happen.

Of course it's not just you. That seems somewhat obvious. Also plenty of people that do like tinkering on their homelab or self host. It's definitely the more interesting choice!

I personally don't see the value in that. I'd rather spend time fixing something than getting people to fix it for me, and having someone just to point fingers at them after a failure feels cowardly. But maybe I'm just not a managerial type.

That's a time vs. outsourcing trade-off. When everything is working, it's invisible and great, but if something breaks, and if you're in the middle of the something, you either have to postpone it, or stop everything and fix the problem to be able to continue your life unaffected.

I'm into self-hosting, and host some of my services myself, but get some services too, because I have limited time and don't want to stop drop and roll when something breaks.

I completely understand the feeling, but when's the last time pointing fingers actually got you anywhere? :)

Sounds like a recipe for getting people with knowledge of security best practices and people with knowledge of the systems being developed, but not both at the same time.

It's enough different components to not be a good fit for an all-in-one image at all. That road lies madness.

docker-compose is a much better fit. That already exists. What we need is more people actually deploying and engaging (;

Just cut out the middleman and use Google directly.

It looks like you used to be able to configure Firefox iOS to use a different FxA/Sync server, but I tried to find the setting and it's gone.

> For Firefox for iOS version 9.0 or later:

> - Go to Settings.

> - Tap on the Version number 5 times.

> - Tap on “Advance Account Settings”

> - Enter your content-server URL

> - Toggle “Use Custom Account Service” to on.

https://mozilla-services.readthedocs.io/en/latest/howtos/run...

Is the different names options under “Advanced Sync Settings” the same thing just with different name? “Use Custom FxA Content Server”?

Garuda Linux has been hosting their own Firefox Sync server for a while now for their FireDragon browser [1]:

   defaultPref("identity.sync.tokenserver.uri", "https://ffsync.garudalinux.org/token/1.0/sync/1.5"); // --> FIREDRAGON CONFIG

(They seem to be running the older non-Rust sync server though. [2])

[1] https://firedragon.garudalinux.org/

[2] https://gitlab.com/garuda-linux/infra-nix/-/blob/main/docker...

A little off topic, but I had to refresh my memory as to what Garuda was. As a long time user of arch in various "forms" (initially manjaro until I grew frustrated with all the extra/different things, then antergos because it was mostly just plain arch with a nice installer/sane default packages, then endeavouros because it was the closest thing to what antergos provided me), I'm not sure how I missed garuda. Some of their utilities look convenient.

Thanks for the mention! I'll have to try them again on my next machine.

This reminded me to post my own experience:

https://kyzer.me.uk/syncserver/

It can be done, but by golly is it a slog. The previous version (abandoned and hopelessly out of date, written in Python 2) was SOOO much better.

It doesn't require Docker though, nor does it require Python. It does require MySQL for now, but there's like one volunteer guy working on supporting sqlite, like the old version supported out of the box. Meanwhile, the main contributors are LARPing being a corporation and writing justifications for collecting metrics and introducing new metrics frameworks, rather than make it any easier to self-host.

Take a look/follow https://github.com/canvas-ai/canvas-browser-extensions

Its meant to sync your browser tabs (chrome/firefox) to a self-hosted virtual "directory" tree powered by lmdb and roaring bitmap indexes. In retrospect, we should have used rxdb or pouchdb from day one (keeping the bitmaps) but eventually we get there.

Workflow

- Create a new context path based on the task you are working on(universe://travel/2024/03/barcelona for example)

- Depending on your configuration, your browser stores your current tabs and closes them, you then start your airbnb journey adding new tabs to the context path

- Your significant other may even open the same context and work on it with you(as in, add additional ones, if auto-sync is enabled close yours etc)

Server component is under a rewrite atm, my amateurism at play! You can do stupid things with a 2 user user-base (like a full api + backend + module rewrite started simultaneously, breaking existing functionality beyond repair) but with the new tools available these days (cursor, claude 3.7), the most important limiting factor for an idea becomes .. well, time!

I did that for a while, including hosting the auth myself. Then I realized Mozilla is probably one of the few companies I trust with my data, even though it's probably misplaced trust. The sync server is relatively straightforward to setup, but auth is quite more involved. I don't think it's worth it, compared to using the public hosted service

Mozilla just burned out the last remaining bits of good will with the "terms of use" fiasco. At least there's LibreWolf to keep a privacy-respecting fork.

Mozilla rebranding themselves as an advertising company and getting the majority of their funds (and assumedly direction) from Google should have you at least questioning that trust.

Gonna admit, that aged poorly, and very quickly.

Thank you, this is the only thing what slowed me down switching to Librewolf.

Seems that MySQL and Spanner are the only supported DB-Engines by now, but there are efforts to support sqlite and Postgres[1]. I really hope these get merged soon, would make it much easier to backup your whole sync storage contained in one single sqlite file. However, a multiuser instance would probably be better stored via postgres due to better parallel update support.

EDIT: After trying this out: How exactly do I force Firefox (Librewol) using the self-hosted URL? Is there an about:config setting?

1: https://github.com/mozilla-services/syncstorage-rs/issues/49...

In the documentation (https://github.com/mozilla-services/syncstorage-rs?tab=readm...), it looks like updating an about:config variable should be all you need to do:

> In Firefox, go to about:config. Change identity.sync.tokenserver.uri to http://localhost:5000/1.0/sync/1.5.

You'd need to update the URL to match your own, of course

You can use Librewolf with Mozilla's sync server; you don't have to self-host to have sync with it.

What's the privacy benefit, given that Firefox Sync is E2E encrypted?

https://support.mozilla.org/en-US/kb/sync

To best of my awareness, the login page (including JS code) is still served over the Internet, and encryption keys are derived solely from the password. That’s not a good design for something as sensitive as one’s full browser data.

send a pr.

but don't know how adding a key owned by moz would make this more trustworthy

It used to be that you could have separate login and encryption passwords. It used to be that you could self-host it as easily as pointing to a separate WebDAV server (trivial to spin up if you have _any_ web server at all).

Not anymore.

As AshamedCaptain notes, Mozilla actively don’t want an actually-secure password storage protocol. They chose to weaken their security.

Something similar to Firefox Sync is Floccus. If you only need bookmark and tabs sync it looks straightforward in setting up

https://floccus.org/

Would be nice if librewolf and WaterFox changed their flow to make it clear that Mozilla's Sync Server is being used, and make it opt in with something like:

To sync please select one of the options below:

[x] Custom Sync Server: ___________

[ ] Mozilla Sync

LibreWolf is opt in and if you click the checkbox named "Firefox Sync" thinking you're getting some kind of sync that isn't Firefox Sync I don't think there's any UI change that can help you.

Your parent comment's idea is quite sensible, no need to be contemptuous like this. Firefox branding is a mess, being both a browser and a set of services. It's reasonable to mix things up or to be confused.

Sync is nothing but an awfully overengineered blob storage (I have implemented an alternative Accounts+Sync server when 1.5 came out, https://gitlab.com/drdaeman/firesync/ - long abandoned and the code is awful proof-of-concept inly, but the point here is to prove that I have a slightly educated opinion on the matter).

IMHO, It would be best (for technical users) if Firefox forks would rather replace Sync with something more standard like S3 or WebDAV. If my memory serves me right, there’s absolutely nothing in Sync that those protocols cannot do equally well (and even better because they’re actual mostly-interoperable standards).

Heck, it would be nice if Mozilla would do the same thing, but I’m afraid NIH is way too strong with them. I’m very happy if that gives us Rust, but Sync is simply garbage that shouldn’t have been ever made.

Is there a specification somewhere for the sync server that you followed, or did you figure out the endpoints/logic by looking at mozilla's sync server?

I was thinking about taking a crack at this as well, but I can't find a spec, and I want to leave looking at the python/rust sync server source code as a last resort.

The API docs are linked here: https://github.com/mozilla-services/syncstorage-rs?tab=readm...

It is documented here: https://mozilla-services.readthedocs.io/en/latest/storage/ap...

I'm not sure about the current status of this document. Years ago there were subtle nuances that I gad to reverse engineer looking at the source code and Firefox's actual behavior.

With only object storage, handling sync conflicts [0] becomes more complicated. I can understand why they don’t do that.

[0] https://mozilla-services.readthedocs.io/en/latest/storage/ap...

Yet stuff like CalDAV which is used by millions transparently handles syncing on top of WebDAV (arguably not just dumb storage , but not much better than it). Not only that, but Firefox Sync _itself_ used to be WebDAV.

For the record, for a lot of time Firefox Sync used to be exactly like that.

https://blog.mozilla.org/services/2011/03/22/get-syncing-wit...

Not only one of the first questions would be whether you want to use a Custom Sync Server, but this "Custom Sync Server" could be something as trivial as WebDAV server (if you have any kind of public-facing HTTP server, it almost certainly already supports this, or does so with minimal extra dependencies) . Then it would also ask you for the E2EE key, separately from the login key.

Congrats on getting this self-hosted. However, the solution looks more complicated than necessary. You shouldn't have to set up a database server and Docker just to sync a little bit of data. Ideally, there would be a sync server implementation that uses SQLite and can just run stand-alone.

In theory you could set up a simple sync server that implements the necessary endpoints and nothing more, nothing less. Guides like these run the official, Mozilla-maintained sync server, which is obviously based on serving millions of users rather than being a minimal implementation for someone in a home lab.

One could fork https://github.com/mozilla-services/syncstorage-rs, take out the plumbing, and reimplement it all without a secondary database and the Google Cloud base architecture, I'm sure, but like with so many things, you'd first need to gather a group of people with enough interest and spare time to do the development.

Certainly, but at least it’s available at all for self-hosting, which isn’t the case for many browsers.

There's a MR open:

https://github.com/mozilla-services/syncstorage-rs/issues/49...

There’s also this: https://github.com/mozilla-services/go-syncstorage

While it was never fully deployed into production it was tested with a few hundred thousand users. It is stable at least.

Given the recent Firefox fiasco. What are the viable options for a free software browser? Can anyone point to the most promising alternatives (apart from the other corporate browser engines)?

Ladybird browser: https://ladybird.org/

“Promising” maybe, but it’s pre-alpha and as such won’t be an actual alternative for some time to come.

Librewolf, Falkon, somthing-chrome-derived (I forgot)

Right now, there are none. The "fiasco" is still about the least bad option available.

Some could become viable in a few years, but I wouldn't count on it to "just happen". You'll still need to actively try to get your specific use cases well-supported for a while, and if not enough people do that these projects will never get there.

In terms of open source there are really only Chromium- and Firefox-derived browsers. With Chromium browsers, you’re still subject to Google’s whims in the long term, such as removal of V3 extension support. (I.e. a conceivable fork with V3 compatibility will inevitably become too difficult to keep up to date with the mainline.) If Mozilla dies, Firefox and derivatives will in all likelihood wither away as well. IMO there is no alternative to supporting Mozilla, and also keeping them accountable and criticizing them where criticism is due. They are still roughly the good guys, even if sometimes misguided.

Regarding the “fiasco”, there was an update that you may have missed which is at least a partial fix: https://blog.mozilla.org/en/products/firefox/update-on-terms...

Closed article after "docker".

Oh god... It's still seems faster to hand-merge bookmarks from all of my browsers and devices together.

Why is this downvoted?

Why does this bookmark syncing server need to be anything other than a dumb file storage? It used to be a simple WebDAV server -- Mozilla unilaterally changed it to this monstrosity of a service with little real gain for the user to be seen.

It's not about bookmarks only. Also history. A syncserver is the only way to access firefox mobile history, as its api (browser.history) is severely restricted or unavailable. So short of rooting your device to directly access places.sqlite, there's no alternative.

> A syncserver is the only way to access firefox mobile history, as its api (browser.history) is severely restricted or unavailable. So short of rooting your device to directly access places.sqlite, there's no alternative.

What I'm pointing is that in previous versions, Firefox ITSELF used to allow setting an arbitrary WebDAV server as "sync server", and STILL sync history, tabs, whatever the heck you can think of. https://arstechnica.com/information-technology/2010/02/mozil... . It is much later that the option to use a plain WebDAV server disappeared from Firefox Sync, and also later that the option to set your own E2EE key also disappeared (and forced to use the login pw instead, making it easier for Mozilla to capture). The "restriction" you are pointing out is entirely Mozilla-made. All in the name of user convenience of course.

Again, why is anything other than a dumb storage server required, considering than previous versions used to be able to do it with just a dumb storage server ?

FYI You can use Floccus and a WebDAV server to sync bookmarks across browsers, set and forget

I'll jump to the wolfside but I'm going to absolutely miss "send tab to device", where I'm often driving this from mobile firefox, sending to one of a few other devices for Next time I'm sitting at a desk

There are alternative techniques but maybe none with so few clicks, where the outcome is something literally in your face. A new tab opening on a computer elsewhere. It's that "I'm almost forced to see this" that I like

I'll miss this, too. But it's an acceptable loss.

Quick thoughts on how to get it back: Orion Browser by Kagi has an iOS app, and they support Firefox plugins. So in spite of how much effort it takes to get a mobile app to do what you want, it might be possible for LibreWolf and Orion for iOS to sync.

Asking a related question: is there anyway to sync history between browsers, fully self-hosted?

Firefox Sync does sync history. If you mean cross-browser, I don’t think there is a way.

https://floccus.org/

Great timing, with Mozilla's new privacy policy problems the other day

why would those matter since your sync data is encrypted and they can't see it anyway?

The issue is not Mozilla getting access to your data, but losing access to Sync itself.

Why have your data sitting on a timebomb when quantum risks are just around the corner. Self host if you care about your data!

How do you protect your self host against quantum risks?

Self host one have control over the post quantum custom encryption you want to use, not at wimp of some corporate over Lord who know they are really doing it for user benefit or just a user acquisition trick.

> the post quantum custom encryption

Do you have a custom encryption that is post quantum?

because they can update the client tomorrow to not use e2e...

meta did it. google did it. so not without precedent.

If you self host, you don't need to wonder whether you need to respect their acceptable use policy.

It should be possible to use syncthing to achieve sync across multiple devices, without having a third-party server - even if untrusted, right?

No, any concurrent modifications would cause problems.

The main difficulty with sync is conflict resolution and none of Syncthing's solutions work well for the things that Firefox syncs.

I’m no longer using Firefox. They have stated clearly that they sell your data.

What's your alternative?

Brave, for now.

So the alternative to Firefox is some cryptocurrency software?

I've use Firefox as my tertiary browser option, for some specific uses like for example I have it always have an VPN plugin on. But I'm starting to feel anything in its vicinity should be avoided like hostile malware, regardless if an attempt to remove bad parts of it is made.

A corporation can do a lot of bad stuff with their software and terms, and most have probably normalized that in their mind. But after now Firefox for example implicitly lawfully claims all the google docs you edit as their IP, that should be a bridge too far for anyone using it even slightly professionally. God help if you also use it as a person that Mozilla considers you their political blood enemy (which isn't exactly hard nowadays). Not exactly the Firefox 1.0 release I still remember. Not even as benign as its worst enemy it was built to be against at the time. Truly Dark Knight -esque.

Mozilla is changing the wording: https://blog.mozilla.org/en/products/firefox/update-on-terms...

There is still cause for criticism, but if you think Mozilla is worse than Google (Chrome) or Microsoft (Edge), then that’s hard to take seriously.

Hard to? Missed the part about Mozilla basically being an advertising company nowadays, and "a global band of activists"?

Something Chrome-adjacent, like Chromium or Brave doesn't have the stain Firefox and everything close to it now has. That is not a position that is hard to maintain.

Firefox has come so far from what it once was. I did once cheer for it, but it simply has become its own worst enemy. Almost nothing is left, crazy to think they're the only competing browser engine independent from the chromelikes.

Reply on news.ycombinator.com