MongoDB deprecates Atlas Device sync

Yeah the client has to serialize/deserialize - think of the DB as a mini Mongo where you insert JSON into collections as the mental model. Reason for this is that in our experience, sync is easier when the system is schema-less because the nature of the system ties folks across teams, so this means the schema tends to live above each individual client (iOS, Android, backend, etc and in larger orgs those are each separate teams). Furthermore, advances such as Codable in Swift and other patterns means client devs are fairly used to handling this with JSON/REST patterns anyway.

More similar is that Ditto uses a query-based sync approach - this was an area I am particularly passionate about and helped push at Realm. The challenge with this vs. strict channels is then scalability of the backend system - but we have solved these scalability challenges in several ways.

Most prominently, internally Ditto works different than at least my understanding of how Realm Sync worked originally (it might have evolved more under Mongo). We use CRDTs to handle the conflict resolution, so any JSON inserted into Ditto, ends up being a nested CRDT structure on disk. This enables the same predictive conflict resolution, but its a crucial trade off compared to other approaches.

First, CRDTs enable P2P sync where no server is needed to mediate the system. This opens up use-cases that Ditto powers where mobile devices sync over Bluetooth and P2P WiFi even without a device accessing the internet. Furthermore, it also opens up more scalability on the backend because the server nodes themselves can communicate P2P.

At a more fundamental level this is a trade off of using extra metadata (CRDTs include more context like version vectors and such to know how to merge with each other) compared to Realm's original approach of Operational Transformation which is an algorithmic approach. We felt that the metadata approach was holistically better because of the access to new use-cases and scalability meanwhile there are clever ways to make the metadata cost not a real-world issue (i.e. compression in various ways). Conversely, the algorithmic approach is limited to a client server architecture and has computational scalability issues.

All in all, we are very sympathetic to transition costs, but are confident folks will find Ditto can meet the same and more capabilities in Device Sync. Our current challenge is going to be introducing a pricing scheme to support the broader set of Mongo users. To date, we have been very focused on larger enterprise deployments (gotta ensure the bills are paid!) so we will be creative to ensure anyone transitioning is not affected by pricing if they like us!

You can also check out ObjectBox (Vivien, CEO of Objectbox here)

Sorry I did not see this reply sooner!

1. Yes Ditto uses query-based sync. Candidly, the flexible sync in Atlas has roots back to when I was VP of Product at Realm. It was a must for me, but I think got abandoned for performance issues then brought back. Ditto was built always to support flexible/query sync. We recently have added our own more advanced query engine - DQL (SQLish) and it will enable JOINs and other advanced queries while still being schemaless JSON collections. In this regard Ditto is a better compliment to Mongo than Realm is from data model perspective.

2. Yes this partner-first approach is huge opportunity for us. Given Realm had to sell, we were very focused on building a sustainable business. This meant though more focus on enterprise and specific use-cases than going wide to all developers (you can see how well that worked with Firebase, Parse, and many other developer products when you have VC backing). This means we have a solid foundation technically (large-scale deployments in mission critical systems) to now handle wider awareness. Mongo will really help us bring the awareness of Ditto.

3. Right now candidly our permission system is nascent compared to Atlas - a webhook fires from Ditto to allow you to integrate into external identity systems and apply permissions. The focus with enterprise meant we targeted use-cases that didn't have too much dynamism in their permissions and data models, so you will find our system naive comparatively. This is a top priority where we will be adding built-in dynamic permission functionality that is similar to Device Sync later this year. You can read more on our system here: https://docs.ditto.live/auth-and-authorization/data-authoriz...