Things: The State of Sync Part 2

Wow, I can't believe they haven't built a centralised solution and that they think web-based sync is slow because of all the disk accesses. I don't care how big their user-base is, they are still only storing tasks - not images or videos.

Client to authorative server sync seems to be the way to go to me… it doesn't have to be live (they could queue up the syncing asynchronously) and they could control whatever subscription model they decide to put in place for one of the most expensive task management apps in existence (especially when you buy a copy for your Mac, iPhone and iPad).

For example: Remember the Milk seems to have solved this problem without having a slow application, and they have a web-based version (the default) of their app to boot with more features than Things would have.

I think the problem is they approached the wrong 'web developers' when they asked for a synchronisation solution. I could easily see 5+ million users on a single, active DB server with under 64GB of RAM being able to hold a complete index set in memory and being able complete 10,000+ queries per second with communication occurring via a queueing product like RabbitMQ or ActiveMQ or a custom long-lived Java/NIO web server written in under a couple-hundred lines of code to keep connections open to 100's of thousands of devices simultaneously (http://blog.urbanairship.com/blog/2010/08/24/c500k-in-action...).

Suggestion: get a competent web developer on the team.

Problem solved.

The problem is far easier to resolve than they seem to think. I'm pretty sure that even with a million active users, you could run something like this on a few hundred dollars a month of AWS hardware at the most. Compare that to the two years of software engineering time they've wasted on this so far.

It seems to me to be a very bad business decision to decide to write a framework first, before delivering on your core product.

As the deadlines have ended up slipping, they have ended up with a set of very defensive posts about why they haven't yet delivered.

I don't see why you would jeopardize (that might be a bit of a strong word, but I'm sure they haven't done amazingly in sales in the last few months compared to what they had been) your business by attempting something far more complex when your product was already behind others in the marketplace.

There's no reason why they couldn't have gotten cloud syncing working just for them, then rolling out the code into a framework later on. That seems to be the smarter way to go about doing it - not writing a series of defensive posts saying that it's so hard.

"We ended up with detailed plans to create a JavaScript-based cross-platform data model framework with Git-inspired sync built in. This strategy required substantial portions of all versions of Things to be rewritten. It was clearly the most ambitious project we ever took on. Dissatisfied with our previous attempts, we didn’t want to settle with anything short of perfection."

Wow, this reads like a textbook case of over-engineering. I'd have thought that experienced developers like Cultured Code, with a shipping and successful product, would know a little bit better than that...

Seriously, in all this time, nobody ever thought of just hacking something together that's good enough until a better solution is found?

Sync is harder than you think.

But see: http://en.wikipedia.org/wiki/Merkle_tree

Unfortunate, really unfortunate, because the software is good. Back to org-mode and Dropbox then.