Derby v0.5.0 – The first OT-powered realtime web app framework
blog.derbyjs.com
blog.derbyjs.com
If there's been one issue with Derby so far, it's been Racer. The data sync-ing engine leaks memory and is largely unstable. We've suffered far too much downtime due to these issues. Moving to a fresh backend with ShareJS enables Nate, Brian, Joseph and the rest of the team to rework the scope of the framework with some of the larger goals in mind – things like horizontal scaling and powerful conflict resolution. We have yet to upgrade to 0.5 (the upgrade guide was literally just released), but it appears to be a much more stable foundation on which to build a "real-time" web application. I'm sure we'll have to put up with bugs for quite a few weeks, but this is a powerful step towards reaching 1.0. Thanks guys!
[0] https://habitrpg.com (source: http://github.com/lefnire/habitrpg)
I'm curious why a "real-time" framework is needed for this type of app, though -- why not Backbone with REST? Is it so that you can view others' pages to see their habits/dailies/todos get updated in real-time...?
tl;dr: No big reason. He likely agrees with you.
The gist of it seems to be that its a set of data models that make handling concurrency control and conflict resolution easier in the context of a single blob of data being edited by multiple users simultaneously (similar to Google/Apache Wave)
Firepad doesn't actually do OT of JSON. In Firepad, a string OT algorithm is used to merge text edits, and the metadata describing rich text is separately managed as span objects, which get modified in accordance with text changes.
Firebase provides conflict resolution of JSON structures using their Transaction primitive, which allows them to create a Software Transactional Memory used to update the journal that powers the text OT in Firepad. ShareJS inside of Derby uses a similar approach to allow multiple servers to access the journal, but it is done in Redis with Lua scripting.
I really want Derby, or something like it (eg AirBnB's rendr combined with realtime, OT models and collections) to succeed, however it really seems like Derby needs more community support and evangelism behind it, because I can't help but get the impression that most of the community focus in this space currently is on Meteor. As far as I am concerned, Meteor is the PHP of realtime 'holy grail' frameworks, as enjoys from just the same kind of popularity through 'worse is better' (where 'worse' is for the reasons I mentioned above). However, without a palpable, visible sense of community enthusiasm, a lot of fence-sitters such as myself are less likely to jump on board.
Also guys please do something about racerjs.com, it doesn't inspire confidence in the project when it is continuously down.
Why would you start such a confusion? Why would you want to fight with that software for page ranking? Why does this mostly happen to JS technologies?
There are far more software project starts than good names out there. Finding a good name that is also not used by any other project can take a bunch of time and energy away from your code.
> Why would you want to fight with that software for page ranking?
The vast majority of software projects never make it far enough for the name to matter. You don't know which category your project will end up in when you start it.
> Why does this mostly happen to JS technologies?
Because JS is popular and makes it easy to start new ad-hoc code libraries. There are also a lot of half-baked libraries out there that will waste your time. So many developers end up making their own.
So true. I don't know how many times I've started building a native audio streaming app and all-of-a-sudden I realize I've really built a web-based real time strategy game.
Your points are valid... just thought that one statement was a bit absurd and quite humorous.
>The vast majority of software projects never make it far enough for the name to matter. You don't know which category your project will end up in when you start it.
The "categories" are:
1) Projects where the name will matter because the project is successful
and
2) Projects where the name won't matter because the project dies before reaching critical mass.
EDIT: apologies, this was supposed to be a reply to examancer
There are far more software project starts than good names out there.
You don't know which category your project will end up in when you start it.
Because JS is popular
There are also a lot of half-baked libraries out there that will waste your time. So many developers end up making their own.
A good name is pronoucable, spellable, memorable. Common English words can be good choices. Compare the number of common English words (thousands) with the number of Sourceforge, Github, Google Code, Codeplex, Bitbucket, etc. projects (millions).
> Ok, why not just name it jQuery then?
jQuery is a) perhaps the world's most popular project in the domain of Javascript libraries, and b) not a proper English word.
'Derby' is a relatively common English word and two Derby projects are in mildly different domains. I don't personally know of anyone using either project (unlike of course 'jQuery').
However, I will have to inform Mr. Cruise that his e-meter will not, in the near future, be able to predict his website's uptime.
Ahh, the tribulations of modern development.
Or do you mean something else? I'll admit to only really watching all of this from afar, I could have my terminology backwards.
As far as I know, Meteor doesn't even serve those PhantomJS rendered pages to the average user - only to crawlers - presumably for the reasons above.
Otherwise look into AirBnB Rendr and Yahoo Mojito if you want to support both client side and server side rendering.
However Sails wants to add OT's, and add-ons that do Meteor-style syncing. I know this is wishful thinking, but maybe Sails and Derby could find some way to merge, or otherwise work together?
Here's the Sails group - https://groups.google.com/forum/#!forum/sailsjs
But if that'd be the case, it's a very different thing from Derby/Meteor/SocketStream, where so much code is shared from client+server that the apps can often run offline and then sync data.
Aside from that, Sails brings a lot to the table (awesome ORM, asset pipeline, built-in API, etc). All we need are OTs and things'll be rockin :D
It essentially does the work of a Django + Tastypie with... no work.
The reason I ask is that I'm thinking of the feasibility of modelling a disconnected client that continues working from some state. Could you think of this as collecting its list of edits and then applying them on top of the state of the world when that client re-connects (sort of like a git rebase)?
Importantly, "edits" here are discrete and independent. They can conflict (in which case it just does last one wins), but they can't affect the meaning of other edits. This scheme couldn't be used for collaborative editing of a linear text document, since e.g. "insert 'a' at position 15" means something different depending on whether it comes before or after "delete positions 5-10".
That problem is what OT solves (by the transform function that lets you commute edits while preserving their meaning). So, yes, that kind of scheme is much simpler than OT, but less powerful. Whether you can use it depends on the semantics of your edits.
I love OT RT editing, but want to use it for small parts of my products and don't want the hassle of setting up servers.
If you're looking for your next idea; please, take that and run with it :)
This is very awesome stuff! =). The real-time syncing is definitely awesome, can't wait to try this in some of my apps.
Good props on the video tutorial as well.
One thing sorely lacking form DerbyJS is tutorials, or any kind of help for beginners - would be awesome to see them release more of these.
Cheers, Victor
I get that it's cool, but the question is why? What are the advantages??
OT is an algorithm that allows sending only changed parts of a document and resolving conflicts made if multiple users are editing the same thing.