CouchDB ships 1.2.0, Windows packages, new website
couchdb.apache.org
couchdb.apache.org
I think the new website is fantastic, as I now know all of this and more just from a quick glace. Great work.
I gave up on CouchDB two years ago. Partly it was a poor experience with the community, but mostly technical issues. Writing is done to append only journals. This means not only the added/changed data is written but also all changed btree nodes. I was starting with around 2GB of JSON data (10 million documents). With CouchDB it would turn into 43GB on disk (including views/indices). With tuning, tweaks and compaction I was able to get it to half of that, but it was still a massive explosion of data size.
I tried MongoDB with the same data and ended up with just over 4GB consumed on disk. In addition my data load time was 6 minutes compared to 75 minutes for CouchDB. I was even experiencing unindexed queries in MongoDB being faster than indexed ones in CouchDB!
I'm sure CouchDB has improved a bit since then, but I don't think the implementation can do much better especially at the data sizes I was doing. (Personally I think that is a rather small amount of data, one of the CouchDB developers called a million documents large!)
I noticed that it is almost identical to Apache Cordova's [1]. Is there an ongoing effort to bring a consistent look to all Apache projects websites?
If other projects want to co-ordinate with us, I would welcome that.
To me, that'd be a much more compelling reason to use CouchDB, since syncing between server and client is a pain, and having CouchDB on both servers and client would make development life much easier for me.
If not, then publish the replication protocol. I tried looking for it once, and only found scant information on the mailing lists and documentation.
http://www.dataprotocols.org/en/latest/couchdb_replication.h... https://github.com/couchbaselabs/TouchDB-iOS/wiki/Replicatio...
From the end of http://wiki.apache.org/couchdb/Replication
While building TouchDB Jens documented the replication protocol here: https://github.com/couchbaselabs/TouchDB-iOS/wiki/Replicatio...
I am working on (actually am right now) the in browser implementation of CouchDB called PouchDB (https://github.com/mikeal/pouchdb) it is nearing a beta release.
I'll see what I can do about getting a good publication of the replication protocol. People have been asking about it for a long time. I've only come to fully grok it late last year when I tried to write PouchDB and I talked through it with Dale when he picked up my slack. We can probably get that together.
A great first example that I'd love to see is using PouchDB to implement Backbone.Sync for a Backbone app.
Its very basic but has a working Todo example.
The focus client-side is on speed, battery-life, ease of programming, and perhaps most of all, maintaining separation of concerns between the server-side data model and the client data model. The server must be free to evolve independently of the client, and many remote clients will run legacy versions of your software for extended periods of time.
Moreover, rarely is data distribution sufficient to model high-level interactions between client and server, which is more likely to introduce bugs, complexity, and significantly detract from the long-term maintainability of the client/server protocol.
The end result is that you don't particularly benefit from CouchDB client-side, and especially not replication -- you need a well-defined supportable network model.
As far as the maintainability concerns, it's an interesting point. There are certainly apps which may want to sacrifice the ease of use and performance advantages of having sync done for you, because they have specialized data sharing requirements.
But for a whole lot of apps the benefits of sync far outweigh the contraints of having the same data model on the client and server.
Some example use cases where sync can simplify programming (or even make a class of apps possible which would be much harder without it): point-of-sale, retail, medical records (doctor with an iPad and limited wifi), military (need that data no matter what).
Here's a pdf of the slides I presented yesterday at Chicago's WindyCityGo conference: http://dl.dropbox.com/u/14074521/syncpoint-windycity-small.p...
You're just moving the costs around, and likely transmitting far more data than needs be sent.
In addition, you're front-loading the transmission of data that users don't need right now.
Having built many network-facing mobile applications with exceedingly large user bases, I'm unconvinced. The maintenance complexity, the excess transmission of data, issues of data consistency over time, the lack of integration with client-side business rules without considerable investment in code that significantly replaces the sync logic.
I think that the reality of the situation is that in the rare cases you need offline, fully synchronized data, it's not particularly difficult to just download what you need as an atomic unit and save it. The majority of the time, however, on-demand request handling is more efficient, well-defined, and maintainable.
As far as ease of writing apps (and maintaining them), time will tell, some of our users have been at it for a few years now. The things I've seen people do, show me the synchronizing document model works.
[edit] To be clear - Couchbase Server is mostly used for the explicit network model you advocate. For instance, the OMG POP mobile success story is using Couchbase as regular backend database storage.
I like the layout and high contrast of the new site. My only comment is perhaps a header link to the "Quick Links" at the footer. I tend to want to look at documentation quite a bit and the scroll to the footer was not intuitive.
When I read Damien's note back in January, I was prepared to never see updates to CouchDB.
Apache CouchDB remains the original CouchDB.
It is where most of the developers are, and where most of the community is.
The one and only! HUZZAH! ;)
However, it is worth to mention that the database file format has changed. Once on 1.2 your files get upgraded to the new format. There is no going back.
-- [1] when Safari Reader fails to capture the main content
Highlights I like:
* Speed improvements (native JSON parser)
* Autocompaction (no need to create daemons to compact db periodically anymore).
* Snappy compression enabled by default (should reduce db sizes and perhaps provide some speed improvements)
* Added built-in changes feed filter _view. This basically means you can use your view to filter your changes feed.