Tailing the MongoDB Replica Set Oplog with Scala and Akka Stream

This is useful for any collection that is also a capped collection (that is, the collection has only a certain amount of finite space allocated to it), not just the oplog. In fact, the oplog is just a special capped collection, hence why tailable cursors are available for it.

Also note that non-capped collections don't support tailable cursors, so this won't work. Documentation: https://docs.mongodb.org/manual/tutorial/create-tailable-cur...

*Edit: clarified that the oplog is a capped collection for context

There is a nice higher-level Python wrapper around the oplog that is intended for syncing updates in MongoDB to other datastores [1].

We used this to enable streaming updates in MongoDB to Neo4j.[2]

[1] https://github.com/mongodb-labs/mongo-connector [2] https://github.com/neo4j-contrib/neo4j_doc_manager

I actually just wrote a node.js package that streams our MongoDB oplog to RethinKDB. It works quite well and based on the following NPM package https://github.com/cayasso/mongo-oplog.

The oplog is super useful for auditing and eventing.

This is such a complex way to do what's built into RethinkDB. I just love how useful the activity streaming is.

Some years, I wrote something similar: https://github.com/otto-de/mongodb-profiler.

*Akka Streams with an S