Pretty nice tool :-)
But what if the data store would do all of that already?
- retaining the history of your data:
- free deduplication through copy on write and sharing unchanged structures
- further store space savings through page-level versioning (not all records in a page have to be written: just the changed/inserted/deleted ones plus depending on the algorithm a few more to fast track it's reconstruction)
- read and write performance is balanced using a sliding snapshot algorithm: This avoids having to read long chains of incremental page changes and it avoids write peaks from intermittant full page snapshots. Instead they get trailed over a few (usually three or four revisions).
- the sorage manager stores timestamps for all revision in a RevisionRootPage, which itself is indexed
- Optional commit author and message
- Reverting to and committing a past "good" revision is just adding a few bytes to index a new RevisionRootPage which references the past revision data (simple pointer)
- You can diff taking rolling hashes into account
- Currently I'm working on a diffing JSON format and a web front-end to display these in special space filling approaches
- Sophisticated time travel queries
- typed, user-defined and versioned secondary index structures (based on AVL trees)
- a path summary kept up to date always
- a lot of metadata stored and kept up to date for each record/node
It's already implemented in https://sirix.io / https://github.com/sirixdb/sirixI'm working on getting it stable, but help from you, the Open Source community is always needed :-)