Extending Tokyo Cabinet DB with Lua

igvita.com

28 points·igrigorik··5 comments

For those considering TT:

* Do not use Tokyo Tyrant in a production setting without replication. Make frequent backups.

* as far as I can tell, you have to call sync manually.. if you don't and the writes fill up your memory, writes slow to a crawl and there's a chance that the db file will corrupt when you finally sync or terminate the daemon.

* if you run out of disk space, your db will become corrupt

* if you accidentally start two instances of tokyo tyrant on the same file, it may become corrupt

* If you're using the hash db, be sure to tune the bnum param. If you're working with a lot of data, tune xmsiz as high as possible.

* 'putlist' stores duplicate records for b+ tree, which may or may not be what you expect.

* "bulk loading" the db is much much slower than mysql's select data infile

great tips, kudos!

I am considering using TT in production and I am curious why you are stressing on backup and corruption. Are you giving out these tips like generally good practices or are you referring it especially for TT because of its inherent properties?

I don't think they are specific to TT/TC, but are just good practices in general; I have been burned by corrupted Berkeley DB databases so many times that I now refuse to use it for any project where persistence actually matters. The one variable that TT adds to the equation is that your application code may not be local to the system running the DB, so it is easy to forget that if you don't sync your data no one else will.

I really like the idea of being able to customize TC for whatever specific requirement is needed. That's something that most data stores are not very good at (and some may argue should not be good at), but it seems pretty powerful when it's there.

Reply on news.ycombinator.com