It's mostly because I remember filesystems from my university time and trying to implement one in a file myself. Continuously adding data is fine, but when you remove data either you leave a "hole" in the file and have to keep track of the "empty" blocks, or you shift everything to fill the hole. Off-by-one errors corrupt files.
Those are all resolved problems for filesystems, but additionally, I learned the hard way (performance) that SQLite wasn't good as a production db. The situation here is different of course (db access isn't very frequent), but I can't quite shake it.
Maybe I'll dig into the internals to understand what kind of optimisations they made. The wiki and bug-tracking are definitely a plus though.