It becomes a problem when you: (1) scale up (2) have to deal with multiple relationships between objects. The "Database design" by Adrienne Watt posted in another comment covers the scale concerns well, but another scale problem she doesn't mention is hitting inode limit, at least if you're on a single machine. You can of course use a distributed filesystem as database, but at that point, you might want to use a database proper.