Hacker News Reader

Comment by madao | Hacker News Reader

I have to disagree with the logfiles, I work for a really large domain hosting company so our logs are quite vast, dealing with spam issues can amount to a large amount of time being used just looking for the information you need.

we have greylog in place to help query these logs from our syslog server, it has helped us identify alot of different issues since we have put it in place, not to be rude to the poster but I dont think you have the experience with really big log files and how painful it can be (or how long it can take) when you have to grep through this information by hand, zcating an old compressed log file can take up to ten minutes before you even start seeing data let alone relevant data.

View on HN

Your experience with logs doesn't invalidate the point the author was trying to make. Storing your logs in your production database is a stupid idea.

You're already doing what he was suggesting anyway - storing your logs somewhere other than your application database.

I'm trying to understand why storing your logs in the production database is such a bad idea. Is it because of potential disk contention? Lock issues? Could you be more specific about why it is a stupid idea?

Probably just disk contention - locking is unlikely to be a problem since you're just adding rows.

The disk contention issue could be a problem because writes are hard to scale, but depending on the regularity of logging and the indexing used (I'm assuming it's done by timestamp here) it's probably not actually the biggest of deals - you should end up mostly doing append-only writes. In oracle, for example, all that'll happen is that for each log transaction some stuff will get altered in memory (i.e. a block of the table will get appended to, and an append will occur on the index) and the roll forward log will get written to on disk. Every so often the table and index blocks will get written to disk, but that will be much less than once per transaction in an appending case.

That said, overall I'd still recommend having the logging done on a separate system, or at least having profiling set up on your log table/user/whatever such that logging can't eat your production server if someone decides to excessively ramp up the log levels.

Why not just move the logging table into a different tablespace/file group and place this on a different disk? That would fix write performance problems.

This is absolutely a good thing to do, but it doesn't make you safe - the disks running the database's redo logs are the ones that get hammered in a logging scenario (since the writes to the table and indexes are largely appending, and don't have to be written to disk to commit a transaction). These logs are usually global to the DB instance, so you can't easily (to my knowledge) separate them out. This being the case, they can significantly affect the performance of the rest of your system if too much logging starts happening.

Reply on news.ycombinator.com