Hacker News Reader

Comment by sigil | Hacker News Reader

Don't use logging.py. It's awful, and the complete opposite of the "do one thing and do it well" philosophy.

Consider:

    $ python
    >>> import sys
    >>> x = set(sys.modules.keys())
    >>> import logging
    >>> y = set(sys.modules.keys())
    >>> len(y - x)
    31

Yes that's right, you're importing 31 new modules just to log a string. This is only the beginning, I could go on and on.

At most python shops I've worked, we replace logging.py with a 100-200 line module that outputs strings to stderr at levels configurable per-module. You can pipe that logging output into programs that timestamp each line, or ship them over the network, or send email notifications on errors, etc. There are plenty of good reasons why your program shouldn't concern itself with the delivery of logging messages beyond sending them to stderr.

Unfortunately, this particular battery included is a D, even if you only needed an AAA.

View on HN

Yes that's right, you're importing 31 new modules just to log a string.

Does it really matter? Maybe the use of namespaces and such isn't ideal, but let's be honest, if importing a few modules is going to make someone lose any sleep, Python isn't really the language for them anyway.

At most python shops I've worked, we replace logging.py with a 100-200 line module that outputs strings to stderr at levels configurable per-module.

Well, OK, that's your choice.

In contrast, we have a system built on the standard Python logging package that provides several loggers. Each logger has its own customised formatting, including tidy recording of multi-line log entries if required. Each logger records to (among other things) different files on disk. We're also setting up e-mail alerts and database-backed logging at the moment for some loggers/levels. Since the originating log requests come from multiple web server processes, we co-ordinate everything by running a centralised control process that runs the real handlers, accepting log entries from any of the other processes via a socket, thus avoiding any nasty race conditions with the concurrency.

We use a couple of tricks that aren't available out-of-the-box, but since we're using the standard logging package it was easy to find web pages with working examples of how to do similar things and implement what we needed for our own system.

The code required to record a log entry is still the same one-liner it would be to print something simple to stdout. The total overhead for all of the multi-process, multi-handler, multi-formatted logging system is just over 100 lines.

I can't speak for anyone else, but I doubt I'm a good enough programmer to achieve all of that functionality in 100-200 lines starting from scratch. Even if I were, I doubt I could implement it in only the couple of hours we spent setting up the system I've described here.

Python has its shared of dubious included batteries, but IMHO a powerful, flexible logging framework is one of its stronger assets. Sure, it's mildly irritating to type a few lines of boilerplate if you want to use the same system for something that doesn't really need all of that power and flexibility, but even then, it's probably fewer than a dozen lines of code that you're going to type once in the entire lifetime of a project, or that you're going to shove into your own logger module and just import every time you start a new quick-and-dirty project.

the complete opposite of the "do one thing and do it well" philosophy.

This is like saying subprocess is a mess because it does more than just call system(). It does one thing: logging, and it does it well.

There are plenty of good reasons why your program shouldn't concern itself with the delivery of logging messages beyond sending them to stderr.

So who, outside my program, am I to give information about which messages should go to syslog, which to stderr, and which sent to a remote logserver? You're advocating the use of dumb pipes and message parsing for something that can be perfectly well done in-process, where all the state is available.

When I had to optimise some Python code, the logging module was the first module that got replaced. I have no idea what it was doing, but it was causing a 10x or so hit to performance. Writing my own logging.py (with the same API so I didn't have to change any other code) gave me a massive improvement.

the trouble is that the alternative is a pile of different solutions. it gets worse when 3rd party libraries dream up their own approaches. if everyone used the standard life would be much easier.

the advantages of a consistent log interface across an entire system outweigh the poor implementation and irritable maintainer.

i guess what we really need is a new, better implementation of the same api...

if you take all the python libraries, multiply them all by the five or six lines of completely custom, non-standard, everyone-does-it-differently lines of logging code you're encouraging everyone write, then we have way, way more than 100-200 lines of logging code. Except unlike logging.py's lines of code, none of it works together.

Reply on news.ycombinator.com