The Logging Mess in Java | Hacker News Reader

The Logging Mess in Java

java.dzone.com

40 pointsgulbrandr21 comments

Someone should invent a name for this problem. It's the reason we get a zillion test frameworks, untold thousands of database layers and uncountable reimplementations of linked lists. I can't get through the code of a sufficiently large system without running into someone's crappy command line option parser package, more likely two or three. Don't get me started on rewrites of functions that are already in the library (strcmp, I'm looking at you).

"All these existing packages are terrible; nothing that exists measures up. I'm going to write my own that is just going to /rock/."

That seems to be the issue; a low bar for entry, and it's hard to wade through the spam and the chaff of stuff that's already around and find something that will work well. It's a little like the satellite debris problem we have in LEO right now; there's an inflection point beyond which the sea of junk starts spawning new garbage.

Basically, when you have programmers, you get code. For years that was Microsoft's problem ("We have all these people, and they just won't. stop. typing! Augggh!")

The strength and the weakness of Java is the huge amount of open source code that's out there. Logging is the worst example, but you can find it anywhere. It's not unusual for a modern web application in Java to have five or more JSON codecs built in. PHP might have a stupid API, but PHP programs regularly use json_encode() and json_decode() and you never need to think about different behaviors in corner cases and different ways to deal with the impedance mismatch between static Java and dynamic JSON.

There are a some references here which will help identify the problem:

http://en.wikipedia.org/wiki/Not_Invented_Here

http://en.wikipedia.org/wiki/Analysis_paralysis

http://en.wikipedia.org/wiki/Tyranny_of_small_decisions

I think it's more like this:

http://xkcd.com/927/

Ha I like that :)

What I miss in (almost?) all the logging framework is the concept of logging structured data types for easy machine parsing and analysis. All the logging packages are focused toward formatting and logging short text messages for humans to read.

This is nice initially, but with large volumes of logging you want to perform statistics (what errors / error codes are most common? which users make most use of a certain API call? etc). In which case it's kind of duplicate work to write formatters and parsers for every message type.

I know there is the "RFC5424 structured logging" standard but it's not really widely supported yet. So generally I end up dumping to some JSON-based format instead of using a logging toolkit.

Then again, I'm not up-to-date in Java logging. Please enlighten me if I'm overlooking something.

SLF4J and java.util.logging take string messages, but commons-logging and log4j take objects, so one could add custom appenders to handle non-string messages.

It's not Java specific, but there's some work out there on standards for logging. IBM rolled out this big thing called "CBE" for "Common Base Events"[1],[2] a few years ago, as part of their "Autonomic Computing" initiative. Eventually that somehow got all rolled up into one of the many WS-* specs, and become part of something called WSDM-WEF for "Web Services Distributed Management - Web Event Framework."[3],[4]. There's an OASIS spec[5] for this stuff, and an ASF project - "Muse"[6] - to provide a Java implementation. There are also classes to implement this stuff in the Eclipse Hyades[7] project.

[1]: https://www.ibm.com/developerworks/wikis/display/CBEbestprac...

[2]: http://www.ibm.com/developerworks/autonomic/books/fpy0mst.ht...

[3]: http://en.wikipedia.org/wiki/Common_Base_Event

[4]: http://en.wikipedia.org/wiki/Web_Services_Distributed_Manage...

[5]: http://www.oasis-open.org/committees/tc_home.php?wg_abbrev=w...

[6]: http://ws.apache.org/muse/

[7]: http://archive.eclipse.org/tptp/4.0.0/javadoc/Platform/publi...

I am developing Java for food for around 6 years and it's impressive how much time I spent dealing with different logging implementations and configurations. Almost every time I step in an existing project there is a "creative" way to configure it (and for some reason this configuration does not work properly).

I agree wholeheartedly with this. I want to add that when you can't get logging working it complicates other troubleshooting because when something is not working the first thing you should do is "look in the logs". I wish there was some way to bring up logging in a kind of safe mode so that it wouldn't do all the fancy things but at least you could get either console or simple log files.

More than once I have had to look at some java code running is a context or environment that I was not familiar with and had so much trouble getting logging working that I wrote my own quick and dirty logger to debug the original issue. I never tried to turn these throw away logging implementations into a stand alone project or anything. It took about 20 lines of code to get a simple one working. That is deterministic, I can write that kind of code in no time. Getting the real logging working seems non-deterministic sometimes. Too many times I have had to go to google and end up learning something about logging. Learning something that I will never use again.

If this whole mess is entirely built of libraries built by this one guy (Ceki Gülcü), can't we just stop him?

:-)

Just others great projects from him ( and maybe others):

Compiler Assisted Localization, abbreviated as CAL10N (pronounced as "calion") is a java library for writing localized (internationalized) messages using resource bundles you are already familiar with, but with much greater comfort. http://cal10n.qos.ch/

Mistletoe is a junit extension intended for integration testing. In technical terms, mistletoe is a junit test suite runner presenting the test results via HTTP as a web-page. http://mistletoe.qos.ch/

Funny, but it's not all on Ceki. The folks who created java.util.logging instead of simply adopting log4j, and the people who created commons-logging and the couple of other miscellaneous frameworks that are floating around, also bear some responsibility for this mess.

I've been using JUL in libraries and Logback in new applications (Still using log4j in some legacy ones). It's worked out pretty well, keeping any logging dependencies/configuration out of my reusable code while not putting much/any extra baggage on the apps themselves.

I guess I should do a followup to this: http://efsavage.com/blog/posts/logging-like-its-2002/

I don't think this problem only applies to Java... .Net has the same problem, and I am sure this problems exists in most languages as well.

This is a problem that definitely needs fixing. It makes me wonder why I haven't spent any time trying to fix it, even if I have spent huge amounts of time building it into applications.

I'm not sure if clojure's tools.logging helps or hurts in this regard. It adds another layer to the onion, though with the aim of leveraging macros' non-evaluation of the logging args.

http://github.com/clojure/tools.logging

I really like this idea: http://www.chrononsystems.com/post-execution-logging/

It looks interesting. I'll have to check it out.

This is the typical result of "enterprise" applications which are more interested in decoupling the logging system from the framework versus picking one that works.

Over-abstraction at work.

There is a "commons logging" for .Net as well which replicates the entire log4net API 1:1. That is where the crazy starts to set in.

That's a reasonable (if knee-jerk) first reaction. But reality is a bit more complicated.

log4j was the first de facto standard (excluding System.out). Making it part of the standard API would have been a no-brainer. Unfortunately, java.util.logging sucks (hard to customize, coupling with J2EE app servers). So people keep using log4j.

Library writers now have a problem: How do you choose which logging framework to use? You really want to use the same framework as your users, the developers making apps. Thus was born commons-logging.

commons-logging has a number of problems, including non-deterministic setup and being bundled with some common app servers (which creates its own share of problems).

Thus: slf4j. Which works.

I agree with the author that slf4j is the way to go. Alternatively you can just use log4j directly. But slf4j has one feature which log4j doesn't: format strings. If you've ever had to add "if (log.isDebuggingEnabled())" checks to logging code you'll appreciate what a difference this makes.

In conclusion: Java logging is a clusterfuck, but if you know what to use you'll be fine. The end.

Even for ruby you can find many logging libraries:

https://github.com/parolkar/active_log

https://github.com/colbygk/log4r

https://github.com/TwP/logging

https://github.com/relevance/log_buddy

I believe one of the reasons is that logging is not a simple problem, but it is ubiquitous to all applications.

It is not simple because:

1. your logging is usually a crosscutting concern to applications, and is really fit for Aspect Oriented Programming Mechanisms.[1] In language with mixins, like Ruby and Scala, this is not a big issue, as you don't need full AOP to solve this. AspectJ is not really widespread, and adds quite a lot of complexity to a small project, and is not standard Java, which makes a bad situation worse on Java.

2. your logging must be easy to disable, and only compute what it is actually needed. With languages with closures and Ast Metaprogramming, like Lisp Macros, you can do this very easily. Scala is particularly suitable to this, as you can use call by name[2] semantics to make the closures implicit.

[1] http://en.wikipedia.org/wiki/Aspect-oriented_programming

[2] http://www.scala-lang.org/node/138

Reply on news.ycombinator.com