Extending Log4J

# Adding Conversion Characters to PatternLayout

"Paul Glezen" January 2001

## Abstract

This article describes a systematic way to extend the log4j API to include additional attributes formatted using the `PatternLayout` class.

## Contents

- Introduction
  - The Case Study
  - A Peek Under the Hood
- The Process
  - Extending `LoggingEvent`
  - Extending `PatternLayout`
  - Extending `PatternParser` and `PatternConverter`
  - Extending `Category`
  - Extending `CategoryFactory`
- Usage
  - Configurators
- Further Enhancements

## Introduction

This article assumes familiarity with the log4j User Manual. It builds on fundamental classes described in both the User Manual and the Javadoc API. To assist in illustrating the concepts, a simple case study will be developed along side the explanations. The resulting classes may be used as a template for your own extensions. Condensed (i.e. statements compressed, comments removed) snippets of the case study code are included in this document.

### The Case Study

The case study was developed in a CORBA environment in which the following information for each log entry was needed. The letters in parenthesis represent the corresponding character to be used by the `PatternLayout` class for formatting.

- Host Name (h) - the IP address or hostname of the physical machine on which the Category was running.
- Server Name (s) - The name of the application server process. In this context, a server refers to a process that accepts requests rather than referring to a machine. The term host refers to a physical machine. Several servers may run on the same host.
- Component Name (b) - Rather than getting bogged down on what constitutes a component, let's assume for the case study that a component is a unit of software worth denoting in the logs.
- Version (v) - the version of the component from which the log entry originated.

It seems odd to use "b" for the component name. Presently `PatternLayout` already defines both "C" and "c" for class name and category name respectively.

### A Peek Under the Hood

In principle, if the steps described below are followed closely, there is not a need to understand how the extended classes will be used by log4j. But sometimes software development can be entirely unprincipled. You may wish to extend log4j in a different manner than describe here or you may make a mistake that requires knowledge of what is really going on. (Heaven forbid there be a mistake in this document). In any case, it doesn't hurt to get an idea of what's going on.

The following describes a "typical" logging scenario in the un-extended log4j case.

1. Application code invokes a log request on a `Category` object. Let's say the `info` method was invoked.

2. The first thing `info` does is to check if logging has been turned off entirely for the info level. If so, it returns immediately. We'll assume for this scenario that logging has not been turned off for the info level.

3. Next `info` compares the `Priority` level for this category against `Priority.INFO`. Assuming the priority warrants logging the message, the category instantiates a `LoggingEvent` object populated with information available for logging.

4. The `Category` instance passes the `LoggingEvent` instance to all its `Appender` implementations.

5. Most (but not all) `Appender` implementations should have an associated subclass of `Layout`. The `Layout` subclass is passed the `LoggingEvent` instance and returns the event's information formatted in a `String` according to the configuration of the `Layout`.

   When the `Layout` subclass is `PatternLayout`, the format of the event's information is determined by a character sequence similar to the C language library's `printf` routine. `PatternLayout` delegates the parsing of this character sequence to a `PatternParser` instance.

   When the `PatternLayout` was constructed, it created a `PatternParser` to tokenize the character sequence. Upon recognizing a token, the `PatternParser` constructs an appropriate `PatternConverter` subclass, passing it formatting information from the token. Often the `PatternConverter` subclasses are implemented as static inner classes of `PatternParser`. The `parse` method of the `PatternParser` returns a linked list of these `PatternConverter` subclasses.

6. `PatternLayout.format()` passes the `LoggingEvent` to each `PatternConverter` subclass in the linked list. Each link in the list selects a particular item from the `LoggingEvent`, converts this item to a `String` in the proper format and appends it to a `StringBuffer`.

7. The `format` method returns the resulting `String` to the `Appender` for output.

The above discussing involved most of the classes that we must extend or implement.

- `org.apache.log4j.PatternLayout`
- `org.apache.log4j.Category`
- `org.apache.log4j.spi.CategoryFactory`
- `org.apache.log4j.spi.LoggingEvent`
- `org.apache.log4j.helpers.PatternParser`
- `org.apache.log4j.helpers.PatternConverter`

## The Process

Below are steps required to add additional attributes available for logging by extending log4j. This will allow you to specify their output formats in the same manner as those provided by the `PatternLayout` class. The steps are numbered for reference only. It makes no difference in which order they are followed.

It's helpful if you know the attributes you wish to add and a `PatternLayout` symbol for each one before you begin. Be sure to consult the `PatternLayout` documentation to ensure the symbols you select are not already in use.

Before we dig in, I should give the standard lecture on comments. If the log4j library were not well documented, it would be useless to everyone but the log4j creators; likewise with your extensions. Much like eating vegetables and saving the environment, we all agree commenting code properly should be done. Yet it is often sacrificed for more immediate pleasures. We all write code faster without comments; especially those pesky Javadoc comments. But the reality is that the utility of undocumented code fades exponentially with time.

Since the log4j product comes with Javadoc comments together with the documentation it produces, it makes sense to include Javadoc comments in your extensions. By their very nature, logging tools are strong candidates for reuse. They can only be independently re-used if they are supported by strong documentation component.

This all having been said, I have elected to remove most comments from examples in the interest of space rather than including them to serve as a nagging reminder. The reader is referred to the case study source code files for a Javadoc version and a Javadoc website for more information on Javadoc conventions.

### 1. Extending `LoggingEvent`

Extending the `LoggingEvent` class should be one of the trivial steps. All that is needed in the extension is the addition of public data members representing the new attributes and a new constructor to populate them.

    import org.apache.log4j.Category;
    import org.apache.log4j.Priority;
    import org.apache.log4j.spi.LoggingEvent;

    public class AppServerLoggingEvent extends LoggingEvent
                                       implements java.io.Serializable
    {
       public String hostname;
       public String component;
       public String server;
       public String version;

       public AppServerLoggingEvent( String    fqnOfCategoryClass,
                                     AppServerCategory  category,
                                     Priority  priority,
                                     Object    message,
                                     Throwable throwable)
       {
          super( fqnOfCategoryClass,
                 category,
                 priority,
                 message,
                 throwable );

          hostname  = category.getHostname();
          component = category.getComponent();
          server    = category.getServer();
          version   = category.getVersion();
       }
    }

The constructor demonstrates that in most cases, the `Category` subclass will contain most of the information necessary to populate the attributes of the `LoggingEvent` subclass. Extensions to `LoggingEvent` seem no more than a collection of strings with a constructor. Most of the work is done by the super class.

### 2. Extending `PatternLayout`

Extending the `PatternLayout` class should be another simple matter. The extension to `PatternLayout` should differ from its parent only in the creation of a `PatternParser` instance. The extended `PatternLayout` should create an extended `PatternParser` class. Fortunately, this task in `PatternLayout` is encapsulated within a single method.

    import org.apache.log4j.PatternParser;
    import org.apache.log4j.PatternLayout;

    public class AppServerPatternLayout extends PatternLayout
    {
       public AppServerPatternLayout()
       {
          this(DEFAULT_CONVERSION_PATTERN);
       }

       public MyPatternLayout(String pattern)
       {
          super(pattern);
       }

       public PatternParser createPatternParser(String pattern)
       {
          PatternParser result;
          if ( pattern == null )
             result = new AppserverPatternParser( DEFAULT_CONVERSION_PATTERN );
          else
             result = new AppServerPatternParser ( pattern );

          return result;
      }
    }

### 3. Extend `PatternParser` and `PatternConverter`

Recall from our peek under the hood that the `PatternParser` does much of its work in its `parse` method. The `PatternLayout` object instantiates a `PatternParser` object by passing it the pattern string. The PatternLayout then invokes the `parse` method of `PatternParser` to produce a linked list of `PatternConverter` subclass instances. It is this linked list of converters that is used to convert an event instance into a string used by appenders.

Our job will be to subclass `PatternParser` to properly interpret formatting characters we wish to add. Fortunately, `PatternParser` has been designed so that only the one step in the parsing process differing for each formatting character has to be overridden. The grunt work of parsing is still performed by the `PatternParser.parse()` method. Only the `PatternParser.finalizeConverter` method has to be overridden. This is the method that decides which `PatternConverter` to create based on a formatting character.

The extension to `PatternParser`, `AppServerPatternParser`, is similar to its super class. It uses

- constants to identify the various formats
- a converter defined as private static inner-classes of `AppServerPatternParser`.
- a `finalizeConverter` method which instantiates the appropriate converter for a given format character.

`AppServerPatternParser` differs principally by dedicating a separate converter type for each logging attribute to be formatted. Rather than placing switch logic in the converter, like its parent class, each converter only converts one format character. This means the decision of which converter subclass to instantiate is made at layout instantiation time rather than in a switch statement at logging time.

It also differs in that the format constants are characters rather than integers.

    import org.apache.log4j.*;
    import org.apache.log4j.helpers.FormattingInfo;
    import org.apache.log4j.helpers.PatternConverter;
    import org.apache.log4j.helpers.PatternParser;
    import org.apache.log4j.spi.LoggingEvent;

    public class AppServerPatternParser extends PatternParser
    {
       static final char HOSTNAME_CHAR  = 'h';
       static final char SERVER_CHAR    = 's';
       static final char COMPONENT_CHAR = 'b';
       static final char VERSION_CHAR   = 'v';

       public AppServerPatternParser(String pattern)
       {
          super(pattern);
       }

       public void finalizeConverter(char formatChar)
       {
          PatternConverter pc = null;
          switch( formatChar )
          {
             case HOSTNAME_CHAR:
                pc = new HostnamePatternConverter( formattingInfo );
                currentLiteral.setLength(0);
                addConverter( pc );
                break;
             case SERVER_CHAR:
                pc = new ServerPatternConverter( formattingInfo );
                currentLiteral.setLength(0);
                addConverter( pc );
                break;
             case COMPONENT_CHAR:
                pc = new ComponentPatternConverter( formattingInfo );
                currentLiteral.setLength(0);
                addConverter( pc );
                break;
             case VERSION_CHAR:
                pc = new VersionPatternConverter( formattingInfo );
                currentLiteral.setLength(0);
                addConverter( pc );
                break;
             default:
                super.finalizeConverter( formatChar );
          }
       }

       private static abstract class AppServerPatternConverter extends PatternConverter
       {
          AppServerPatternConverter(FormattingInfo formattingInfo)
          {
             super(formattingInfo);
          }

          public String convert(LoggingEvent event)
          {
             String result = null;
             AppServerLoggingEvent appEvent = null;

             if ( event instanceof AppServerLoggingEvent )
             {
                appEvent = (AppServerLoggingEvent) event;
                result = convert( appEvent );
             }
             return result;
          }

          public abstract String convert( AppServerLoggingEvent event );
       }

       private static class HostnamePatternConverter extends AppServerPatternConverter
       {
          HostnamePatternConverter( FormattingInfo formatInfo )
          {  super( formatInfo );  }

          public String convert( AppServerLoggingEvent event )
          {  return event.hostname;  }
       }

       private static class ServerPatternConverter extends AppServerPatternConverter
       {
          ServerPatternConverter( FormattingInfo formatInfo )
          {  super( formatInfo );  }

          public String convert( AppServerLoggingEvent event )
          {  return event.server;  }
       }

       private static class ComponentPatternConverter extends AppServerPatternConverter
       {
          ComponentPatternConverter( FormattingInfo formatInfo )
          {  super( formatInfo );  }

          public String convert( AppServerLoggingEvent event )
          {  return event.component;  }
       }

       private static class VersionPatternConverter extends AppServerPatternConverter
       {
          VersionPatternConverter( FormattingInfo formatInfo )
          {  super( formatInfo );  }

          public String convert( AppServerLoggingEvent event )
          {  return event.version;  }
       }
    }

### 4. Extending `Category`

Extending `Category` and its factory will be more straight forward than extending `PatternParser` and the converters. The following tasks are involved in overridding `Category` for our purposes.

- Add fields corresponding to the new attributes with their corresponding getters and setters.

- Override the constructor to accept or acquire information about new attributes.

- Override the `forcedLog` method to ensure that a correctly populated instance of `AppServerLoggingEvent` is instantiated rather than the default `LoggingEvent`.

- Override the `getInstance` method to use our `CategoryFactory` (described in the next step). This will require that we hold a static reference to our factory and provide a way to initialize it.

Most of the code below is standard getter/setter verbage which has been somewhat abbreviated. The notable parts are in bold. We add five more attributes to `Category`: the four new logging attributes plus a static `AppServerCategoryFactory` reference. This is pre-initialized to an instance with attributes set to null as a precautionary measure. Otherwise the `getInstance` method will result in a null pointer exception if invoked before the `setFactory` method.

The `getInstance` method simply invokes its parent class method that accepts a `CategoryFactory` reference in addition to the category name.

The `forcedLog` method follows closely the corresponding parent class method. The most important difference is the instantiation of the `AppServerLoggingEvent`. A minor yet necessary difference is the use of the `getRendererMap()` method rather than accessing the data member directory as in `Category`. `Category` can do this because the rendererMap is package level accessible.

The `setFactory` method is provided to allow application code to set the factory used in the `getInstance` method.

    import org.apache.log4j.Priority;
    import org.apache.log4j.Category;
    import org.apache.log4j.spi.CategoryFactory;
    import org.apache.log4j.spi.LoggingEvent;

    public class AppServerCategory extends Category
    {
       protected String component;
       protected String hostname;
       protected String server;
       protected String version;
       private static CategoryFactory factory =
                      new AppServerCategoryFactory(null, null, null);

       protected  AppServerCategory( String categoryName,
                                     String hostname,
                                     String server,
                                     String component,
                                     String version )
       {
          super( categoryName );
          instanceFQN = "org.apache.log4j.examples.appserver.AppServerCategory";

          this.hostname  = hostname;
          this.server    = server;
          this.component = component;
          this.version   = version;
       }

       public String getComponent()
       { return (component == null ) ? "" : result; }

       public String getHostname()
       {  return ( hostname == null ) ? "" : hostname; }

       public static Category getInstance(String name)
       {
          return Category.getInstance(name, factory);
       }

       public String getServer()
       {  return ( server == null ) ? "" : server; }

       public String getVersion()
       {  return ( version == null ) ? "" : version; }

       protected void forcedLog( String    fqn,
                                 Priority  priority,
                                 Object    message,
                                 Throwable t)
       {
          LoggingEvent event = new AppServerLoggingEvent(fqn, this, priority, message, t);
          callAppenders( event );
       }

       public void setComponent(String componentName)
       { component = componentName; }

       public static void setFactory(CategoryFactory factory)
       { AppServerCategory.factory = factory; }

       public void setHostname(String hostname)
       { this.hostname = hostname; }

       public void setServer(String serverName)
       { server = serverName; }

       public void setVersion(String versionName)
       { version = versionName; }
    }

### 5. Extending `CategoryFactory`

The last step is to provide an implementation of the `CategoryFactory` interface that will correctly instantiate our `AppServerCategory` objects. It will obtain the hostname of the machine on which it runs using the `java.net` API. Aside from providing getters and setters for the attributes introduced, the only method to be implemented is the `makeNewCategoryInstance`.

Below is a snippet from `AppServerCategoryFactory` with getters, setters and comments removed.

    import org.apache.log4j.Category;
    import org.apache.log4j.spi.CategoryFactory;
    import java.net.InetAddress;
    import java.net.UnknownHostException;

    public class AppServerCategoryFactory implements CategoryFactory
    {
       protected String hostname;
       protected String server;
       protected String component;
       protected String version;
       protected ResourceBundle messageBundle;

       protected  AppServerCategoryFactory( String serverName,
                                            String componentName,
                                            String versionName )
       {
          try
          {
             hostname = java.net.InetAddress.getLocalHost().getHostName();
          }
          catch ( java.net.UnknownHostException uhe )
          {
             System.err.println("Could not determine local hostname.");
          }

          server    = serverName;
          component = componentName;
          version   = versionName;
       }

       public Category makeNewCategoryInstance(String name)
       {
           Category result = new AppServerCategory( name,
                                                    hostname,
                                                    server,
                                                    component,
                                                    version);
           return result;
       }
    }

## Usage

We now arrive at how to use what we have created. We must remember to initialize log4j by creating an instance of `AppServerCategoryFactory` and passing it to `AppServerCategory`. Once done, we can obtain a `AppServerCategoryInstance` anytime by using the static `getInstance` method of `AppServerCategory`. This will ensure that `AppServerLoggingEvent` instances are generated by the category logging methods.

    import org.apache.log4j.*;
    import org.apache.log4j.appserver.AppServerCategory;
    import org.apache.log4j.appserver.AppServerCategoryFactory;
    import org.apache.log4j.appserver.AppServerPatternLayout;

    public class test
    {
       private static String formatString =
          "---------------------------------------------------%n" +
          "Time:      %d%n" +
          "Host:      %h%n" +
          "Server:    %s%n" +
          "Component: %b%n" +
          "Version:   %v%n" +
          "Priority:  %p%n" +
          "Thread Id: %t%n" +
          "Context:   %x%n" +
          "Message:   %m%n";

       public static void main(String[] args)
       {
          AppServerCategoryFactory factory;
          factory = new AppServerCategoryFactory("MyServer", "MyComponent", "1.0");
          AppServerCategory.setFactory( factory );
          Category cat = AppServerCategory.getInstance("some.cat");

          PatternLayout layout = new AppServerPatternLayout( formatString );
          cat.addAppender( new FileAppender( layout, System.out) );

          cat.debug("This is a debug statement.");
          cat.info("This is an info statement.");
          cat.warn("This is a warning statement.");
          cat.error("This is an error statement.");
          cat.fatal("This is a fatal statement.");
       }
    }

### Configurators

There is one a word of caution concerning the use of configurators that may create `Category` instances (such as `PropertyConfigurator` and `DOMConfigurator`). Since these configurators do not know about our extensions, any `Category` instances they create will not be `AppServerCategory` instances. To prevent this problem, any `AppServerCategory` that one might want to be configured through a configurator should be instantiated before the configure method is invoked. In this way, the configurator will configure the `AppServerCategory` that already exists rather than creating an instance of its super class.

The consequence of a configurator creating the super class by mistake is merely that the extra attributes will not appear in the log output. All other attributes are conveyed properly.

## Further Enhancements

There are some other directions in which this log4j extension may be enhanced:

1. The hostname attribute could incorportate a formatting convention similar to that of class and category names whereby only a certain number of the more significant components are displayed. But whereas with class and category names, the most significant component is on the right, with host names, it is on the left.

2. Specifying a version number could be dangerous since programmers are apt to change versions of the code without changing the string constant in the code which specifies the version. Some source control programs provide for insertion of a version number into source. For those that don't, including the version number as a constant is likely to lead to confusion later on. It would be nice to see this short-coming addressed.