15.7. logging
— Logging facility for Python¶
This module defines functions and classes which implement a flexible event logging system for applications and libraries.
The key benefit of having the logging API provided by a standard library module is that all Python modules can participate in logging, so your application log can include your own messages integrated with messages from third-party modules.
The module provides a lot of functionality and flexibility. If you are unfamiliar with logging, the best way to get to grips with it is to see the tutorials (see the links on the right).
The basic classes defined by the module, together with their functions, are listed below.
- Loggers expose the interface that application code directly uses.
- Handlers send the log records (created by loggers) to the appropriate destination.
- Filters provide a finer grained facility for determining which log records to output.
- Formatters specify the layout of log records in the final output.
15.7.1. Logger Objects¶
Loggers have the following attributes and methods. Note that Loggers are never
instantiated directly, but always through the module-level function
logging.getLogger(name)
.
-
class
logging.
Logger
¶
-
Logger.
propagate
¶ If this evaluates to false, logging messages are not passed by this logger or by its child loggers to the handlers of higher level (ancestor) loggers. The constructor sets this attribute to 1.
-
Logger.
setLevel
(lvl)¶ Sets the threshold for this logger to lvl. Logging messages which are less severe than lvl will be ignored. When a logger is created, the level is set to
NOTSET
(which causes all messages to be processed when the logger is the root logger, or delegation to the parent when the logger is a non-root logger). Note that the root logger is created with levelWARNING
.The term ‘delegation to the parent’ means that if a logger has a level of NOTSET, its chain of ancestor loggers is traversed until either an ancestor with a level other than NOTSET is found, or the root is reached.
If an ancestor is found with a level other than NOTSET, then that ancestor’s level is treated as the effective level of the logger where the ancestor search began, and is used to determine how a logging event is handled.
If the root is reached, and it has a level of NOTSET, then all messages will be processed. Otherwise, the root’s level will be used as the effective level.
-
Logger.
isEnabledFor
(lvl)¶ Indicates if a message of severity lvl would be processed by this logger. This method checks first the module-level level set by
logging.disable(lvl)
and then the logger’s effective level as determined bygetEffectiveLevel()
.
-
Logger.
getEffectiveLevel
()¶ Indicates the effective level for this logger. If a value other than
NOTSET
has been set usingsetLevel()
, it is returned. Otherwise, the hierarchy is traversed towards the root until a value other thanNOTSET
is found, and that value is returned.
-
Logger.
getChild
(suffix)¶ Returns a logger which is a descendant to this logger, as determined by the suffix. Thus,
logging.getLogger('abc').getChild('def.ghi')
would return the same logger as would be returned bylogging.getLogger('abc.def.ghi')
. This is a convenience method, useful when the parent logger is named using e.g.__name__
rather than a literal string.New in version 3.2:
New in version 3.2.
-
Logger.
debug
(msg, *args, **kwargs)¶ Logs a message with level
DEBUG
on this logger. The msg is the message format string, and the args are the arguments which are merged into msg using the string formatting operator. (Note that this means that you can use keywords in the format string, together with a single dictionary argument.)There are three keyword arguments in kwargs which are inspected: exc_info which, if it does not evaluate as false, causes exception information to be added to the logging message. If an exception tuple (in the format returned by
sys.exc_info()
) is provided, it is used; otherwise,sys.exc_info()
is called to get the exception information.The second optional keyword argument is stack_info, which defaults to False. If specified as True, stack information is added to the logging message, including the actual logging call. Note that this is not the same stack information as that displayed through specifying exc_info: The former is stack frames from the bottom of the stack up to the logging call in the current thread, whereas the latter is information about stack frames which have been unwound, following an exception, while searching for exception handlers.
You can specify stack_info independently of exc_info, e.g. to just show how you got to a certain point in your code, even when no exceptions were raised. The stack frames are printed following a header line which says:
Stack (most recent call last):
This mimics the Traceback (most recent call last): which is used when displaying exception frames.
The third keyword argument is extra which can be used to pass a dictionary which is used to populate the __dict__ of the LogRecord created for the logging event with user-defined attributes. These custom attributes can then be used as you like. For example, they could be incorporated into logged messages. For example:
FORMAT = '%(asctime)-15s %(clientip)s %(user)-8s %(message)s' logging.basicConfig(format=FORMAT) d = { 'clientip' : '192.168.0.1', 'user' : 'fbloggs' } logger = logging.getLogger('tcpserver') logger.warning('Protocol problem: %s', 'connection reset', extra=d)
would print something like
2006-02-08 22:20:02,165 192.168.0.1 fbloggs Protocol problem: connection reset
The keys in the dictionary passed in extra should not clash with the keys used by the logging system. (See the
Formatter
documentation for more information on which keys are used by the logging system.)If you choose to use these attributes in logged messages, you need to exercise some care. In the above example, for instance, the
Formatter
has been set up with a format string which expects ‘clientip’ and ‘user’ in the attribute dictionary of the LogRecord. If these are missing, the message will not be logged because a string formatting exception will occur. So in this case, you always need to pass the extra dictionary with these keys.While this might be annoying, this feature is intended for use in specialized circumstances, such as multi-threaded servers where the same code executes in many contexts, and interesting conditions which arise are dependent on this context (such as remote client IP address and authenticated user name, in the above example). In such circumstances, it is likely that specialized
Formatter
s would be used with particularHandler
s.New in version 3.2:
New in version 3.2: The stack_info parameter was added.
-
Logger.
info
(msg, *args, **kwargs)¶ Logs a message with level
INFO
on this logger. The arguments are interpreted as fordebug()
.
-
Logger.
warning
(msg, *args, **kwargs)¶ Logs a message with level
WARNING
on this logger. The arguments are interpreted as fordebug()
.
-
Logger.
error
(msg, *args, **kwargs)¶ Logs a message with level
ERROR
on this logger. The arguments are interpreted as fordebug()
.
-
Logger.
critical
(msg, *args, **kwargs)¶ Logs a message with level
CRITICAL
on this logger. The arguments are interpreted as fordebug()
.
-
Logger.
log
(lvl, msg, *args, **kwargs)¶ Logs a message with integer level lvl on this logger. The other arguments are interpreted as for
debug()
.
-
Logger.
exception
(msg, *args)¶ Logs a message with level
ERROR
on this logger. The arguments are interpreted as fordebug()
. Exception info is added to the logging message. This method should only be called from an exception handler.
-
Logger.
addFilter
(filt)¶ Adds the specified filter filt to this logger.
-
Logger.
removeFilter
(filt)¶ Removes the specified filter filt from this logger.
-
Logger.
filter
(record)¶ Applies this logger’s filters to the record and returns a true value if the record is to be processed.
-
Logger.
addHandler
(hdlr)¶ Adds the specified handler hdlr to this logger.
-
Logger.
removeHandler
(hdlr)¶ Removes the specified handler hdlr from this logger.
-
Logger.
findCaller
(stack_info=False)¶ Finds the caller’s source filename and line number. Returns the filename, line number, function name and stack information as a 4-element tuple. The stack information is returned as None unless stack_info is True.
-
Logger.
handle
(record)¶ Handles a record by passing it to all handlers associated with this logger and its ancestors (until a false value of propagate is found). This method is used for unpickled records received from a socket, as well as those created locally. Logger-level filtering is applied using
filter()
.
-
Logger.
makeRecord
(name, lvl, fn, lno, msg, args, exc_info, func=None, extra=None, sinfo=None)¶ This is a factory method which can be overridden in subclasses to create specialized
LogRecord
instances.
-
Logger.
hasHandlers
()¶ Checks to see if this logger has any handlers configured. This is done by looking for handlers in this logger and its parents in the logger hierarchy. Returns True if a handler was found, else False. The method stops searching up the hierarchy whenever a logger with the ‘propagate’ attribute set to False is found - that will be the last logger which is checked for the existence of handlers.
New in version 3.2:
New in version 3.2.
15.7.2. Handler Objects¶
Handlers have the following attributes and methods. Note that Handler
is never instantiated directly; this class acts as a base for more useful
subclasses. However, the __init__()
method in subclasses needs to call
Handler.__init__()
.
-
Handler.
__init__
(level=NOTSET)¶ Initializes the
Handler
instance by setting its level, setting the list of filters to the empty list and creating a lock (usingcreateLock()
) for serializing access to an I/O mechanism.
-
Handler.
createLock
()¶ Initializes a thread lock which can be used to serialize access to underlying I/O functionality which may not be threadsafe.
-
Handler.
acquire
()¶ Acquires the thread lock created with
createLock()
.
-
Handler.
setLevel
(lvl)¶ Sets the threshold for this handler to lvl. Logging messages which are less severe than lvl will be ignored. When a handler is created, the level is set to
NOTSET
(which causes all messages to be processed).
-
Handler.
addFilter
(filt)¶ Adds the specified filter filt to this handler.
-
Handler.
removeFilter
(filt)¶ Removes the specified filter filt from this handler.
-
Handler.
filter
(record)¶ Applies this handler’s filters to the record and returns a true value if the record is to be processed.
-
Handler.
flush
()¶ Ensure all logging output has been flushed. This version does nothing and is intended to be implemented by subclasses.
-
Handler.
close
()¶ Tidy up any resources used by the handler. This version does no output but removes the handler from an internal list of handlers which is closed when
shutdown()
is called. Subclasses should ensure that this gets called from overriddenclose()
methods.
-
Handler.
handle
(record)¶ Conditionally emits the specified logging record, depending on filters which may have been added to the handler. Wraps the actual emission of the record with acquisition/release of the I/O thread lock.
-
Handler.
handleError
(record)¶ This method should be called from handlers when an exception is encountered during an
emit()
call. By default it does nothing, which means that exceptions get silently ignored. This is what is mostly wanted for a logging system - most users will not care about errors in the logging system, they are more interested in application errors. You could, however, replace this with a custom handler if you wish. The specified record is the one which was being processed when the exception occurred.
-
Handler.
format
(record)¶ Do formatting for a record - if a formatter is set, use it. Otherwise, use the default formatter for the module.
-
Handler.
emit
(record)¶ Do whatever it takes to actually log the specified logging record. This version is intended to be implemented by subclasses and so raises a
NotImplementedError
.
For a list of handlers included as standard, see logging.handlers
.
15.7.3. Formatter Objects¶
Formatter
objects have the following attributes and methods. They are
responsible for converting a LogRecord
to (usually) a string which can
be interpreted by either a human or an external system. The base
Formatter
allows a formatting string to be specified. If none is
supplied, the default value of '%(message)s'
is used.
A Formatter can be initialized with a format string which makes use of knowledge
of the LogRecord
attributes - such as the default value mentioned above
making use of the fact that the user’s message and arguments are pre-formatted
into a LogRecord
‘s message attribute. This format string contains
standard Python %-style mapping keys. See section Old String Formatting Operations
for more information on string formatting.
The useful mapping keys in a LogRecord
are given in the section on
LogRecord attributes.
-
class
logging.
Formatter
(fmt=None, datefmt=None, style='%')¶ Returns a new instance of the
Formatter
class. The instance is initialized with a format string for the message as a whole, as well as a format string for the date/time portion of a message. If no fmt is specified,'%(message)s'
is used. If no datefmt is specified, the ISO8601 date format is used.The style parameter can be one of ‘%’, ‘{‘ or ‘$’ and determines how the format string will be merged with its data: using one of %-formatting,
str.format()
orstring.Template
.Changed in version 3.2:
Changed in version 3.2: The style parameter was added.
-
format
(record)¶ The record’s attribute dictionary is used as the operand to a string formatting operation. Returns the resulting string. Before formatting the dictionary, a couple of preparatory steps are carried out. The message attribute of the record is computed using msg % args. If the formatting string contains
'(asctime)'
,formatTime()
is called to format the event time. If there is exception information, it is formatted usingformatException()
and appended to the message. Note that the formatted exception information is cached in attribute exc_text. This is useful because the exception information can be pickled and sent across the wire, but you should be careful if you have more than oneFormatter
subclass which customizes the formatting of exception information. In this case, you will have to clear the cached value after a formatter has done its formatting, so that the next formatter to handle the event doesn’t use the cached value but recalculates it afresh.If stack information is available, it’s appended after the exception information, using
formatStack()
to transform it if necessary.
-
formatTime
(record, datefmt=None)¶ This method should be called from
format()
by a formatter which wants to make use of a formatted time. This method can be overridden in formatters to provide for any specific requirement, but the basic behavior is as follows: if datefmt (a string) is specified, it is used withtime.strftime()
to format the creation time of the record. Otherwise, the ISO8601 format is used. The resulting string is returned.This function uses a user-configurable function to convert the creation time to a tuple. By default,
time.localtime()
is used; to change this for a particular formatter instance, set theconverter
attribute to a function with the same signature astime.localtime()
ortime.gmtime()
. To change it for all formatters, for example if you want all logging times to be shown in GMT, set theconverter
attribute in theFormatter
class.
-
formatException
(exc_info)¶ Formats the specified exception information (a standard exception tuple as returned by
sys.exc_info()
) as a string. This default implementation just usestraceback.print_exception()
. The resulting string is returned.
-
formatStack
(stack_info)¶ Formats the specified stack information (a string as returned by
traceback.print_stack()
, but with the last newline removed) as a string. This default implementation just returns the input value.
15.7.4. Filter Objects¶
Filters
can be used by Handlers
and Loggers
for more sophisticated
filtering than is provided by levels. The base filter class only allows events
which are below a certain point in the logger hierarchy. For example, a filter
initialized with ‘A.B’ will allow events logged by loggers ‘A.B’, ‘A.B.C’,
‘A.B.C.D’, ‘A.B.D’ etc. but not ‘A.BB’, ‘B.A.B’ etc. If initialized with the
empty string, all events are passed.
-
class
logging.
Filter
(name='')¶ Returns an instance of the
Filter
class. If name is specified, it names a logger which, together with its children, will have its events allowed through the filter. If name is the empty string, allows every event.-
filter
(record)¶ Is the specified record to be logged? Returns zero for no, nonzero for yes. If deemed appropriate, the record may be modified in-place by this method.
-
Note that filters attached to handlers are consulted whenever an event is
emitted by the handler, whereas filters attached to loggers are consulted
whenever an event is logged to the handler (using debug()
, info()
,
etc.) This means that events which have been generated by descendant loggers
will not be filtered by a logger’s filter setting, unless the filter has also
been applied to those descendant loggers.
You don’t actually need to subclass Filter
: you can pass any instance
which has a filter
method with the same semantics.
Changed in version 3.2:
Changed in version 3.2: You don’t need to create specialized Filter
classes, or use other
classes with a filter
method: you can use a function (or other
callable) as a filter. The filtering logic will check to see if the filter
object has a filter
attribute: if it does, it’s assumed to be a
Filter
and its filter()
method is called. Otherwise, it’s
assumed to be a callable and called with the record as the single
parameter. The returned value should conform to that returned by
filter()
.
Although filters are used primarily to filter records based on more sophisticated criteria than levels, they get to see every record which is processed by the handler or logger they’re attached to: this can be useful if you want to do things like counting how many records were processed by a particular logger or handler, or adding, changing or removing attributes in the LogRecord being processed. Obviously changing the LogRecord needs to be done with some care, but it does allow the injection of contextual information into logs (see Using Filters to impart contextual information).
15.7.5. LogRecord Objects¶
LogRecord
instances are created automatically by the Logger
every time something is logged, and can be created manually via
makeLogRecord()
(for example, from a pickled event received over the
wire).
-
class
logging.
LogRecord
(name, level, pathname, lineno, msg, args, exc_info, func=None, sinfo=None)¶ Contains all the information pertinent to the event being logged.
The primary information is passed in
msg
andargs
, which are combined usingmsg % args
to create themessage
field of the record.Parameters: - name – The name of the logger used to log the event represented by this LogRecord.
- level – The numeric level of the logging event (one of DEBUG, INFO etc.)
Note that this is converted to two attributes of the LogRecord:
levelno
for the numeric value andlevelname
for the corresponding level name. - pathname – The full pathname of the source file where the logging call was made.
- lineno – The line number in the source file where the logging call was made.
- msg – The event description message, possibly a format string with placeholders for variable data.
- args – Variable data to merge into the msg argument to obtain the event description.
- exc_info – An exception tuple with the current exception information, or None if no exception information is available.
- func – The name of the function or method from which the logging call was invoked.
- sinfo – A text string representing stack information from the base of the stack in the current thread, up to the logging call.
-
getMessage
()¶ Returns the message for this
LogRecord
instance after merging any user-supplied arguments with the message. If the user-supplied message argument to the logging call is not a string,str()
is called on it to convert it to a string. This allows use of user-defined classes as messages, whose__str__
method can return the actual format string to be used.
Changed in version 3.2:
Changed in version 3.2: The creation of a
LogRecord
has been made more configurable by providing a factory which is used to create the record. The factory can be set usinggetLogRecordFactory()
andsetLogRecordFactory()
(see this for the factory’s signature).
This functionality can be used to inject your own values into a LogRecord at creation time. You can use the following pattern:
old_factory = logging.getLogRecordFactory()
def record_factory(*args, **kwargs):
record = old_factory(*args, **kwargs)
record.custom_attribute = 0xdecafbad
return record
logging.setLogRecordFactory(record_factory)
With this pattern, multiple factories could be chained, and as long as they don’t overwrite each other’s attributes or unintentionally overwrite the standard attributes listed above, there should be no surprises.