Logging

Changed in version 0.4.0: Caproto’s use of Python logging framework has been completely reworked to follow Python’s documented best practices for libraries.

Caproto uses Python’s logging framework, which enables sophisticated log management. Users who are familiar with that framework or who need to route logs to multiple destinations may wish to skip ahead to Caproto’s Logging-Related API. But for common simple cases, including viewing logs in the terminal or writing them to a file, the next section illustrates streamlined, copy/paste-able examples.

Command-line tools

Caproto’s commandline tools, including both the clients (e.g. caproto-get) and the server modules (python3 -m caproto.ioc_examples.*), accept -v and -vvv to control log verbosity.

Basic Examples

In scripts or interactive sessions, the convenience function config_caproto_logging() can address common use cases succinctly. An equivalent configuration can be achieved using the standard Python logging interface.

Log warnings

This is the recommended standard setup for interactive use.

from caproto import config_caproto_logging
config_caproto_logging()

It will display log records of WARNING level or higher in the terminal (standard out) with a formatting tailored to caproto.

Maximum verbosity

This will display all Channel Access commands sent or received, comprising a complete account of the Channel Access traffic handled by caproto.

from caproto import config_caproto_logging
config_caproto_logging(level='DEBUG')

Important

We strongly recommend setting levels on handlers not on loggers. In previous versions of caproto, we recommended adjusting the level on the logger, as in some_pv.log.setLevel('DEBUG'). We now recommended that you avoid setting levels on loggers because it would affect all handlers downstream, potentially inhibiting some other part of the program from collecting the records it wants to collect.

Log to a file

This will direct all log messages to a file instead of the terminal (standard out).

from caproto import config_caproto_logging
config_caproto_logging(file='/tmp/caproto.log', level='DEBUG')

Filter by PV, Address, or Role

To debug a particular issue, it is often convenient to focus on log records related to a specific PV, address, or role (CLIENT or SERVER). To add filters, you will need a reference to a handler. Typically, you will want to set the handler to 'DEBUG' or 'INFO'. Capture its return value in a variable.

handler = config_caproto_logging(level='INFO')

You can later access the current caproto global handler at any point by using get_handler().

from caproto import get_handler
handler = get_handler()

Several easy-to-use filters allow users to very specifically customize logging, based on one or more of the following:

  1. Show only records related to specific PVs—or partial PVs with a * wildcard.

    from caproto import PVFilter
    
    handler.addFilter(PVFilter('random_walk:*', 'simple:B'))
    
  2. Show only records related to specific hosts or addresses.

    from caproto import AddressFilter
    
    # Multiple ways to specify:
    handler.addFilter(AddressFilter('10.2.227.105'))  # host (all ports)
    handler.addFilter(AddressFilter('10.2.227.105:59384'))  # host:port
    handler.addFilter(AddressFilter(('10.2.227.105', 59384))  # equivalent
    
  3. In special situations (usually testing) one process may be acting as client and server, and it may be useful to filter by role ('CLIENT' or 'SERVER').

    from caproto import RoleFilter
    
    handler.addFilter(RoleFilter('CLIENT'))
    

Note that if multiple filters are added to the same handler, they are composed using “logical AND” by Python’s logging framework. See the section on Filters below for more information or composing filters or writing complex filters.

Important

In the examples above we add filters to handlers, as in handler.addFilter(...). The Python logging framework also allows filters to be added to loggers, as in some_pv.log.addFilter(...). But we recommend that you avoid adding filters to loggers because it would affect all handlers downstream, potentially inhibiting some other part of the program from collecting the records it wants to collect.

Advanced Example

The flow of log event information in loggers and handlers is illustrated in the following diagram:

https://docs.python.org/3/_images/logging_flow.png

For further reference, see the Python 3 logging howto: https://docs.python.org/3/howto/logging.html#logging-flow

As an illustrative example, we will set up two handlers using the Python logging framework directly, ignoring caproto’s convenience function.

Suppose we set up a handler aimed at a file:

import logging
file_handler = logging.FileHandler('caproto.log')

And another aimed at Logstash:

import logstash  # requires python-logstash package
logstash_handler = logstash.TCPLogstashHandler(<host>, <port>, version=1)

We can attach the handlers to the caproto logger, to which all log records created by caproto propagate:

logger = logging.getLogger('caproto')
logger.addHandler(logstash_handler)
logger.addHandler(file_filter)

We can set the verbosity of each handler. Suppose want maximum verbosity in the file but only medium verbosity in logstash.

logstash_handler.setLevel('INFO')
file_handler.setLevel('DEBUG')

And suppose that we only want the file to receive records related to PV that being with ‘xyz’. We can add a filter.

file_handler.addFilter(PVFilter('xyz*'))

The filter does not interfere with the logstash_handler in any way. Finally, ensure that “effective level” of logger is at least as verbose as the most verbose handler—in this case, 'DEBUG'. By default, at import, its level is not set

In [1]: logging.getLevelName(logger.level)

In [2]: 'NOTSET'

and so it inherits the level of Python’s default “handler of last resort,” logging.lastResort, which is 'WARNING'.

In [3]: logging.getLevelName(logger.getEffectiveLevel())

In [4]: 'WARNING'

In this case we should set it to 'DEBUG', to match the most verbose level of the handler we have added.

logger.setLevel('DEBUG')

This makes DEBUG-level records available to all handlers. Our logstash handler, set to 'INFO', will filter out DEBUG-level records.

To globally disable the generation of any log records at or below a certain verbosity, which may be helpful for optimising performance, Python provides logging.disable().