Configure Logging

NiFi Ingest components produce log messages to help you troubleshoot any problems that you encounter. This section explains where to find the log files, and describes how to customize logging on your system.

There is a separate log file for each NiFi Ingest component, by default, in the following directory.

logs/idol

To customize logging, set the following properties in the file conf/nifi.properties.

# IDOL Logging Example
idol.components.log.enabled=true
idol.components.log.use.component.dir=false
idol.components.log.level=INFO
idol.components.log.max.history=5
idol.components.log.max.size=10 mb
idol.components.log.max.total.size=50 mb

NOTE: NiFi Ingest components read these properties when NiFi starts or when the component is created, so if you make changes in nifi.properties you might need to restart NiFi.

NOTE: These properties are not supported by NiFi Ingest components from versions earlier than 12.12.

Property name Default Description
idol.components.log.enabled True

A Boolean value that specifies whether to use the IDOL logging options defined in nifi.properties, and write log messages from each IDOL component to a separate log file.

If you set this property to FALSE, all of the remaining properties are ignored and the settings in conf/logback.xml are used instead.

idol.components.log.use.component.dir False

A Boolean value that specifies whether to write log files to component directories:

./idol_repository/[component_id]/logs

... rather than the default directory:

./logs/idol

Using the default location puts all log files in a single directory. Using the component directory means that log files for a component are deleted if the component is removed.

idol.components.log.level INFO

The type of messages to log. You can choose from the following values (from least to most verbose):

  • OFF
  • ERROR - Errors only.
  • WARN - Errors and warnings.
  • INFO - Errors, warnings, and additional information.
  • DEBUG - Not recommended for production systems because this amount of logging can reduce performance.
  • TRACE
  • ALL
idol.components.log.max.history 10 The maximum number of log files to keep for each component.
idol.components.log.max.size 10 mb The maximum size of an individual log file; when exceeded the file is renamed with a datestamp and file index.
idol.components.log.max.total.size 100 mb The maximum size of all log files for a single component; the oldest file is deleted if this limit is exceeded.

You can override properties for NiFi Ingest components of a certain type. In the following example, all GetFileSystem processors use the DEBUG log level, while all other components use the INFO log level.

idol.components.log.level=INFO
idol.components.GetFileSystem.log.level=DEBUG

You can also override properties for individual NiFi Ingest components in your dataflow, by specifying the component ID in the property name. For example:

idol.components.b0601e0e-94c5-443d-8f43-c6a4c7894720.log.level=WARN

Properties that include a component ID override properties that include a component class, and global properties have the lowest priority.