Hierarchical, namespaced logging
Safe and straightforward to use in multi-threaded applications.
Native support for structured logging (old-style string logging is also supported).

Getting Started

Install the package from nimble:

nimble install namespaced_logging

Then, in your application, you can use the logging system like so:

import namespaced_logging

# On the main thread
let logService = initLogService()
logService.addAppender(initConsoleAppender(LogLevel.INFO))

# On any thread, including the main thread
let logger = logService.getLogger("app/service/example")
logger.info("Log from the example service")

# Only get logs at the WARN or higher level from the database module
let logger = logService.getLogger("app/database", threshold = some(Level.lvlWarn))
logger.error("Database connection failed")

Loggers and Appenders

The logging system is composed of two main components: loggers and appenders. Loggers are used to create log events, which are then passed to the appenders. Appenders take log events and write them to some destination, such as the console, a file, or a network socket. Appenders also have a logging level threshold, which determines which log events are acted upon by the appender, and, optionally, a namespace filter, which determines from which loggers the appender accepts log events.

Heirarchical Logging and Namespaces

Loggers are organized hierarchically, with the hierarchy defined by the logger name. A logger with the name app/service/example is a child of the logger with the name app/service. By default, appenders accept log events from all loggers, but this can be restricted by setting a namespace filter on the appender. An appender with a namespace set will accept log events from all loggers with names that start with the namespace. For example, an appender with the namespace app will accept log events from the loggers app, app/service, and app/service/example, but not from api/service.

The other impact of the logger heirarchy is in the effective logging level of the logger. Any logger can have an explicit logging level set, but if it does not, the effective logging level is inherited from ancestor loggers upwards in the logger heirarchy. This pattern is explained in detail in the logback documentation and applies in the same manner to loggers in this library.

Notes on Use in Multi-Threaded Applications

The loggers and appenders in this library are thread-safe and behaves more intuitively in a multi-threaded environment than std/logging, particularly in environments where the logging setup code may be separated from the thread-management code (in an HTTP server, for example).

The LogService object is the main entry point for the logging system and should be initialized on the main thread. The LogService contains the "source of truth" for logging configuration and is shared between all threads. Internally all access to the LogService is protected by a mutex.

Logging can be very noisy and if the LogService needed to be consulted for every log event, it could easily become a performance bottleneck. To avoid this, the getLogger procedure makes a thread-local copy of the logging system configuration (loggers defined and appenders attached).

Note that this means that the thread-local cache of the logging system configuration can become stale if the logging system configuration is changed after the thread-local copy is made (if another appender is added, for example). This is a trade-off to avoid the performance penalty of consulting the LogService for every log event.

This thread-local caching mechanism is the primary advantage of this logging system over std/logging in a multi-threaded environment as it means that the logging system itself is responsible for making sure appenders are configured for every thread where loggers are used, even if the thread initialization context is separated from the logging setup code.

If you find yourself needing to change the logging configuration after the logging system has been initialized, the reloadThreadState procedure can be used to update the thread-local cache of the logging system configuration, but it must be called on the thread you wish to update.

As a final note, the advice to initialize the LogService on the main thread is primarily to simplify the configuration of the logging service and avoid the need to manually reload caches on individual threads. A LogService reference is required to call getLogger, but it can be created on any thread.

Custom Appender Implementations

Due to the thread-safety of the logging system, there are a few additional considerations when implementing custom appenders. The LogAppender abstract class is the base class for all appenders. To implement a custom appender, two methods must be implemented:

`appendLogMessage`

method appendLogMessage*(appender: CustomLogAppender, msg: LogMessage): void {.base, gcsafe.}

This is the primary appender implementation that takes a LogMessage and writes it to the appender's destination. As the signature suggests, the implementation must be GC-safe. As a multi-method, the CustomLogAppender type should be replaced by the actual name of your custom appender.

Because the LogAppender uses multi-methods for dynamic dispatch, the custom appender class must also be a ref type.

`initThreadCopy`

method initThreadCopy*(app: LogAppender): LogAppender {.base, gcsafe.}

This method is used to create a thread-local copy of the appender. It is called by the reloadThreadState procedure to update the thread-local cache of the logging system configuration. The implementation will be passed the appender instance that was provided to the addAppender procedure and must return a thread-local copy of that appender.

The initThreadCopy implementations for the built-in ConsoleLogAppender and FileLogAppender provide simple examples of how to implement this method by simply copying state into the local thread, but this method can also be used to perform any other thread-specific initialization that may be required for the appender implementation.

Example Custom Appender

The following defines a simple custom appender that writes log messages to a database table. It uses the [waterpark][] connection pooling library to manage database connections as waterpark is also thread-safe and makes implementation straight-forward.

import db_connectors/db_postgres
import namespaced_logging, waterpark, waterpark/db_postgres

type DbLogAppender = ref object of LogAppender
  dbPool: PostgresPool

let dbPool: PostgresPool = newPostgresPool(10, "", "", "", connectionString)

method initThreadCopy*(app: LogAppender): LogAppender =
  result = DbLogAppender(dbPool: dbPool) # copy semantics as PostgresPool is an object

method appendLogMessage*(appender: DbLogAppender, msg: LogMessage): void {gcsafe.} =
  appender.withConnection conn:
    conn.insert(
      "INSERT INTO log_events " &
      "  (level, scope, message, error, timestamp, custom_fields) " &
      "VALUES " &
      "  (?, ?, ?, ?, ?, ?)",
      msg.level,
      msg.scope,
      msg.message,
      if msg.error.isSome: msg.error.msg
      else: "",
      msg.timestamp,
      msg.additionalData)