Skip to content

Processors, Formatters & Handlers

Processors live in plainlog.processors:

  • Processors transform a log record in the Core's background thread.
  • Formatters are processors that render a record into a string, stored in record["formatted_message"].
  • Handlers are processors that perform output.

But overall they are simply processors with the same interface. Naming is just there to clarify what they do.

Every processor is a callable with the signature __call__(record: Record) -> Record and may optionally provide a close() method for cleanup:

  • __call__(record: Record) -> Record — runs in the Core's background thread after dequeueing. This is where I/O happens. Return an empty dict {} to stop processing for that record.
  • close() — cleanup resources (close files, wait for futures, etc.). Optional.

Processors are configured as a Sequence and run in order. A typical setup is a formatter followed by a handler:

from plainlog import configure_log
from plainlog.processors import SimpleFormatter, Stream

configure_log(processors=[SimpleFormatter(), Stream()])

Built-in Processors

Formatters

Formatter Description
SimpleFormatter Single-line format with timestamp and level
JsonFormatter Serializes a record as JSON string

Note

With a custom fmt, any field on the record can be used as a placeholder (for example {name}, {level_name}, or a user-supplied field). There is no special extra field — keyword arguments are stored directly on the record — so an unknown placeholder such as {extra} raises KeyError.

Note

JsonFormatter serializes a fixed set of keys (message, name, created, level_name, level_no, process_id, process_name) plus any keys named in its additional_keys argument. User-supplied fields are not included automatically; list them in additional_keys or use a formatter that renders the whole record.

Output Handlers

Handler Description
Stream Writes to any file-like stream
FileWriter Writes to a file, supports rotation watching
AsyncBridge Base class for async integrations

Composition Handlers

Handler Description
SubProcessor Runs a nested processor pipeline on a record copy
FingersCrossed Buffers until a threshold level triggers a flush
WrapStandardHandler Bridges plainlog records to stdlib handlers

Filters, Helpers & Utilities

Processor Description
format_message Helper that fills record["message"] from msg and the record's fields
print_processor_error Prints a record's processor error to stderr
redact_fields(*fields, mask="***REDACTED***") Masks exact record keys (case-insensitive, recurses into nested dicts and lists)
redact_by_pattern(*patterns, mask="***REDACTED***") Masks record keys containing a substring (case-insensitive, recurses into nested dicts and lists)
filter_None Drops records whose name is None
filter_all Drops every record
filter_by_name("parent") Drops records whose name starts with parent
allow_by_name("parent") Keeps only records whose name starts with parent
filter_by_level(level_per_module) Per-module minimum levels; False blocks a module
FilterList(blacklist, whitelist=None) Name-based deny/allow list processor
WhitelistLevel(whitelist) Per-module level allow list
ConsoleRenderer Developer-friendly colored line renderer

All of these are available from plainlog.processors.

Note

format_message is a helper, not a required pipeline step: SimpleFormatter, JsonFormatter and ConsoleRenderer already fill record["message"] internally. Add it explicitly only when a handler has no formatter in front of it, for example SubProcessor([format_message, Stream()]).

Usage

Formatter and Stream

from plainlog import configure_log
from plainlog.processors import JsonFormatter, Stream

configure_log(processors=[JsonFormatter(), Stream()])

FingersCrossed — Buffer Until Error

from plainlog import configure_log
from plainlog.processors import FingersCrossed, SimpleFormatter, Stream

# Buffer up to 100 records, flush everything on ERROR (level 40)
handler = FingersCrossed(
    Stream(),
    action_level=40,
    buffer_size=100,
)
configure_log(processors=[SimpleFormatter(), handler])

AsyncBridge Subclass

import asyncio
from plainlog.processors import AsyncBridge

class MyAsyncBridge(AsyncBridge):
    async def write(self, message):
        # e.g. send to a network sink
        await asyncio.sleep(0)
        print(message)

Redacting Sensitive Fields

redact_fields matches exact key names; redact_by_pattern matches any key containing a given substring. Both are case-insensitive and recurse through nested dicts and lists, including dicts inside lists. Matching applies to leaf values: when a matched key holds a container, the processor descends into it and masks matching leaves instead of replacing the container. Place them early in the pipeline, before any formatter, so formatted/serialized output never contains the raw value. They scrub matching keys anywhere in the record — at the top level and inside nested containers. Secrets interpolated directly into the message string (e.g. log.info(f"password={pw}")) are not caught.

import sys
from plainlog import configure_log, logger
from plainlog.processors import (
    ConsoleRenderer,
    Stream,
    redact_by_pattern,
)

configure_log(
    processors=[
        redact_by_pattern("password", "token", "secret", "api_key"),
        ConsoleRenderer(colors=False),
        Stream(sys.stdout),
    ]
)

log = logger.new()
log.info("login attempt", username="alice", password="hunter2")
# -> ... username=alice password=***REDACTED***  (field order may vary)

API Reference

Stream

Writes a record's formatted formatted_message to a file-like stream.

Parameters:

Name Type Description Default
stream Optional[IO[Any]]

A file-like object with a write method. Defaults to sys.stderr.

None

SubProcessor

Processor that runs sub processors in order on a copy of the record.

Parameters:

Name Type Description Default
processors Optional[Iterable[UniversalProcessorProtocol]]

List of callables run one after another.

None

Example::

from plainlog.processors import Stream, SubProcessor, format_message

handler = SubProcessor([format_message, Stream()])

WrapStandardHandler

Wraps a stdlib logging.Handler to receive plainlog records.

Converts plainlog record dicts into logging.LogRecord instances and forwards them to the wrapped handler.

Parameters:

Name Type Description Default
handler Handler

A stdlib logging.Handler instance.

required

FingersCrossed

Buffers records until a threshold level triggers a flush.

Records are buffered up to buffer_size. When a record at or above action_level arrives, all buffered records plus the triggering record are forwarded to the wrapped handler.

Parameters:

Name Type Description Default
processor UniversalProcessorProtocol

Wrapped processor with a close() method to flush to.

required
action_level Optional[Union[str, int]]

Log level number that triggers the flush. Defaults to 40 (ERROR).

None
buffer_size Optional[int]

Maximum number of records to buffer. Defaults to 1.

None
reset Optional[bool]

If True, the handler resets after each flush and buffers again. Defaults to False.

None

Inspired by Monolog's FingersCrossedHandler.

FileWriter

Writes formatted log records to a file.

Supports delayed file creation and log rotation detection via inode watching (useful with logrotate).

Parameters:

Name Type Description Default
path str | PathLike[str]

File path to write to.

required
delay bool

Defer file creation until the first log record.

False
watch bool

Reopen the file if the inode changes (log rotation).

False
mode str

File open mode. Defaults to "a".

'a'
buffering int

File buffering. Defaults to 1 (line buffered).

1
encoding str

File encoding. Defaults to "utf8".

'utf8'

AsyncBridge

Base handler for async integrations.

Schedules writes via asyncio.run_coroutine_threadsafe on the given event loop. Subclasses must override write() to perform the actual async I/O.

Parameters:

Name Type Description Default
loop Optional[AbstractEventLoop]

The asyncio.AbstractEventLoop to schedule writes on. If None (default) and a loop is currently running in the building thread, that loop is captured; otherwise the handler is a no-op until a loop is supplied. Construction never raises.

None

SimpleFormatter

JsonFormatter

ConsoleRenderer

Developer-friendly colored renderer for plainlog records.

Renders the record's message, level, logger name, timestamp, and remaining key-value pairs into record["formatted_message"]. Instances are used as ordinary processors and run in the Core thread.

Parameters:

Name Type Description Default
pad_event int

Width used to pad the event/message column. Defaults to 40.

_EVENT_WIDTH
colors bool

Enable ANSI color output. Defaults to auto-detection for the current platform.

_use_colors
repr_native_str bool

Render string values with repr() when True. Defaults to False.

False
level_styles dict | None

Mapping from level name to style string. When None, the built-in level styles are used.

None
exception_formatter Callable[..., None]

Callable that writes exception details to the provided stream. Defaults to :func:default_exception_formatter.

default_exception_formatter
sort_keys bool

Sort additional record keys in the output. Defaults to True.

True
short_level bool

Render levels as their first character. Defaults to True.

True
log_name bool

Include the logger name in the output. Defaults to True.

True

Attributes:

Name Type Description
formatted_message

The rendered output is stored on this record key.

get_default_level_styles(colors=True) staticmethod

Get the default styles for log levels

FilterList

Name-based deny/allow list processor.

Parameters:

Name Type Description Default
blacklist Iterable[str]

Iterable of dotted names to deny.

required
whitelist Optional[Iterable[str]]

Optional iterable of dotted names to allow. When omitted, only the blacklist applies.

None

Attributes:

Name Type Description
_whitelist

Frozen set of whitelisted names.

_blacklist

Frozen set of blacklisted names.

WhitelistLevel

Per-module minimum-level allow list.

Only records whose name or a dotted prefix appears in whitelist are kept. The configured level for a matching name is applied as a minimum level.

Parameters:

Name Type Description Default
whitelist Dict[str, int]

Mapping from logger name to minimum level.

required

Attributes:

Name Type Description
_whitelist_names

Frozen set of whitelisted names.

_whitelist_levels

Mapping from logger name to level.

Helper functions

Return a processor that masks fields whose names exactly match.

Matching is case-insensitive and recurses through nested dictionaries and lists.

Parameters:

Name Type Description Default
*fields Any

Field names to mask.

()
mask str

Replacement value for masked fields.

'***REDACTED***'

Returns:

Type Description
Callable[[Record], Record]

A processor callable.

Return a processor that masks fields whose names contain a pattern.

Matching is case-insensitive and recurses through nested dictionaries and lists.

Parameters:

Name Type Description Default
*patterns Any

Substrings to match against field names.

()
mask str

Replacement value for masked fields.

'***REDACTED***'

Returns:

Type Description
Callable[[Record], Record]

A processor callable.

Drop records whose name is None.

Parameters:

Name Type Description Default
record Record

Log record to filter.

required

Returns:

Type Description
Record

The unchanged record, or an empty dict when dropped.

Drop every record.

Parameters:

Name Type Description Default
record Record

Log record to filter.

required

Returns:

Type Description
Record

An empty dict.

Return a processor that drops records whose name starts with parent.

Parameters:

Name Type Description Default
parent str

Prefix to match against the record's name.

required

Returns:

Type Description
Callable[[Record], Record]

A processor callable.

Return a processor that keeps only records whose name starts with parent.

Parameters:

Name Type Description Default
parent str

Prefix to match against the record's name.

required

Returns:

Type Description
Callable[[Record], Record]

A processor callable.

Return a per-module minimum-level filter.

Parameters:

Name Type Description Default
level_per_module Dict[str, Any]

Mapping from module name to minimum level. Use False to block a module entirely.

required

Returns:

Type Description
Callable[[Record], Record]

A processor callable.

Fill record["message"] from msg when it is missing.

Parameters:

Name Type Description Default
record Record

Log record to update.

required

Returns:

Type Description
Record

The updated record.

Print a processor error stored on the record to stderr.

Parameters:

Name Type Description Default
record Record

Log record to inspect.

required

Returns:

Type Description
Record

The unchanged record.

Return a record's rendered message.

Prefers formatted_message, then message, and finally the raw msg field.

Parameters:

Name Type Description Default
record Record

Log record to inspect.

required

Returns:

Type Description
str

The rendered message as a string.