Processors, Formatters & Handlers¶
Processors live in plainlog.processors:
- Processors transform a log record in the Core's background thread.
- Formatters are processors that render a record into a string, stored in
record["formatted_message"]. - Handlers are processors that perform output.
But overall they are simply processors with the same interface. Naming is just there to clarify what they do.
Every processor is a callable with the signature __call__(record: Record) -> Record and
may optionally provide a close() method for cleanup:
__call__(record: Record) -> Record— runs in the Core's background thread after dequeueing. This is where I/O happens. Return an empty dict{}to stop processing for that record.close()— cleanup resources (close files, wait for futures, etc.). Optional.
Processors are configured as a Sequence and run in order. A typical setup is a formatter followed by a handler:
from plainlog import configure_log
from plainlog.processors import SimpleFormatter, Stream
configure_log(processors=[SimpleFormatter(), Stream()])
Built-in Processors¶
Formatters¶
| Formatter | Description |
|---|---|
SimpleFormatter |
Single-line format with timestamp and level |
JsonFormatter |
Serializes a record as JSON string |
Note
With a custom fmt, any field on the record can be used as a placeholder
(for example {name}, {level_name}, or a user-supplied field). There is
no special extra field — keyword arguments are stored directly on the
record — so an unknown placeholder such as {extra} raises KeyError.
Note
JsonFormatter serializes a fixed set of keys (message, name,
created, level_name, level_no, process_id, process_name) plus any
keys named in its additional_keys argument. User-supplied fields are
not included automatically; list them in additional_keys or use a
formatter that renders the whole record.
Output Handlers¶
| Handler | Description |
|---|---|
Stream |
Writes to any file-like stream |
FileWriter |
Writes to a file, supports rotation watching |
AsyncBridge |
Base class for async integrations |
Composition Handlers¶
| Handler | Description |
|---|---|
SubProcessor |
Runs a nested processor pipeline on a record copy |
FingersCrossed |
Buffers until a threshold level triggers a flush |
WrapStandardHandler |
Bridges plainlog records to stdlib handlers |
Filters, Helpers & Utilities¶
| Processor | Description |
|---|---|
format_message |
Helper that fills record["message"] from msg and the record's fields |
print_processor_error |
Prints a record's processor error to stderr |
redact_fields(*fields, mask="***REDACTED***") |
Masks exact record keys (case-insensitive, recurses into nested dicts and lists) |
redact_by_pattern(*patterns, mask="***REDACTED***") |
Masks record keys containing a substring (case-insensitive, recurses into nested dicts and lists) |
filter_None |
Drops records whose name is None |
filter_all |
Drops every record |
filter_by_name("parent") |
Drops records whose name starts with parent |
allow_by_name("parent") |
Keeps only records whose name starts with parent |
filter_by_level(level_per_module) |
Per-module minimum levels; False blocks a module |
FilterList(blacklist, whitelist=None) |
Name-based deny/allow list processor |
WhitelistLevel(whitelist) |
Per-module level allow list |
ConsoleRenderer |
Developer-friendly colored line renderer |
All of these are available from plainlog.processors.
Note
format_message is a helper, not a required pipeline step: SimpleFormatter,
JsonFormatter and ConsoleRenderer already fill record["message"]
internally. Add it explicitly only when a handler has no formatter in front
of it, for example SubProcessor([format_message, Stream()]).
Usage¶
Formatter and Stream¶
from plainlog import configure_log
from plainlog.processors import JsonFormatter, Stream
configure_log(processors=[JsonFormatter(), Stream()])
FingersCrossed — Buffer Until Error¶
from plainlog import configure_log
from plainlog.processors import FingersCrossed, SimpleFormatter, Stream
# Buffer up to 100 records, flush everything on ERROR (level 40)
handler = FingersCrossed(
Stream(),
action_level=40,
buffer_size=100,
)
configure_log(processors=[SimpleFormatter(), handler])
AsyncBridge Subclass¶
import asyncio
from plainlog.processors import AsyncBridge
class MyAsyncBridge(AsyncBridge):
async def write(self, message):
# e.g. send to a network sink
await asyncio.sleep(0)
print(message)
Redacting Sensitive Fields¶
redact_fields matches exact key names; redact_by_pattern matches any key
containing a given substring. Both are case-insensitive and recurse through
nested dicts and lists, including dicts inside lists. Matching applies to leaf
values: when a matched key holds a container, the processor descends into it
and masks matching leaves instead of replacing the container. Place them early
in the pipeline, before any formatter, so formatted/serialized output never
contains the raw value. They scrub matching keys anywhere in the record — at
the top level and inside nested containers. Secrets interpolated directly into
the message string (e.g. log.info(f"password={pw}")) are not caught.
import sys
from plainlog import configure_log, logger
from plainlog.processors import (
ConsoleRenderer,
Stream,
redact_by_pattern,
)
configure_log(
processors=[
redact_by_pattern("password", "token", "secret", "api_key"),
ConsoleRenderer(colors=False),
Stream(sys.stdout),
]
)
log = logger.new()
log.info("login attempt", username="alice", password="hunter2")
# -> ... username=alice password=***REDACTED*** (field order may vary)
API Reference¶
Stream¶
SubProcessor¶
Processor that runs sub processors in order on a copy of the record.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
processors
|
Optional[Iterable[UniversalProcessorProtocol]]
|
List of callables run one after another. |
None
|
Example::
from plainlog.processors import Stream, SubProcessor, format_message
handler = SubProcessor([format_message, Stream()])
WrapStandardHandler¶
Wraps a stdlib logging.Handler to receive plainlog records.
Converts plainlog record dicts into logging.LogRecord instances
and forwards them to the wrapped handler.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
handler
|
Handler
|
A stdlib |
required |
FingersCrossed¶
Buffers records until a threshold level triggers a flush.
Records are buffered up to buffer_size. When a record at or above
action_level arrives, all buffered records plus the triggering
record are forwarded to the wrapped handler.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
processor
|
UniversalProcessorProtocol
|
Wrapped processor with a |
required |
action_level
|
Optional[Union[str, int]]
|
Log level number that triggers the flush. Defaults to 40 (ERROR). |
None
|
buffer_size
|
Optional[int]
|
Maximum number of records to buffer. Defaults to 1. |
None
|
reset
|
Optional[bool]
|
If |
None
|
Inspired by Monolog's FingersCrossedHandler.
FileWriter¶
Writes formatted log records to a file.
Supports delayed file creation and log rotation detection via inode watching (useful with logrotate).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
path
|
str | PathLike[str]
|
File path to write to. |
required |
delay
|
bool
|
Defer file creation until the first log record. |
False
|
watch
|
bool
|
Reopen the file if the inode changes (log rotation). |
False
|
mode
|
str
|
File open mode. Defaults to |
'a'
|
buffering
|
int
|
File buffering. Defaults to 1 (line buffered). |
1
|
encoding
|
str
|
File encoding. Defaults to |
'utf8'
|
AsyncBridge¶
Base handler for async integrations.
Schedules writes via asyncio.run_coroutine_threadsafe on the
given event loop. Subclasses must override write() to perform
the actual async I/O.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
loop
|
Optional[AbstractEventLoop]
|
The |
None
|
SimpleFormatter¶
JsonFormatter¶
ConsoleRenderer¶
Developer-friendly colored renderer for plainlog records.
Renders the record's message, level, logger name, timestamp, and remaining
key-value pairs into record["formatted_message"]. Instances are used as
ordinary processors and run in the Core thread.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
pad_event
|
int
|
Width used to pad the event/message column. Defaults to 40. |
_EVENT_WIDTH
|
colors
|
bool
|
Enable ANSI color output. Defaults to auto-detection for the current platform. |
_use_colors
|
repr_native_str
|
bool
|
Render string values with |
False
|
level_styles
|
dict | None
|
Mapping from level name to style string. When |
None
|
exception_formatter
|
Callable[..., None]
|
Callable that writes exception details to the
provided stream. Defaults to :func: |
default_exception_formatter
|
sort_keys
|
bool
|
Sort additional record keys in the output. Defaults to
|
True
|
short_level
|
bool
|
Render levels as their first character. Defaults to
|
True
|
log_name
|
bool
|
Include the logger name in the output. Defaults to |
True
|
Attributes:
| Name | Type | Description |
|---|---|---|
formatted_message |
The rendered output is stored on this record key. |
get_default_level_styles(colors=True)
staticmethod
¶
Get the default styles for log levels
FilterList¶
Name-based deny/allow list processor.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
blacklist
|
Iterable[str]
|
Iterable of dotted names to deny. |
required |
whitelist
|
Optional[Iterable[str]]
|
Optional iterable of dotted names to allow. When omitted, only the blacklist applies. |
None
|
Attributes:
| Name | Type | Description |
|---|---|---|
_whitelist |
Frozen set of whitelisted names. |
|
_blacklist |
Frozen set of blacklisted names. |
WhitelistLevel¶
Per-module minimum-level allow list.
Only records whose name or a dotted prefix appears in whitelist are kept. The configured level for a matching name is applied as a minimum level.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
whitelist
|
Dict[str, int]
|
Mapping from logger name to minimum level. |
required |
Attributes:
| Name | Type | Description |
|---|---|---|
_whitelist_names |
Frozen set of whitelisted names. |
|
_whitelist_levels |
Mapping from logger name to level. |
Helper functions¶
Return a processor that masks fields whose names exactly match.
Matching is case-insensitive and recurses through nested dictionaries and lists.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
*fields
|
Any
|
Field names to mask. |
()
|
mask
|
str
|
Replacement value for masked fields. |
'***REDACTED***'
|
Returns:
| Type | Description |
|---|---|
Callable[[Record], Record]
|
A processor callable. |
Return a processor that masks fields whose names contain a pattern.
Matching is case-insensitive and recurses through nested dictionaries and lists.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
*patterns
|
Any
|
Substrings to match against field names. |
()
|
mask
|
str
|
Replacement value for masked fields. |
'***REDACTED***'
|
Returns:
| Type | Description |
|---|---|
Callable[[Record], Record]
|
A processor callable. |
Drop records whose name is None.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
record
|
Record
|
Log record to filter. |
required |
Returns:
| Type | Description |
|---|---|
Record
|
The unchanged record, or an empty dict when dropped. |
Drop every record.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
record
|
Record
|
Log record to filter. |
required |
Returns:
| Type | Description |
|---|---|
Record
|
An empty dict. |
Fill record["message"] from msg when it is missing.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
record
|
Record
|
Log record to update. |
required |
Returns:
| Type | Description |
|---|---|
Record
|
The updated record. |
Print a processor error stored on the record to stderr.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
record
|
Record
|
Log record to inspect. |
required |
Returns:
| Type | Description |
|---|---|
Record
|
The unchanged record. |
Return a record's rendered message.
Prefers formatted_message, then message, and finally the raw
msg field.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
record
|
Record
|
Log record to inspect. |
required |
Returns:
| Type | Description |
|---|---|
str
|
The rendered message as a string. |