Benchmarks

Note

Benchmarks are hard.

Repeatedly calling the same function in a tight loop will lead to the instruction cache staying hot and branches being highly predictable. That’s not representative of real world access patterns. It’s also hard to write a nonbiased benchmark. I wrote structtype, naturally whatever benchmark I publish it’s going to perform well in.

Even so, people like to see benchmarks. I’ve tried to be as nonbiased as I can be, and the results hopefully indicate a few tradeoffs you make when you choose different serialization formats. I encourage you to write your own benchmarks before making these decisions.

In all cases benchmarks were run on my local development machine (~2020 x86 Linux desktop) using CPython 3.14.

Library Comparison

This benchmark compares structtype against msgspec and pydantic across two common data shapes:

E-commerce data — flat structs with lists and optional fields (Item / Order):

class Item(Struct):
    name: str
    price: float
    tags: list[str] = []
    metadata: dict[str, str] | None = None

class Order(Struct, kw_only=True):
    id: int
    customer: str
    items: list[Item]
    created_at: str
    status: str = "pending"

A set of 500 randomized orders is used to measure:

  • struct → dict serialization

  • dict → struct validation

  • struct → JSON encoding

  • JSON → struct decoding

Tagged union data — recursively nested File / Dir tree:

class File(Struct, tag="file", kw_only=True):
    name: str
    size: int

class Dir(Struct, tag="dir", kw_only=True):
    name: str
    contents: list[File | Dir]

A single deeply nested tree (depth 4, branching up to 5) is used to measure JSON encode/decode performance specifically.

The full benchmark source can be found here.

Python 3.14, structtype 0.1.1, msgspec 0.21.1, pydantic 2.13.4
Dump (struct → dict)
-------------------------------------------------------
  structtype           1257.3 μs   (1.00x)
  msgspec              1302.6 μs   (1.04x)
  pydantic             4247.2 μs   (3.38x)

Load (dict → struct)
-------------------------------------------------------
  structtype           1237.3 μs   (1.00x)
  msgspec              1333.2 μs   (1.08x)
  pydantic             6483.4 μs   (5.24x)

Dump JSON (struct → bytes)
-------------------------------------------------------
  structtype           1025.1 μs   (1.00x)
  msgspec              1039.8 μs   (1.01x)
  pydantic             4136.8 μs   (4.04x)

Load JSON (bytes → struct)
-------------------------------------------------------
  structtype           2330.7 μs   (1.00x)
  msgspec              2445.7 μs   (1.05x)
  pydantic             8064.7 μs   (3.46x)

Dump JSON (tagged union)
-------------------------------------------------------
  structtype              1.6 μs   (1.05x)
  msgspec                 1.6 μs   (1.00x)
  pydantic               21.1 μs   (13.61x)

Load JSON (tagged union)
-------------------------------------------------------
  structtype              4.3 μs   (1.02x)
  msgspec                 4.2 μs   (1.00x)
  pydantic               22.4 μs   (5.32x)

For flat data, structtype and msgspec perform within ~5% of each other across all operations, while pydantic is 3–5x slower. The tagged union benchmark tells the same story: both libraries are essentially tied, with pydantic 5–14x behind across all measurement types.

Structs

Here we benchmark common structtype.Struct operations, comparing their performance against other similar libraries. The cases compared are:

For each library, the following operations are benchmarked:

  • Time to define a new class. Many libraries that abstract away class boilerplate add overhead when defining classes, slowing import times for libraries that make use of these classes.

  • Time to create an instance of that class.

  • Time to compare two instances for equality (== / !=).

  • Time to compare two instances for order (< / > / <= / >=)

The full benchmark source can be found here.

  • Standard Python classes are the fastest to import (any library can only add overhead here). Still, structtype isn’t that much slower, especially compared to other options.

  • Structs are optimized to be cheap to create, and that shows for the creation benchmark. They’re roughly 4x faster than standard classes/attrs/dataclasses, and 17x faster than pydantic.

  • For equality comparison, structtype Structs are roughly 4x to 30x faster than the alternatives.

  • For order comparison, structtype Structs are roughly 5x to 60x faster than the alternatives.

Garbage Collection

structtype.Struct instances implement several optimizations for reducing garbage collection (GC) pressure and decreasing memory usage. Here we benchmark structs (with and without gc=False) against standard Python classes (with and without __slots__).

For each option we create a large dictionary containing many simple instances of the benchmarked type, then measure:

  • The amount of time it takes to do a full garbage collection (gc) pass

  • The total amount of memory used by this data structure

The full benchmark source can be found here.

  • Standard Python classes are the most memory hungry (since all data is stored in an instance dict). They also result in the largest GC pause, as the GC has to traverse the entire outer dict, each class instance, and each instance dict. All that pointer chasing has a cost.

  • Standard classes with __slots__ are less memory hungry, but still results in an equivalent GC pauses.

  • structtype.Struct instances have the same memory layout as a class with __slots__ (and thus have the same memory usage), but due to deferred GC tracking a full GC pass completes in a fraction of the time.

  • structtype.Struct instances with gc=False have the lowest memory usage (lack of GC reduces memory by 16 bytes per instance). They also have the lowest GC pause (75x faster than standard classes!) since the entire composing dict can be skipped during GC traversal.