Benchmarks

How mcodec compares to popular Scala 3 JSON libraries on the two axes that matter for a derivation-based codec: how long it takes the compiler to derive codecs, and how fast those codecs run.

Numbers below are regenerated by benchmark/scripts/run_all.sh. They are single-machine measurements — treat them as indicative, not authoritative. The environment for the committed run is under Environment.

Takeaways

• vs. GenCodec (the design mcodec copies): ahead on writes, level on reads. write is 2.68M / 62k / 94k / 5.0k against GenCodec's 2.77M / 54k / 83k / 3.9k — parity on Primitives, ahead everywhere else. read is within noise of GenCodec on every model.
• Writes vs. the Scala 3 field: strong. mcodec beats circe, play-json and borer on every model and is level with uPickle; only jsoniter-scala and zio-json are faster. On reads it sits with circe / uPickle / GenCodec and behind zio-json and borer — the recursive-ADT read is the one model where it trails the field.
• jsoniter-scala is its own league — 2–5× everyone else on both axes.
• Compile time is the weak spot. The transparent inline derivation is the steepest-scaling in the group — ~9 s for 50 + 12 codecs (~13 s at N=100) vs. ~5–8 s for the Scala 3 macro libraries and ~4 s for GenCodec's Scala 2 macro (≈2.3× mcodec's own ancestor). It also emits ~3.5× more bytecode than circe or jsoniter. The cost is concentrated in the typer and inlining phases.

Libraries under test

Scenarios

Four models, each stressing a different part of a codec — see benchmark/src/models/Models.scala:

Serialization / deserialization

Throughput of value → String (write) and String → value (read), measured with JMH, Mode.Throughput. The suite's annotations request @Fork(3), 5×1s warmup, 10×1s measurement; the committed run used -f 3 -wi 5 -i 10 on a laptop — enough to rank the libraries, not a substitute for a full run on a dedicated box.

Every library is measured on the same A ↔ String substrate. jsoniter-scala and borer are byte-native and pay a UTF-8 conversion here that they would not pay against Array[Byte] — so their String numbers understate their native path.

Each library reads back its own output — wire shapes differ between libraries (nested-discriminator vs. "type" field, etc.) and that is expected; the benchmark measures each library's own round trip, not cross-compatibility.

GenCodec runs from a separate Scala 2.13 build (benchmark/gencodec/) — same models, payloads and JMH settings, through JsonStringOutput / JsonStringInput. Same JVM, so throughput is directly comparable.

Throughput, ops/s (higher is better) — write.

Throughput, ops/s (higher is better) — read.

Charts show throughput relative to mcodec on each model (mcodec = 1.0, taller is faster); absolute ops/s are in the tables above.

Compilation time

For each library, a clean compile (scala-cli compile --server=false, no Bloop, no incremental compiler) of a generated project containing N derived codecs: N independent case classes of 8–14 fields plus N/4 sealed hierarchies of 6 cases each. run_all.sh full sweeps N ∈ {0, 1, 10, 25, 50, 100}; the committed run goes to 100, 3 runs each, mean reported. (Nesting depth is a runtime concern, covered by the Company model above; this sweep isolates how compile time scales with the number of derived codecs.)

The N=0 row is the fixed cost (JVM + scalac startup + dependency classpath); the slope from N=1 upward is the derivation cost. Generator and driver: benchmark/compile/.

GenCodec is included here as a cross-ecosystem reference — it is compiled by Scala 2.13, so it is not a like-for-like measurement against the Scala 3 compiler, but it does answer "is mcodec's Scala 3 inline derivation cheaper or dearer than the Scala 2 macro it replaces?". Scala 2.13 has no built-in -Yprofile-trace, so GenCodec has no phase-breakdown row.

Clean compile wall time, seconds (lower is better); last column is total emitted .class bytes. N = derived codecs.

¹ GenCodec is compiled by Scala 2.13.18, not the Scala 3 compiler — a cross-ecosystem data point, not a like-for-like measurement.

Compiler phase breakdown

Seconds per scalac phase for the N=50 project. Macro / mirror derivation and inline expansion land in typer and inlining — their sum is the derivation cost.

Caveats

• Single machine, single JDK. Absolute numbers do not transfer between machines; the ratios between libraries are the portable part.
• String substrate. mcodec's public API (Json.read / Json.write) is String-only, so the whole matrix uses String. This understates the byte-native libraries (jsoniter-scala, borer), which are built for Array[Byte] and pay a UTF-8 conversion here.
• Scala 3.9.0 (Scala Next). mcodec pins 3.9.0 via Made 0.4.1. Every competitor is published for 3.3 LTS and consumed here via TASTy forward-compatibility; a native 3.9.0 build of each might differ slightly.
• Default configuration. Each library uses its out-of-the-box derivation with no hand-tuning (no jsoniter CodecMakerConfig beyond recursion, no circe Configuration, etc.). A tuned setup can move the numbers.
• play-json covers only the three product models — its Scala 3 Json.format macro does not derive sealed hierarchies.
• GenCodec is Scala 2.13 — a different compiler and standard library. Same-JVM runtime throughput is a fair comparison; its compile-time row is a cross-ecosystem reference (Scala 2 macro vs. Scala 3 inline), flagged as such and not stacked in the phase chart.
• JIT warm-up is handled by JMH forking + warmup iterations; the compile sweep deliberately does the opposite (cold JVM every run).

Environment

Committed run:

date:      2026-09-02
cpu:       Apple M4 Pro
ram:       48 GB
os:        macOS 26.6.1 (25G76)
jdk:       OpenJDK 26.0.1 (Homebrew)
scala:     3.9.0
scala-cli: 1.15.0
jmh:       1.37   (-f 3 -wi 5 -i 10)

A laptop under normal desktop load — fine for ranking, not for absolute numbers. Re-run benchmark/scripts/run_all.sh full on a quiet dedicated machine for a citable dataset.

Reproducing

# everything (~1h): JMH matrix + compile sweep + report + charts
benchmark/scripts/run_all.sh full

# quick sanity pass: 1 fork, fewer sizes (a few minutes)
benchmark/scripts/run_all.sh quick

# just one slice
scala-cli --power run benchmark --exclude gencodec --jmh -- 'CompanyBench.*' -p lib=mcodec,circe
scala-cli --power run benchmark/gencodec --jmh -- 'CompanyBench.*'   # GenCodec (Scala 2.13)
python3 benchmark/compile/bench_compile.py --libs mcodec,circe --sizes 0,25,50