Benchmarks

How mcodec compares to popular Scala 3 JSON libraries on the two axes that matter for a derivation-based codec: how long it takes the compiler to derive codecs, and how fast those codecs run.

Numbers below are regenerated by ./mill benchmark.runAll. They are single-machine measurements — treat them as indicative, not authoritative. The environment for the committed run is under Environment.

Takeaways

• vs. GenCodec (the design mcodec copies): ahead on writes, close on reads. write is 3.4M / 67k / 93k / 4.9k against GenCodec's 3.3M / 54k / 83k / 4.2k — ahead on every model. read sits close to GenCodec, trading places model by model.
• Writes vs. the Scala 3 field: strong. mcodec beats circe, play-json, borer, uPickle and GenCodec on every model; only jsoniter-scala and zio-json are faster, and on the nested Company model it's within 3% of jsoniter-scala. On reads it sits with circe / uPickle / GenCodec and behind zio-json / borer / jsoniter-scala — the recursive-ADT read is the one model where it trails the field.
• Bytecode is no longer the outlier — it's now one of the leaner libraries. Collapsing made's per-field match-type indirection into non-inline helpers brought emitted bytecode at N=100 down to 2,900 KB — less than borer, GenCodec, uPickle and zio-json, behind only play-json / circe / jsoniter-scala. (This benchmark measured 9,105 KB before the made 0.6.0 rework, and briefly regressed to 6,330 KB when a well-intentioned .toArray.map(...) → .toArrayOf compile-time tweak turned out to duplicate its loop body into every derivation call site — a reminder that bytecode has to be measured, not assumed, once inline is involved.)
• Compile time has closed most of the gap. mcodec is still the slowest-scaling Scala 3 library in the group — ~6.8 s for 50 + 12 codecs (~9.6 s at N=100) vs. ~8.4–8.5 s for circe / borer / zio-json and ~4.9 s for GenCodec's Scala 2 macro — and it's now faster than uPickle (10.0 s) at N=100, where it used to be the clear outlier. The cost is concentrated in the typer and inlining phases.

Libraries under test

Scenarios

Four models, each stressing a different part of a codec — see benchmark/src/models/Models.scala:

Serialization / deserialization

Throughput of value → String (write) and String → value (read), measured with JMH, Mode.Throughput. The suite's annotations request @Fork(3), 5×1s warmup, 10×1s measurement; the committed run used -f 3 -wi 5 -i 10 on a laptop — enough to rank the libraries, not a substitute for a full run on a dedicated box.

Every library is measured on the same A ↔ String substrate. jsoniter-scala and borer are byte-native and pay a UTF-8 conversion here that they would not pay against Array[Byte] — so their String numbers understate their native path.

Each library reads back its own output — wire shapes differ between libraries (nested-discriminator vs. "type" field, etc.) and that is expected; the benchmark measures each library's own round trip, not cross-compatibility.

GenCodec runs from a separate Scala 2.13 build (benchmark/gencodec/) — same models, payloads and JMH settings, through JsonStringOutput / JsonStringInput. Same JVM, so throughput is directly comparable.

Throughput, ops/s (higher is better) — write.

Throughput, ops/s (higher is better) — read.

Charts show throughput relative to mcodec on each model (mcodec = 1.0, taller is faster); absolute ops/s are in the tables above.

Compilation time

For each library, a clean compile (scala-cli compile --server=false, no Bloop, no incremental compiler) of a generated project containing N derived codecs: N independent case classes of 8–14 fields plus N/4 sealed hierarchies of 6 cases each. ./mill benchmark.runAll sweeps N ∈ {0, 1, 10, 25, 50, 100}; the committed run goes to 100, 3 runs each, mean reported. (Nesting depth is a runtime concern, covered by the Company model above; this sweep isolates how compile time scales with the number of derived codecs.)

The N=0 row is the fixed cost (JVM + scalac startup + dependency classpath); the slope from N=1 upward is the derivation cost. Generator and driver: benchmark/compile/.

GenCodec is included here as a cross-ecosystem reference — it is compiled by Scala 2.13, so it is not a like-for-like measurement against the Scala 3 compiler, but it does answer "is mcodec's Scala 3 inline derivation cheaper or dearer than the Scala 2 macro it replaces?". Scala 2.13 has no built-in -Yprofile-trace, so GenCodec has no phase-breakdown row.

Clean compile wall time, seconds (lower is better); last column is total emitted .class bytes. N = derived codecs.

¹ GenCodec is compiled by Scala 2.13.18, not the Scala 3 compiler — a cross-ecosystem data point, not a like-for-like measurement.

Compiler phase breakdown

Seconds per scalac phase for the N=50 project. Macro / mirror derivation and inline expansion land in typer and inlining — their sum is the derivation cost.

Caveats

• Single machine, single JDK. Absolute numbers do not transfer between machines; the ratios between libraries are the portable part.
• String substrate. mcodec's public API (Json.read / Json.write) is String-only, so the whole matrix uses String. This understates the byte-native libraries (jsoniter-scala, borer), which are built for Array[Byte] and pay a UTF-8 conversion here.
• Scala 3.9.0 (Scala Next). mcodec pins 3.9.0 via Made 0.6.0. Every competitor is published for 3.3 LTS and consumed here via TASTy forward-compatibility; a native 3.9.0 build of each might differ slightly.
• Default configuration. Each library uses its out-of-the-box derivation with no hand-tuning (no jsoniter CodecMakerConfig beyond recursion, no circe Configuration, etc.). A tuned setup can move the numbers.
• play-json covers only the three product models — its Scala 3 Json.format macro does not derive sealed hierarchies.
• GenCodec is Scala 2.13 — a different compiler and standard library. Same-JVM runtime throughput is a fair comparison; its compile-time row is a cross-ecosystem reference (Scala 2 macro vs. Scala 3 inline), flagged as such and not stacked in the phase chart.
• JIT warm-up is handled by JMH forking + warmup iterations; the compile sweep deliberately does the opposite (cold JVM every run).

Environment

Committed run:

date:      2026-09-04
cpu:       Apple M4 Pro
ram:       48 GB
os:        macOS 26.6.1 (25G76)
jdk:       OpenJDK 26.0.1 (Homebrew)
scala:     3.9.0
scala-cli: 1.15.0
jmh:       1.37   (-f 3 -wi 5 -i 10)

A laptop under normal desktop load — fine for ranking, not for absolute numbers. Re-run ./mill benchmark.runAll on a quiet dedicated machine for a citable dataset.

Reproducing

# everything (~1h): JMH matrix + compile sweep + report + charts
./mill benchmark.runAll

# quick sanity pass: 1 fork, fewer sizes (a few minutes)
./mill benchmark.runAll --mode quick

# just one slice
./mill benchmark.runJmh 'CompanyBench.*' -p lib=mcodec,circe
./mill benchmark.gencodec.runJmh 'CompanyBench.*'   # GenCodec (Scala 2.13)
python3 benchmark/compile/bench_compile.py --libs mcodec,circe --sizes 0,25,50