bench: add repeatable solver measurements #20

Merged
mcp merged 2 commits from codex/issue-8-benchmarks into main 2026-07-30 17:12:29 +01:00
2 Commits
Author SHA1 Message Date
Codex instance ddf07e730a bench: record clean solver baseline
Record reproducible order-7 and order-8 measurements from the committed benchmark framework with a clean working tree. Include stable work counts, solve-time spread, validation and rendering cost, and interleaved instrumentation overhead.

Order 9 remains supported by the runner but retains its historical timing because each current run takes several minutes.

Benchmark: Apple arm64 Release, one warm-up, three repetitions

Refs: #8
2026-07-30 17:10:01 +01:00
Codex instance 598667b2f3 bench: add repeatable solver measurements
Add an opt-in benchmark probe and JSON runner that separate solve, construction, validation, and rendering time. Record stable search counters, environment metadata, warm-up and repetition policy, timeouts, errors, median spread, and instrumentation overhead.

Compile production solving without counters and interleave counted and plain trials when measuring overhead. Keep heavyweight cases outside the default correctness path while testing counter and report behavior cheaply.

Tests: Debug and Release CTest suites (7 passed each)

Refs: #8
2026-07-30 17:08:05 +01:00