bench: add repeatable solver measurements

Add an opt-in benchmark probe and JSON runner that separate solve, construction, validation, and rendering time. Record stable search counters, environment metadata, warm-up and repetition policy, timeouts, errors, median spread, and instrumentation overhead.

Compile production solving without counters and interleave counted and plain trials when measuring overhead. Keep heavyweight cases outside the default correctness path while testing counter and report behavior cheaply.

Tests: Debug and Release CTest suites (7 passed each)

Refs: #8
This commit is contained in:
Codex instance
2026-07-30 17:08:05 +01:00
parent ce39d0a4d0
commit 598667b2f3
7 changed files with 555 additions and 1 deletions
+13
View File
@@ -6,6 +6,12 @@ set(CMAKE_CXX_STANDARD 20)
add_executable(partridge_cpp
main.cc)
option(PARTRIDGE_BUILD_BENCHMARKS "Build the heavyweight benchmark probe" OFF)
if(PARTRIDGE_BUILD_BENCHMARKS)
add_executable(partridge_benchmark
benchmarks/benchmark.cc)
endif()
include(CTest)
if(BUILD_TESTING)
@@ -16,4 +22,11 @@ if(BUILD_TESTING)
add_test(NAME rendering COMMAND partridge_tests rendering)
add_test(NAME solver-small COMMAND partridge_tests solver-small)
add_test(NAME solver-completion COMMAND partridge_tests solver-completion)
add_test(NAME search-counters COMMAND partridge_tests search-counters)
find_package(Python3 COMPONENTS Interpreter)
if(Python3_Interpreter_FOUND)
add_test(NAME benchmark-format
COMMAND "${Python3_EXECUTABLE}" "${CMAKE_CURRENT_SOURCE_DIR}/benchmarks/run.py"
--self-test)
endif()
endif()