bench: add repeatable solver measurements #20
+16
-4
@@ -66,10 +66,22 @@ in #7 and completion check fix in #14. The earlier Apple M1 Release results in
|
||||
elapsed for order 9; they predate the structured runner and do not contain
|
||||
search counters.
|
||||
|
||||
A clean structured baseline will be recorded here after the benchmark
|
||||
framework itself is committed. Order 9 may use its documented historical
|
||||
result initially because its current runtime is several minutes; the default
|
||||
suite includes it with a per-run timeout.
|
||||
The first clean structured baseline used commit `598667b`, Apple Clang 21.0.0
|
||||
with `-O3 -DNDEBUG`, Apple arm64, one worker, one warm-up, and three measured
|
||||
repetitions. The runner reported a clean working tree:
|
||||
|
||||
| Order | Result | Counted solve median (range) | Nodes | Placements | Backtracks |
|
||||
| --- | --- | --- | ---: | ---: | ---: |
|
||||
| 7 | infeasible | 3.453 s (3.444–3.455 s) | 110,483,315 | 110,483,314 | 110,483,314 |
|
||||
| 8 | solution | 1.817 s (1.814–1.817 s) | 60,485,176 | 60,485,176 | 60,485,140 |
|
||||
|
||||
Counts were stable across repetitions. Interleaved counter-free medians were
|
||||
3.435 seconds for order 7 and 1.805 seconds for order 8, giving counted
|
||||
overheads of 0.53% and 0.64% respectively. Construction time was zero; median
|
||||
independent validation and rendering times were each below 0.02 milliseconds.
|
||||
|
||||
Order 9 was not rerun for this initial baseline because its documented runtime
|
||||
is several minutes. The default suite includes it with a per-run timeout.
|
||||
|
||||
New optimization issues should quote the exact JSON
|
||||
environment, policy, median/spread, stable counters, and counted overhead from
|
||||
|
||||
Reference in New Issue
Block a user