bench: add repeatable solver measurements #20
+16
-4
@@ -66,10 +66,22 @@ in #7 and completion check fix in #14. The earlier Apple M1 Release results in
|
|||||||
elapsed for order 9; they predate the structured runner and do not contain
|
elapsed for order 9; they predate the structured runner and do not contain
|
||||||
search counters.
|
search counters.
|
||||||
|
|
||||||
A clean structured baseline will be recorded here after the benchmark
|
The first clean structured baseline used commit `598667b`, Apple Clang 21.0.0
|
||||||
framework itself is committed. Order 9 may use its documented historical
|
with `-O3 -DNDEBUG`, Apple arm64, one worker, one warm-up, and three measured
|
||||||
result initially because its current runtime is several minutes; the default
|
repetitions. The runner reported a clean working tree:
|
||||||
suite includes it with a per-run timeout.
|
|
||||||
|
| Order | Result | Counted solve median (range) | Nodes | Placements | Backtracks |
|
||||||
|
| --- | --- | --- | ---: | ---: | ---: |
|
||||||
|
| 7 | infeasible | 3.453 s (3.444–3.455 s) | 110,483,315 | 110,483,314 | 110,483,314 |
|
||||||
|
| 8 | solution | 1.817 s (1.814–1.817 s) | 60,485,176 | 60,485,176 | 60,485,140 |
|
||||||
|
|
||||||
|
Counts were stable across repetitions. Interleaved counter-free medians were
|
||||||
|
3.435 seconds for order 7 and 1.805 seconds for order 8, giving counted
|
||||||
|
overheads of 0.53% and 0.64% respectively. Construction time was zero; median
|
||||||
|
independent validation and rendering times were each below 0.02 milliseconds.
|
||||||
|
|
||||||
|
Order 9 was not rerun for this initial baseline because its documented runtime
|
||||||
|
is several minutes. The default suite includes it with a per-run timeout.
|
||||||
|
|
||||||
New optimization issues should quote the exact JSON
|
New optimization issues should quote the exact JSON
|
||||||
environment, policy, median/spread, stable counters, and counted overhead from
|
environment, policy, median/spread, stable counters, and counted overhead from
|
||||||
|
|||||||
Reference in New Issue
Block a user