4.4. Match the whole key
Comparisons match name, phase, activity, backend, dtype, device, and module. A row found only in the
baseline is missing; one found only in the candidate is new.
Those categories are reported without failing by default because instrumentation can change
deliberately. Use --fail-on-new or --fail-on-missing when the key set is part of the benchmark
contract.
The result stores the baseline and candidate values, signed delta, basis-point change, call counts, and one of:
-
improvement; -
within_tolerance; -
regression.