4.6. Collect defensible samples🔗

A p95 inside one report describes calls in that process. It is not a confidence interval over independent runs. For a change that matters:

1. pin the Lean toolchain and Lake manifests;

2. use the same build settings and machine;

3. fix workload, seed, backend, dtype, device, and synchronization policy;

4. isolate warmup from active samples;

5. collect several independent baseline and candidate runs;

6. inspect the distribution before setting a tolerance;

7. retain traces and summaries for failed comparisons.

Never share a baseline between an unsynchronized GPU launch span and a synchronized completion span. They measure different intervals.