TorchLean API

NN.Examples.BugZoo.BatchInvariance

BugZoo: batch-invariance contracts #

Serving systems often batch unrelated requests together. Recent systems discussions and inference engine bug studies point out that dynamic batching, kernel selection, and reduction order can make outputs depend on who else happened to be in the batch.

Reference:

TorchLean cannot prove arbitrary CUDA kernels batch-invariant unless the kernel implementation is also connected to the spec. This file records the semantic target: if a model is lifted across the batch axis by applying the same function independently to every row, then selecting one row of the batched result is exactly the same as evaluating that row alone.

Two requests, each with three features.

Instances For

    Square each row independently using the public leading-axis map.

    Instances For