BugZoo: ignored labels are a reduction contract #
PyTorch issue #75181 reported CrossEntropyLoss(ignore_index=...) returning nan for an all-
ignored target case:
https://github.com/pytorch/pytorch/issues/75181
Represent ignored labels by explicit zero contributions and choose the empty-reduction policy.
A per-example loss contributes exactly when its label is active.
Instances For
Ignored labels contribute no scalar loss.
Active labels contribute their ordinary scalar loss.
One explicit empty-reduction policy: divide by an epsilon-shifted active count.
The type of activeCount does not enforce a nonnegative integer count, and Context alone gives
no positivity law for epsilon. Finiteness therefore depends on the chosen scalar instance and
valid inputs; the theorem below only unfolds the chosen formula.
Instances For
The denominator policy for safeMaskedMean is visible in the definition.