Loss #
TorchLean loss helpers in the style of torch.nn.functional.
These helpers keep training loops close to the familiar torch.nn.functional style:
loss = Loss.mse yhat y
loss = Loss.mse yhat y (reduction := .sum)
They use the shared operation interface, so the eager tape and typed SSA/DAG recorder both work.
PyTorch references #
torch.nn.functional(losses overview): https://pytorch.org/docs/stable/nn.functional.htmlmse_loss: https://pytorch.org/docs/stable/generated/torch.nn.functional.mse_loss.htmlcross_entropy: https://pytorch.org/docs/stable/generated/torch.nn.functional.cross_entropy.htmlnll_loss: https://pytorch.org/docs/stable/generated/torch.nn.functional.nll_loss.htmlbinary_cross_entropy_with_logits: https://pytorch.org/docs/stable/generated/torch.nn.functional.binary_cross_entropy_with_logits.html
Reduction mode for losses that start as elementwise tensors.
PyTorch analogy: reduction="mean" or reduction="sum".
Instances For
Reduce an elementwise loss tensor to a scalar according to reduction.
This is the common final step for losses like MSE and cross-entropy.
Instances For
Mean squared error (MSE) loss between predictions and targets.
This is backend-generic and supports both mean and sum reduction.
Instances For
Weighted mean-squared error.
This returns sum (weights * (prediction - target)^2) without implicit normalization. Weights
whose sum is one therefore define a weighted mean. Zero weights exclude finite coordinate losses;
they do not suppress NaN or infinity, because the loss is evaluated before multiplication.
Instances For
Negative log-likelihood (one-hot targets), assuming inputs are log-probabilities.
This raw multiplication cannot recover a weighted term after a log-probability has overflowed.
Use oneHotCrossEntropy when the original logits are available.
Instances For
Cross-entropy on logits with one-hot targets. Eager and typed-graph backends keep the original
logits inside the loss, so a zero target need not multiply an overflowing log-probability.
The same weighted formula accepts soft targets; both inputs retain their gradients.
reduction selects the sum or mean over samples, without normalizing target weights.
For integer class labels, use crossEntropy, whose
Fin classes target tensor records the class bound in its element type.
Instances For
Cross-entropy from a fixed teacher's logits along axis.
Use this rather than first rounding the teacher's predictions to probabilities: the loss keeps
logit differences available for a cancellation-resistant student gradient. The teacher is detached
in both forward and reverse mode. mean averages class slices, not individual class entries.
Eager CPU, LibTorch CUDA, and typed graphs implement this loss; canonical IR export is not provided.
Instances For
Unreduced indexed negative log-likelihood along an arbitrary class axis.
The shape contract is:
logProbshas shapeprefix ++ [classes] ++ suffix;classAxis = rank prefixidentifies theclassesdimension;targethas shapeprefix ++ suffixand element typeFin classes;- the result has the same shape as
target, with the class axis removed.
Each output coordinate is -logProbs[..., target[...], ...]. In particular, suffix coordinates
remain aligned with their corresponding labels. Invalid class labels are unrepresentable.
Instances For
Indexed negative log-likelihood along an arbitrary class axis.
The logits, labels, and class-axis contract are those of nllUnreduced. The requested reduction is
applied over every coordinate of the class-erased output shape.
Instances For
Weighted indexed negative log-likelihood along an arbitrary class axis.
The logits, labels, weights, and output coordinates obey the nllUnreduced shape contract. This
returns sum (weights * losses) without implicit normalization; normalized weights therefore give
a weighted mean. Zero weights suppress finite coordinate losses only: 0 * inf and 0 * NaN
remain NaN. This function does not skip evaluation of zero-weight coordinates.
Instances For
Indexed cross-entropy along an arbitrary class axis.
This applies log-softmax along classAxis and then uses nll. Shapes follow the
prefix ++ [classes] ++ suffix contract documented on nllUnreduced.
Instances For
Weighted indexed cross-entropy along an arbitrary class axis.
This applies log-softmax along classAxis, multiplies each class-erased loss by the corresponding
weight, and sums without implicit normalization. Shapes follow the contract on nllUnreduced.
Instances For
Binary cross-entropy with logits (elementwise), using the stable identity:
BCEWithLogits(x,y) = y * softplus(-x) + (1-y) * softplus(x).
Targets are expected in [0,1] (typically 0/1), same shape as logits.