Activations over Backend References

Elementwise nonlinearities and final-axis softmax through Ops.

def Runtime.Autograd.Torch.safeLog {m : Type → Type} {α : Type} [TorchLean.Storage α] [Context α] [Ops m α] {s : Spec.Shape} (x : Ref s) (ε : α := Context.defaultEpsilon) : m (Ref s)

Apply log(softplus(x) + epsilon) elementwise.

The derivative is sigmoid(x) / (softplus(x) + epsilon), with epsilon held fixed. Over the reals, positive epsilon keeps the argument positive. Executable arithmetic needs epsilon to remain positive in its selected representation; this does not make nonfinite inputs finite.